Compare commits

..

11 Commits

Author SHA1 Message Date
aks07
cbe3107866 feat: filter fields with no name in field selector 2026-08-11 17:35:39 +05:30
Ashwin Bhatkal
13f2ba7d34 fix(dashboard-v2): stop a time-range change resetting a dynamic variable's selection to ALL (#12416)
## Summary

On a V2 dashboard, picking values in a multi-select variable and then
changing the time range could silently switch the variable to **ALL** —
widening every panel to all values without the user touching the
variable. A value *typed into* a variable was dropped on any refetch for
the same underlying reason.

The post-fetch reconcile compares a selection against freshly-fetched
options and could not tell *why* those options changed: "the user has
nothing selected yet" and "the user's selection was just invalidated by
a refetch" arrived as the same input, and both resolved to the
variable's default — ALL for an ALL-enabled multi-select. A time-range
change refetches every variable, so any window without data for the
selected value hit that path.

Two guarantees now, each with its own mechanism:

| Guarantee | Mechanism |
| --- | --- |
| A refetch nothing else caused (time range, reload) never re-defaults a
selection | The fetch engine tags each cycle with why it was enqueued;
only a value cascade may re-default |
| A typed-in value survives every refetch, whatever caused it | The
selection records which entries were typed, judged at pick time against
the options then offered |

A parent variable's value changing still re-scopes its children — that
behaviour is unchanged and intended. Single-select variables have
preserved a non-empty value since #12178; this brings multi-select in
line, which is the half that was left untouched then.

Also fixed, same family: opening and closing the dropdown without
touching it promoted an explicit pick into a standing ALL whenever the
current window offered only the selected values, and rewrote a dynamic
ALL's `__all__` into concrete values.

Closes https://github.com/SigNoz/pulse-pod/issues/207

## Commits

1. `refactor` — record why each variable fetch cycle was enqueued (full
cycle vs value cascade)
2. `fix` — keep a variable's selection across a time-range refetch
3. `fix` — keep typed-in variable values through every refetch; ALL now
means exactly the option set
4. `fix` — a no-op close of the variable list commits nothing; commit
rule extracted out of the component

Each commit typechecks on its own.

## Test plan

- [x] `jest src/pages/DashboardPageV2` — 136 suites / 1073 tests pass,
including 20 added: the reconcile split by cycle reason, the cycle
tagging in the store, a time-range change tagging every variable as a
full cycle, the typed-value rules, and the commit resolver
- [x] `tsgo --noEmit` clean, at every commit
- [x] `oxlint` and `oxfmt --check` clean on the changed files
- [x] Manual: multi-select variable, pick one value, switch to a window
with no data for it → selection holds, panels show no data rather than
everything
- [x] Manual: type a custom value into a variable, change the time range
and switch a sibling variable → the typed value stays selected
- [x] Manual: namespace → pod pair, change namespace → pod values still
re-scope

## Notes for reviewers

- `customValues` is new on the runtime selection and is persisted with
it. It never reaches the wire or a shared link: `buildVariablesPayload`
and the share-URL builder both project `value` / the `__all__` sentinel
explicitly.
- A selection seeded from a `?variables=` share link carries no
typed-value marker — that URL format stores `name → value` only, so a
typed value from a link is indistinguishable from a fetched one and a
cascade can still drop it.
- The pill can still *read* ALL when the current option list happens to
be a subset of the selection: `CustomMultiSelect` infers that from
`options ⊆ value`, and V1 depends on the inference. It is display-only
now and self-corrects as the window widens; making it exact needs an
explicit prop on the shared control.
2026-08-11 10:23:45 +00:00
Naman Verma
848046de91 fix: allow deletion of legacy dashboards via delete api (#12500)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

If a dashboard failed to migrate to the new schema, currently the delete
API does not delete them. This PR changes it to be able to delete those
un-migrated dashboards as well.

#### Issues closed by this PR

Closes https://github.com/SigNoz/signoz/issues/12390
2026-08-11 04:26:41 +00:00
Swapnil Nakade
68e61af0be test: adding tests for covering GCP integrations across APIs (#12501)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
Added missing tests for covering GCP integrations API, which initially
was just covering AWS

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
https://github.com/SigNoz/platform-pod/issues/2899
2026-08-10 21:49:23 +00:00
Tushar Vats
84780acee1 test(integration): drop the 10k filter-expression fuzz test (#12505)
#### Description

- Removes `test_filter_expressions_no_server_error`, which fires all
9,999 lines of `filter_expressions_10000.txt` at the logs endpoint, one
request at a time. It dominates a local integration run, enough that the
habit is to comment it out first.
- It asserted almost nothing: both `200` and `400` pass, so all it
checked was that the server didn't crash.
- Its corpus has no `[*]` array paths, so it missed `body.<path>[*] IN
[...]` returning a 500 — exactly the failure it exists to catch. That
one is fixed in #12504, with a targeted case that asserts the rows
returned.
- Deletes the corpus file too; nothing else reads it.
`test_not_filter_expression` is untouched and its 11 cases still pass.

#### Additional Information

This does give up cheap crash-fuzzing breadth in CI, where runtime
matters less than it does locally. If that breadth is worth keeping, the
alternative is a much smaller curated corpus that includes the shapes
this one misses — happy to do that instead.
2026-08-10 19:52:39 +00:00
Gaurav Tewari
c70b2be4d5 feat(frontend): save recent searches from summary and infra detail pages (#12054)
## Pull Request

---

### 📄 Summary

The "Recent searches" dropdown (introduced in #11523) is built into the
`QuerySearch` editor and reads from per-signal localStorage buckets, but
entries were only ever **saved** by the QueryBuilder provider's
`handleRunQuery` — a path only the Logs/Traces/Metrics explorers go
through. Every other page embedding `QuerySearch` runs queries through
local handlers, so searches run there displayed explorer recents but
were never captured themselves.

This PR closes that gap for:

| Route | Surface | Save triggers |
|---|---|---|
| `/metrics-explorer/summary` | Metrics Summary search bar | Run button,
Cmd+Enter |
| `/infrastructure-monitoring/hosts` | Host details drawer → Logs /
Traces tabs | Run button, Cmd+Enter |
| `/infrastructure-monitoring/kubernetes` | Entity details drawer → Logs
/ Traces / Events tabs | Run button, Cmd+Enter |

**Approach:** rather than fabricating a fake composite query on each
page, a new expression-level helper
`saveRecentQueryByExpression(dataSource, expression, source?)` is added
to `lib/recentQueries`. It owns the shared policy (trim →
`validateQuery` → signal check → save), and the existing composite-level
`saveRecentQuery` now delegates to it, so validation rules live in
exactly one place. Saves land in the same per-signal buckets the
dropdown already reads, so recents are shared with the explorers both
ways.

Deliberate choices:
- **Infra entity tabs** save the user-typed expression only (not the
combined entity-scoped one), matching what the recents dropdown inserts
back into the editor.
- Expressions are now stored trimmed; dedup was already
trim-insensitive, so this only cleans up display labels.

#### Screenshots / Screen Recordings (if applicable)



https://github.com/user-attachments/assets/852c6273-c797-4281-ac8c-be57eedf5d77


#### Issues closed by this PR

Closes
https://github.com/orgs/SigNoz/projects/39/views/11?filterQuery=assignee%3Atewarig&pane=issue&itemId=210386109&issue=SigNoz%7Cengineering-pod%7C5650
---

###  Change Type
_Select all that apply_

- [ ]  Feature
- [x] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only

---

### 🐛 Bug Context

---

### 🧪 Testing Strategy

- Tests: existing `lib/recentQueries` unit tests still pass (37/37) —
`saveRecentQuery` now routes through the new helper, so its behaviour
stays covered.
- Manual verification: `tsgo --noEmit`, `oxlint` (no new warnings), and
production build all pass.
- Edge cases covered: invalid/partial expressions are rejected by
`validateQuery` before saving; empty/whitespace-only expressions and
unsupported data sources are no-ops.

---

### ⚠️ Risk & Impact Assessment

---

### 📝 Changelog

| Field | Value |
|------|-------|
| Deployment Type | Cloud / OSS / Enterprise |
| Change Type | Bug Fix |
| Description | Searches run on Metrics Summary and Infra Monitoring
entity detail tabs (Logs/Traces/Events) now appear in the "Recent
searches" dropdown, shared with the explorers. |

---

### 📋 Checklist
- [x] Tests added or explicitly not required
- [x] Manually tested
- [x] Breaking changes documented
- [x] Backward compatibility considered

---

## 👀 Notes for Reviewers


---

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-08-10 18:45:18 +00:00
Vikrant Gupta
cda2955b93 feat(members): assign member roles through the user_roles API (#12498)
#### Description

- Member role assignment used the deprecated `POST` / `DELETE
/api/v2/users/{id}/roles`. It now uses `POST` and `DELETE
/api/v2/user_roles`, which had no consumers until now.
- Roles are read from `useGetUser` instead of `useGetRolesByUserID`,
because the delete route is keyed by the `user_role` join row and only
that response carries its id.

#### Issues closed by this PR

Closes SigNoz/platform-pod#2918

#### Screenshots / Screen Recordings



https://github.com/user-attachments/assets/4c0790fb-a93e-4c37-8608-51fe03d4d962


#### Additional Information

- `EditMemberDrawer` already issues the same `useGetUser` query, so the
two share one request and the component itself needed no change.
2026-08-10 23:43:31 +05:30
Aditya Singh
1a293652f7 fix(sentry): drop benign aborted/cancelled requests from error reporting (#12495)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

This PR filters network request aborts via fetch and axios in beforeSend
so they stop surfacing as Sentry issues.
- axios: ECONNABORTED ("Request aborted"), ERR_CANCELED
- native fetch: AbortError

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

<!--If applicable, include screenshots or screen recordings that clearly
show the behavior before the change and the result after the change. -->
#### Screenshots / Screen Recordings

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information

<!--Please delete paragraphs that you did not use before submitting.-->
2026-08-10 16:13:14 +00:00
Vikrant Gupta
0f3fb71067 feat(reset-password): use the v2 endpoint on the reset password page (#12492)
#### Description

- The reset password page still called the deprecated `POST
/api/v1/resetPassword`. It now uses the generated `useResetPassword`
hook, which targets `POST /api/v2/factor_password/reset`.
- Deletes the hand-written v1 client and its types; nothing else
referenced them.

#### Issues closed by this PR

Contributes to SigNoz/platform-pod#2667

#### Screenshots / Screen Recordings



https://github.com/user-attachments/assets/9804f619-5928-4c72-83a2-2aae16855e7f


#### Additional Information

- Manual `loading` and `errorMessage` state give way to the hook's
`isLoading` and `convertToApiError`, matching how `ForgotPassword`
consumes its generated hook.
- Once this merges, `POST /api/v1/resetPassword` has no callers left in
the product.
2026-08-10 15:17:06 +00:00
Aditya Singh
b4f5b3eddf fix(query-builder): normalise doc length in codemirror fixing Selection points outside of document error (#12496)
Setting the query expression to a value containing CRLF line breaks
crashed the search bar with "RangeError: Selection points outside of
document".

CodeMirror normalises CRLF to LF when building a change, so the
resulting document is shorter than the raw string. The selection anchor
used value.length (pre-normalisation), which pointed past the end of the
document.

Build the ChangeSet first and anchor the selection at changes.newLength,
the actual post-change document length. Adds a regression test.


<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
To fix the above mentioned problem, we now switch the cursor position
from `value.length` (which is not yet normalized by CodeMirror) to
`changes.newLength`, which is the normalized length.
Added test case


<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/engineering-pod/issues/5869

<!--If applicable, include screenshots or screen recordings that clearly
show the behavior before the change and the result after the change. -->
#### Screenshots / Screen Recordings

Before


https://github.com/user-attachments/assets/ec7a3182-177f-4545-9bae-83ee0c3a61db

After


https://github.com/user-attachments/assets/06350698-1960-47c2-b65e-81ea2d10b15d
2026-08-10 14:14:28 +00:00
Srikanth Chekuri
b9f4fcd681 chore(telemetrytypes): introduce LogicalField (#12499)
Some checks failed
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
Possible options

1. The compatibility keys maps (the approach already in the code).

`backward_compat_keys.go` makes an alias key at metadata time. We
rejected this option because of evidence. The alias key resolves, but it
reads the wrong data. It prepares to `attributes_string['<alias>']`, and
that physical key does not hold the data.

2. The flat multi-key.

`GetKeys` returns multiple keys in order, and the downstream code uses
the list. The option fails on semantics. It removes one piece of
information that the downstream must have. The downstream must know the
difference between two cases:

- Two keys are the same field with two spellings so we can merge them
into one expression.
- Two keys are different fields with the same name. The condition
builder must make one condition for each key. The operator connects the
conditions.

Three failures show the problem:

- Negative operators connect with OR across the keys. A row that has
only one spelling then always matches. Example: `env != 'prod'` matches
each row that does not have one of the two keys.
- A row that has both spellings with different values gets no clear
result.
- A value position (group-by, select) needs exactly one expression for
one field. A flat list cannot point to that expression.

The information must live somewhere.

3. Annotations on `TelemetryFieldKey`

Maintain the `SemconvMembers` and `SemconvMaterializedColumns` fields on
the keys. The information is the same as in option 4. But it's awkward
because "these N keys are one family, in this order" lives in N copies,
one copy on each key.

4. introduce `LogicalField`

The information is the same as in option 3, but the structure holds it:

- The slice is the ambiguity.
- The group is the family.
- The member order is the precedence. The code sorts the members one
time, by family rank, at construction.
- The members point to the metadata entries. The code copies nothing and
changes nothing.
- The identity (signal, context, data type) is on the group. A merge
across contexts or data types is not possible. The design does not avoid
that merge; the design cannot express it.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 13:40:37 +00:00
52 changed files with 1472 additions and 13036 deletions

View File

@@ -376,7 +376,19 @@ function App(): JSX.Element {
tracesSampleRate: 0, // Ref: https://github.com/SigNoz/platform-pod/issues/2393#issuecomment-4603658055
replaysSessionSampleRate: 0.1, // This sets the sample rate at 10%. You may want to change it to 100% while in development and then sample at a lower rate in production.
replaysOnErrorSampleRate: 1.0, // If you're not already sampling the entire session, change the sample rate to 100% when sampling sessions where errors occur.
beforeSend(event) {
beforeSend(event, hint) {
const error = hint?.originalException as
| { name?: string; code?: string | number }
| undefined;
// Ignore benign aborted/cancelled requests (axios + fetch).
if (error?.code === 'ERR_CANCELED' || error?.code === 'ECONNABORTED') {
return null;
}
if (error?.name === 'AbortError') {
return null;
}
// Drop the event if its level is 'warning' or 'info'
if (event.level === 'warning' || event.level === 'info') {
return null;

View File

@@ -1,31 +0,0 @@
import axios from 'api';
import { ErrorResponseHandlerV2 } from 'api/ErrorResponseHandlerV2';
import { AxiosError } from 'axios';
import { ErrorV2Resp, SuccessResponseV2 } from 'types/api';
import { PayloadProps, Props } from 'types/api/user/resetPassword';
/**
* @deprecated Use the generated `useResetPassword` hook (or `resetPassword` fetcher) from
* `api/generated/services/users` instead. This hand-written client targets the
* same endpoint and will be removed once call sites migrate.
*
* Part of https://github.com/SigNoz/engineering-pod/issues/5289, add a comment or update when removing this method.
*/
const resetPassword = async (
props: Props,
): Promise<SuccessResponseV2<PayloadProps>> => {
try {
const response = await axios.post<PayloadProps>(`/resetPassword`, {
...props,
});
return {
httpStatusCode: response.status,
data: response.data,
};
} catch (error) {
ErrorResponseHandlerV2(error as AxiosError<ErrorV2Resp>);
}
};
export default resetPassword;

View File

@@ -5,10 +5,9 @@ import {
useCreateResetPasswordToken,
useDeleteUser,
useGetResetPasswordToken,
useGetRolesByUserID,
useCreateUserRole,
useDeleteUserRole,
useGetUser,
useRemoveUserRoleByUserIDAndRoleID,
useSetRoleByUserID,
useUpdateMyUserV2,
useUpdateUser,
} from 'api/generated/services/users';
@@ -25,15 +24,14 @@ import EditMemberDrawer, { EditMemberDrawerProps } from '../EditMemberDrawer';
jest.mock('api/generated/services/users', () => ({
useDeleteUser: jest.fn(),
useGetUser: jest.fn(),
useGetRolesByUserID: jest.fn(),
useRemoveUserRoleByUserIDAndRoleID: jest.fn(),
useDeleteUserRole: jest.fn(),
useUpdateUser: jest.fn(),
useUpdateMyUserV2: jest.fn(),
useSetRoleByUserID: jest.fn(),
useCreateUserRole: jest.fn(),
useGetResetPasswordToken: jest.fn(),
useCreateResetPasswordToken: jest.fn(),
getGetRolesByUserIDQueryKey: ({ id }: { id: string }): string[] => [
`/api/v2/users/${id}/roles`,
getGetUserQueryKey: ({ id }: { id: string }): string[] => [
`/api/v2/users/${id}`,
],
}));
@@ -194,11 +192,7 @@ describe('EditMemberDrawer', () => {
isLoading: false,
refetch: jest.fn(),
});
(useGetRolesByUserID as jest.Mock).mockReturnValue({
data: { data: [managedRoles[0]] },
isLoading: false,
});
(useRemoveUserRoleByUserIDAndRoleID as jest.Mock).mockReturnValue({
(useDeleteUserRole as jest.Mock).mockReturnValue({
mutateAsync: mockRemoveMutateAsync.mockResolvedValue({}),
isLoading: false,
});
@@ -210,7 +204,7 @@ describe('EditMemberDrawer', () => {
mutateAsync: jest.fn().mockResolvedValue({}),
isLoading: false,
});
(useSetRoleByUserID as jest.Mock).mockReturnValue({
(useCreateUserRole as jest.Mock).mockReturnValue({
mutateAsync: jest.fn().mockResolvedValue({}),
isLoading: false,
});
@@ -312,12 +306,12 @@ describe('EditMemberDrawer', () => {
expect(onClose).not.toHaveBeenCalled();
});
it('adding a new role calls setRole without removing existing ones', async () => {
it('adding a new role creates a user role without removing existing ones', async () => {
const onComplete = jest.fn();
const user = userEvent.setup({ pointerEventsCheck: 0 });
const mockSet = jest.fn().mockResolvedValue({});
(useSetRoleByUserID as jest.Mock).mockReturnValue({
(useCreateUserRole as jest.Mock).mockReturnValue({
mutateAsync: mockSet,
isLoading: false,
});
@@ -334,15 +328,14 @@ describe('EditMemberDrawer', () => {
await waitFor(() => {
expect(mockSet).toHaveBeenCalledWith({
pathParams: { id: 'user-1' },
data: { name: 'signoz-editor' },
data: { userId: 'user-1', roleId: managedRoles[1].id },
});
expect(mockRemoveMutateAsync).not.toHaveBeenCalled();
expect(onComplete).toHaveBeenCalled();
});
});
it('deselecting a role calls removeRole with the role id', async () => {
it('deselecting a role deletes the user role by its assignment id', async () => {
const onComplete = jest.fn();
const user = userEvent.setup({ pointerEventsCheck: 0 });
@@ -361,7 +354,7 @@ describe('EditMemberDrawer', () => {
await waitFor(() => {
expect(mockRemoveMutateAsync).toHaveBeenCalledWith({
pathParams: { id: 'user-1', roleId: managedRoles[0].id },
pathParams: { id: 'ur-1' },
});
expect(onComplete).toHaveBeenCalled();
});

View File

@@ -183,15 +183,14 @@ function QuerySearch({
isProgrammaticChangeRef.current = true;
}
const changes = view.state.changes({
from: 0,
to: currentValue.length,
insert: value,
});
view.dispatch({
changes: {
from: 0,
to: currentValue.length,
insert: value,
},
selection: {
anchor: value.length,
},
changes,
selection: { anchor: changes.newLength },
});
},
[],

View File

@@ -301,6 +301,66 @@ describe('QuerySearch (Integration with Real CodeMirror)', () => {
dispatchSpy.mockRestore();
});
it('does not crash when the expression contains CRLF line breaks (issue #5869)', async () => {
const dispatchSpy = jest.spyOn(EditorView.prototype, 'dispatch');
const onChange = jest.fn() as jest.MockedFunction<(v: string) => void>;
const initialExpression = "service.name = 'frontend'";
// Filtering on a multi-line log value (CRLF) used to throw
// "RangeError: Selection points outside of document".
const crlfExpression = "body CONTAINS 'line1\r\nline2\r\nline3'";
const baseQueryData = {
...initialQueriesMap.logs.builder.queryData[0],
filter: { expression: initialExpression },
};
const { rerender } = render(
<QuerySearch
onChange={onChange}
queryData={baseQueryData}
dataSource={DataSource.LOGS}
/>,
);
await waitFor(
() => {
const editorContent = document.querySelector(
CM_EDITOR_SELECTOR,
) as HTMLElement;
expect(editorContent.textContent || '').toBe(initialExpression);
},
{ timeout: 3000 },
);
rerender(
<QuerySearch
onChange={onChange}
queryData={{ ...baseQueryData, filter: { expression: crlfExpression } }}
dataSource={DataSource.LOGS}
/>,
);
// The programmatic replace dispatched without throwing, and the selection anchor
// stayed within the CRLF-normalized document (the bug set it past the end).
await waitFor(() => {
const spec = dispatchSpy.mock.calls
.map(
(call) =>
call[0] as {
selection?: { anchor?: number };
changes?: { newLength?: number };
},
)
.find((s) => s?.selection?.anchor != null && s?.changes?.newLength != null);
expect(spec).toBeDefined();
expect(spec?.selection?.anchor).toBeLessThanOrEqual(
spec?.changes?.newLength as number,
);
});
dispatchSpy.mockRestore();
});
it('fetches key suggestions for metrics even without aggregateAttribute.key when showFilterSuggestionsWithoutMetric is true', async () => {
const mockedGetKeys = getKeySuggestions as jest.MockedFunction<
typeof getKeySuggestions

View File

@@ -22,6 +22,7 @@ import { InfraMonitoringEntity } from 'container/InfraMonitoringK8sV2/constants'
import LoadingContainer from 'container/InfraMonitoringK8sV2/LoadingContainer';
import RunQueryBtn from 'container/QueryBuilder/components/RunQueryBtn/RunQueryBtn';
import { ChevronDown, ChevronRight } from '@signozhq/icons';
import { saveRecentQueryByExpression } from 'lib/recentQueries/saveRecentQuery';
import { useQueryState } from 'nuqs';
import { DataSource } from 'types/common/queryBuilder';
import { parseAsJsonNoValidate } from 'utils/nuqsParsers';
@@ -116,6 +117,7 @@ function EntityEventsContent({
: newUserExpression || '',
);
if (validation.isValid) {
saveRecentQueryByExpression(DataSource.LOGS, newUserExpression);
querySearchOnRun(newUserExpression || '');
void logEvent(InfraMonitoringEvents.FilterApplied, {

View File

@@ -29,6 +29,7 @@ import { getOldLogsOperatorFromNew } from 'hooks/logs/useActiveLog';
import useLogDetailHandlers from 'hooks/logs/useLogDetailHandlers';
import useScrollToLog from 'hooks/logs/useScrollToLog';
import { generateFilterQuery } from 'lib/logs/generateFilterQuery';
import { saveRecentQueryByExpression } from 'lib/recentQueries/saveRecentQuery';
import { ILog } from 'types/api/logs/log';
import { DataSource } from 'types/common/queryBuilder';
import { validateQuery } from 'utils/queryValidationUtils';
@@ -132,6 +133,7 @@ function EntityLogsContent({
);
if (validation.isValid) {
saveRecentQueryByExpression(DataSource.LOGS, newUserExpression);
querySearchOnRun(newUserExpression);
void logEvent(InfraMonitoringEvents.FilterApplied, {

View File

@@ -22,6 +22,7 @@ import { InfraMonitoringEntity } from 'container/InfraMonitoringK8sV2/constants'
import RunQueryBtn from 'container/QueryBuilder/components/RunQueryBtn/RunQueryBtn';
import { PER_PAGE_OPTIONS } from 'container/TracesExplorer/ListView/configs';
import { TracesLoading } from 'container/TracesExplorer/TraceLoading/TraceLoading';
import { saveRecentQueryByExpression } from 'lib/recentQueries/saveRecentQuery';
import { useQueryState } from 'nuqs';
import { DataSource } from 'types/common/queryBuilder';
import { parseAsJsonNoValidate } from 'utils/nuqsParsers';
@@ -98,6 +99,7 @@ function EntityTracesContent({
: newUserExpression || '',
);
if (validation.isValid) {
saveRecentQueryByExpression(DataSource.TRACES, newUserExpression);
querySearchOnRun(newUserExpression || '');
void logEvent(InfraMonitoringEvents.FilterApplied, {

View File

@@ -2,6 +2,7 @@ import { useCallback } from 'react';
import QuerySearch from 'components/QueryBuilderV2/QueryV2/QuerySearch/QuerySearch';
import RunQueryBtn from 'container/QueryBuilder/components/RunQueryBtn/RunQueryBtn';
import DateTimeSelectionV2 from 'container/TopNav/DateTimeSelectionV2';
import { saveRecentQueryByExpression } from 'lib/recentQueries/saveRecentQuery';
import { DataSource } from 'types/common/queryBuilder';
import { MetricsSearchProps } from './types';
@@ -23,12 +24,14 @@ function MetricsSearch({
);
const handleStageAndRunQuery = useCallback(() => {
saveRecentQueryByExpression(DataSource.METRICS, currentQueryFilterExpression);
onChange(currentQueryFilterExpression);
onRunQuery?.();
}, [currentQueryFilterExpression, onChange, onRunQuery]);
const handleRunQuery = useCallback(
(expression: string): void => {
saveRecentQueryByExpression(DataSource.METRICS, expression);
setCurrentQueryFilterExpression(expression);
onChange(expression);
},

View File

@@ -387,4 +387,42 @@ describe('useOptionsMenu', () => {
expect(remaining).toHaveLength(seedColumns.length);
});
});
describe('fieldsSelector.value drops legacy columns without a name', () => {
it('excludes entries missing name while keeping valid columns', () => {
(useGetQueryKeySuggestions as jest.Mock).mockReturnValue({
data: { data: { data: { keys: {} } } },
isFetching: false,
});
(usePreferenceContext as jest.Mock).mockReturnValue({
traces: {
preferences: {
columns: [
{ name: 'body', fieldContext: 'log' },
{ key: 'legacy-key-no-name', fieldContext: 'log' },
{ name: 'timestamp', fieldContext: 'log' },
],
formatting: { format: 'table', maxLines: 1, fontSize: 'small' },
},
updateColumns: mockUpdateColumns,
updateFormatting: mockUpdateFormatting,
},
logs: {
preferences: { columns: [], formatting: {} },
updateColumns: mockUpdateColumns,
updateFormatting: mockUpdateFormatting,
},
});
const { result } = renderHook(() =>
useOptionsMenu({
dataSource: DataSource.TRACES,
aggregateOperator: 'count',
}),
);
const fields = result.current.config.fieldsSelector?.value ?? [];
expect(fields.map((f) => f.name)).toStrictEqual(['body', 'timestamp']);
});
});
});

View File

@@ -399,7 +399,7 @@ const useOptionsMenu = ({
onReorder: reorderSelectColumns,
},
fieldsSelector: {
value: preferences?.columns ?? [],
value: preferences?.columns?.filter((item) => has(item, 'name')) ?? [],
onFieldsChange: updateColumns,
},
format: {

View File

@@ -33,7 +33,7 @@ jest.mock('hooks/useNotifications', () => ({
}),
}));
const RESET_PASSWORD_ENDPOINT = '*/resetPassword';
const RESET_PASSWORD_ENDPOINT = '*/api/v2/factor_password/reset';
const mockHistoryPush = history.push as jest.MockedFunction<
typeof history.push

View File

@@ -1,11 +1,12 @@
import { useState } from 'react';
import { useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { useLocation } from 'react-use';
import { Button } from '@signozhq/ui/button';
import { Callout } from '@signozhq/ui/callout';
import { Form, Input as AntdInput } from 'antd';
import { Typography } from '@signozhq/ui/typography';
import resetPasswordApi from 'api/v1/factor_password/resetPassword';
import { convertToApiError } from 'api/ErrorResponseHandlerForGeneratedAPIs';
import { useResetPassword } from 'api/generated/services/users';
import AuthError from 'components/AuthError/AuthError';
import AuthPageContainer from 'components/AuthPageContainer';
import ROUTES from 'constants/routes';
@@ -14,7 +15,6 @@ import { useNotifications } from 'hooks/useNotifications';
import history from 'lib/history';
import { ArrowRight, CircleAlert, KeyRound } from '@signozhq/icons';
import { Label } from 'pages/SignUp/styles';
import APIError from 'types/api/error';
import { FormContainer } from './styles';
@@ -26,40 +26,41 @@ function ResetPassword({ version }: ResetPasswordProps): JSX.Element {
const [confirmPasswordError, setConfirmPasswordError] =
useState<boolean>(false);
const [errorMessage, setErrorMessage] = useState<APIError | null>();
const [isValidPassword, setIsValidPassword] = useState(false);
const [loading, setLoading] = useState(false);
const { t } = useTranslation(['common']);
const { search } = useLocation();
const params = new URLSearchParams(search);
const token = params.get('token');
const { notifications } = useNotifications();
const {
mutate: resetPassword,
isLoading,
error: mutationError,
} = useResetPassword();
const errorMessage = useMemo(
() => convertToApiError(mutationError),
[mutationError],
);
const [form] = Form.useForm<FormValues>();
const handleFormSubmit: () => Promise<void> = async () => {
try {
setLoading(true);
setErrorMessage(null);
const { password } = form.getFieldsValue();
const handleFormSubmit = (): void => {
const { password } = form.getFieldsValue();
await resetPasswordApi({
password,
token: token || '',
});
notifications.success({
message: t('success', {
ns: 'common',
}),
});
history.push(ROUTES.LOGIN);
setLoading(false);
} catch (error) {
setLoading(false);
setErrorMessage(error as APIError);
}
resetPassword(
{ data: { password, token: token || '' } },
{
onSuccess: (): void => {
notifications.success({
message: t('success', {
ns: 'common',
}),
});
history.push(ROUTES.LOGIN);
},
},
);
};
const validatePassword = (): boolean => {
@@ -222,7 +223,7 @@ function ResetPassword({ version }: ResetPasswordProps): JSX.Element {
color="primary"
type="submit"
data-attr="reset-password"
disabled={!isValidPassword || loading}
disabled={!isValidPassword || isLoading}
className="reset-password-submit-button"
suffix={<ArrowRight size={16} />}
>

View File

@@ -1,19 +1,22 @@
import { useCallback, useMemo } from 'react';
import { useQueryClient } from 'react-query';
import type { AuthtypesGettableRoleDTO } from 'api/generated/services/sigNoz.schemas';
import type {
AuthtypesGettableRoleDTO,
AuthtypesUserRoleDTO,
} from 'api/generated/services/sigNoz.schemas';
import {
getGetRolesByUserIDQueryKey,
useGetRolesByUserID,
useRemoveUserRoleByUserIDAndRoleID,
useSetRoleByUserID,
useCreateUserRole,
useDeleteUserRole,
useGetUser,
} from 'api/generated/services/users';
import { retryOn429 } from 'utils/errorUtils';
const enum PromiseStatus {
Fulfilled = 'fulfilled',
Rejected = 'rejected',
}
// Stable identity so the memos below do not recompute on every render.
const EMPTY_USER_ROLES: AuthtypesUserRoleDTO[] = [];
export interface MemberRoleUpdateFailure {
roleName: string;
error: unknown;
@@ -33,31 +36,31 @@ export function useMemberRoleManager(
userId: string,
enabled: boolean,
): UseMemberRoleManagerResult {
const queryClient = useQueryClient();
const { data, isLoading } = useGetRolesByUserID(
const { data, isLoading } = useGetUser(
{ id: userId },
{ query: { enabled: !!userId && enabled } },
);
const userRoles = data?.data?.userRoles ?? EMPTY_USER_ROLES;
const currentRoles = useMemo<AuthtypesGettableRoleDTO[]>(
() => data?.data ?? [],
[data?.data],
() => userRoles.map((userRole) => userRole.role),
[userRoles],
);
const { mutateAsync: setRole } = useSetRoleByUserID({
mutation: { retry: retryOn429 },
});
const { mutateAsync: removeRole } = useRemoveUserRoleByUserIDAndRoleID({
mutation: { retry: retryOn429 },
});
const invalidateRoles = useCallback(
() =>
queryClient.invalidateQueries(getGetRolesByUserIDQueryKey({ id: userId })),
[userId, queryClient],
// DELETE /api/v2/user_roles/{id} is keyed by the user_role join row, not the role.
const assignmentIdByRoleId = useMemo(
() => new Map(userRoles.map((userRole) => [userRole.roleId, userRole.id])),
[userRoles],
);
const { mutateAsync: createUserRole } = useCreateUserRole({
mutation: { retry: retryOn429 },
});
const { mutateAsync: deleteUserRole } = useDeleteUserRole({
mutation: { retry: retryOn429 },
});
const applyDiff = useCallback(
async (
localRoleIds: string[],
@@ -80,30 +83,33 @@ export function useMemberRoleManager(
const allOperations = [
...addedRoles.map((role) => ({
role,
run: (): ReturnType<typeof setRole> =>
setRole({
pathParams: { id: userId },
data: { name: role.name ?? '' },
}),
})),
...removedRoles.map((role) => ({
role,
run: (): ReturnType<typeof removeRole> =>
removeRole({ pathParams: { id: userId, roleId: role.id ?? '' } }),
run: (): ReturnType<typeof createUserRole> =>
createUserRole({ data: { userId, roleId: role.id ?? '' } }),
})),
...removedRoles
.map((role) => ({
role,
assignmentId: assignmentIdByRoleId.get(role.id ?? ''),
}))
.filter(
(
entry,
): entry is {
role: AuthtypesGettableRoleDTO;
assignmentId: string;
} => !!entry.assignmentId,
)
.map(({ role, assignmentId }) => ({
role,
run: (): ReturnType<typeof deleteUserRole> =>
deleteUserRole({ pathParams: { id: assignmentId } }),
})),
];
const results = await Promise.allSettled(
allOperations.map((op) => op.run()),
);
const successCount = results.filter(
(r) => r.status === PromiseStatus.Fulfilled,
).length;
if (successCount > 0) {
await invalidateRoles();
}
const failures: MemberRoleUpdateFailure[] = [];
results.forEach((result, index) => {
if (result.status === PromiseStatus.Rejected) {
@@ -113,7 +119,6 @@ export function useMemberRoleManager(
error: result.reason,
onRetry: async (): Promise<void> => {
await run();
await invalidateRoles();
},
});
}
@@ -121,7 +126,7 @@ export function useMemberRoleManager(
return failures;
},
[userId, currentRoles, setRole, removeRole, invalidateRoles],
[userId, currentRoles, assignmentIdByRoleId, createUserRole, deleteUserRole],
);
return { currentRoles, isLoading, applyDiff };

View File

@@ -19,6 +19,30 @@ type CompositeWithBuilder = {
builder?: { queryData?: IBuilderQuery[] };
};
export function saveRecentQueryByExpression(
dataSource: IBuilderQuery['dataSource'],
expression: string | null | undefined,
source = '',
): void {
const trimmed = expression?.trim();
if (!trimmed) {
return;
}
const validation = validateQuery(trimmed);
if (!validation.isValid) {
return;
}
const signal = toSignal(dataSource);
if (!signal) {
return;
}
store.save({
signal,
source,
filter: { expression: trimmed },
});
}
// Persists each builder query in the composite as a recent entry. Call this
// only from explicit user-driven Run triggers — reacting to stagedQuery or any
// other derived state pollutes recents with navigation/refresh/go-to traffic.
@@ -31,22 +55,10 @@ export function saveRecentQuery(
}
queryData.forEach((q) => {
const expression = q.filter?.expression?.trim();
if (!expression) {
return;
}
const validation = validateQuery(expression);
if (!validation.isValid) {
return;
}
const signal = toSignal(q.dataSource);
if (!signal) {
return;
}
store.save({
signal,
source: q.source ?? '',
filter: q.filter ?? { expression: '' },
});
saveRecentQueryByExpression(
q.dataSource,
q.filter?.expression,
q.source ?? '',
);
});
}

View File

@@ -182,4 +182,56 @@ describe('ValueSelector', () => {
});
});
});
describe('opening and closing without touching the list', () => {
function renderWith(
selection: VariableSelection,
options: string[],
): jest.Mock {
const onChange = jest.fn();
render(
<TooltipProvider>
<ValueSelector
options={options}
variableType="dynamic"
multiSelect
showAllOption
selection={selection}
onChange={onChange}
emptyFallback={{ value: [], allSelected: false }}
testId="variable-select-env"
/>
</TooltipProvider>,
);
return onChange;
}
async function openThenClose(): Promise<void> {
const user = userEvent.setup({ advanceTimers: jest.advanceTimersByTime });
const control = screen.getByTestId('variable-select-env');
await user.click(control.querySelector('input') as HTMLInputElement);
await user.keyboard('{Escape}');
}
it('does not promote a pick that covers every available option to ALL', async () => {
// A narrow time range can leave only the selected value in the list. That is
// still an explicit pick, not "everything, always".
const onChange = renderWith(
{ value: ['checkout-service-prod'], allSelected: false },
['checkout-service-prod'],
);
await openThenClose();
expect(onChange).not.toHaveBeenCalled();
});
it('does not rewrite a dynamic ALL into concrete values', async () => {
const onChange = renderWith({ value: null, allSelected: true }, OPTIONS);
await openThenClose();
expect(onChange).not.toHaveBeenCalled();
});
});
});

View File

@@ -145,6 +145,133 @@ describe('reconcileWithOptions', () => {
),
).toBeNull();
});
describe('preserveSelection (options moved on their own — time range, reload)', () => {
const multi = model({
type: 'DYNAMIC',
multiSelect: true,
showAllOption: true,
dynamicAttribute: 'service.name',
});
it('keeps a multi-select pick the new option list no longer offers', () => {
expect(
reconcileWithOptions(multi, { value: ['frontend'], allSelected: false }, [
'backend',
'cart',
]),
).toStrictEqual({ value: null, allSelected: true });
expect(
reconcileWithOptions(
multi,
{ value: ['frontend'], allSelected: false },
['backend', 'cart'],
{ preserveSelection: true },
),
).toBeNull();
});
it('still materializes ALL, which must track the option list', () => {
expect(
reconcileWithOptions(
model({ type: 'QUERY', multiSelect: true, showAllOption: true }),
{ value: ['a'], allSelected: true },
['a', 'b'],
{ preserveSelection: true },
),
).toStrictEqual({ value: ['a', 'b'], allSelected: true });
});
it('still fills the default when nothing is selected yet', () => {
expect(
reconcileWithOptions(multi, { value: [], allSelected: false }, ['a', 'b'], {
preserveSelection: true,
}),
).toStrictEqual({ value: null, allSelected: true });
});
});
// A typed value is in no option list, so no refetch can invalidate it.
describe('customValues (typed in, never offered by the data)', () => {
const multi = model({
type: 'DYNAMIC',
multiSelect: true,
showAllOption: true,
dynamicAttribute: 'service.name',
});
it('keeps them through a re-scope that drops a fetched value', () => {
expect(
reconcileWithOptions(
multi,
{
value: ['frontend', 'typed-in'],
allSelected: false,
customValues: ['typed-in'],
},
['backend', 'cart'],
),
).toStrictEqual({
value: ['typed-in'],
allSelected: false,
customValues: ['typed-in'],
});
});
it('never re-defaults a selection made only of them', () => {
expect(
reconcileWithOptions(
multi,
{ value: ['typed-in'], allSelected: false, customValues: ['typed-in'] },
['backend', 'cart'],
),
).toBeNull();
});
// An inert marker is not worth a store write + dependent refetch to prune.
it('leaves a stale marker alone when it drops nothing', () => {
expect(
reconcileWithOptions(
multi,
{
value: ['frontend', 'typed-in'],
allSelected: false,
customValues: ['typed-in', 'removed-earlier'],
},
['frontend'],
),
).toBeNull();
});
it('prunes markers for values it does drop', () => {
expect(
reconcileWithOptions(
multi,
{
value: ['stale', 'typed-in'],
allSelected: false,
customValues: ['typed-in'],
},
['frontend'],
),
).toStrictEqual({
value: ['typed-in'],
allSelected: false,
customValues: ['typed-in'],
});
});
it('still drops an unmarked value the list no longer offers', () => {
expect(
reconcileWithOptions(
multi,
{ value: ['frontend', 'stale'], allSelected: false },
['frontend'],
),
).toStrictEqual({ value: ['frontend'], allSelected: false });
});
});
});
describe('configuredDefaultValue', () => {

View File

@@ -0,0 +1,91 @@
import type { VariableSelection } from '../selectionTypes';
import { selectionFromCommittedValues } from '../utils/selectionUtils';
const OPTIONS = ['checkout', 'payments', 'cart'];
const FALLBACK: VariableSelection = { value: null, allSelected: true };
function commit(
values: string[],
overrides: Partial<Parameters<typeof selectionFromCommittedValues>[0]> = {},
): VariableSelection {
return selectionFromCommittedValues({
values,
options: OPTIONS,
showAllOption: true,
emptyFallback: FALLBACK,
...overrides,
});
}
// What a multi-select commit resolves to. The option list is known only here, so this
// is the one place a typed value can be recognised.
describe('selectionFromCommittedValues', () => {
it('marks values the option list did not offer as typed in', () => {
expect(commit(['checkout', 'typed-in'])).toStrictEqual({
value: ['checkout', 'typed-in'],
allSelected: false,
customValues: ['typed-in'],
});
});
it('marks a selection made only of typed-in values', () => {
expect(commit(['a', 'b'])).toStrictEqual({
value: ['a', 'b'],
allSelected: false,
customValues: ['a', 'b'],
});
});
it('records no marker when every pick came from the list', () => {
expect(commit(['checkout', 'cart'])).toStrictEqual({
value: ['checkout', 'cart'],
allSelected: false,
});
});
it('reads a set covering every option as ALL', () => {
expect(commit(OPTIONS)).toStrictEqual({
value: OPTIONS,
allSelected: true,
});
});
// ALL re-materializes to the option set, so recording this as ALL would drop the
// typed value on the next refetch.
it('does not read every option PLUS a typed value as ALL', () => {
expect(commit([...OPTIONS, 'typed-in'])).toStrictEqual({
value: [...OPTIONS, 'typed-in'],
allSelected: false,
customValues: ['typed-in'],
});
});
// Derived from the values + options at commit time, never from the old selection.
it('recomputes the marker: a typed value the data now offers is a normal pick', () => {
expect(
commit(['checkout', 'was-typed'], {
options: [...OPTIONS, 'was-typed'],
}),
).toStrictEqual({ value: ['checkout', 'was-typed'], allSelected: false });
});
it('does not read it as ALL when the variable offers no ALL', () => {
expect(commit(OPTIONS, { showAllOption: false })).toStrictEqual({
value: OPTIONS,
allSelected: false,
});
});
it('resolves an empty commit to the variable fallback', () => {
expect(commit([])).toBe(FALLBACK);
});
it('marks everything while the options have not arrived', () => {
// Nothing to judge against yet; erring this way keeps a value rather than dropping it.
expect(commit(['typed-in'], { options: [] })).toStrictEqual({
value: ['typed-in'],
allSelected: false,
customValues: ['typed-in'],
});
});
});

View File

@@ -4,6 +4,8 @@ import {
emptyVariableFormModel,
type VariableFormModel,
} from '../../DashboardSettings/Variables/variableFormModel';
import { VariableCycleReason } from '../../store/slices/variableFetchSlice';
import { useDashboardStore } from '../../store/useDashboardStore';
import type { VariableSelection } from '../selectionTypes';
import { useAutoSelect } from '../hooks/useAutoSelect';
@@ -15,7 +17,11 @@ function run(
variable: VariableFormModel,
options: string[],
selection: VariableSelection,
cycleReason?: VariableCycleReason,
): VariableSelection | undefined {
useDashboardStore.setState({
variableCycleReasons: cycleReason ? { [variable.name]: cycleReason } : {},
});
const onAutoSelect = jest.fn();
renderHook(() => useAutoSelect(variable, options, selection, onAutoSelect));
return onAutoSelect.mock.calls[0]?.[0];
@@ -70,11 +76,13 @@ describe('useAutoSelect', () => {
expect(next).toStrictEqual({ value: ['a', 'b'], allSelected: true });
});
it('falls back to ALL, not the first option, when every selected value is gone', () => {
// Re-scoped options only — a time-range refetch must NOT re-default; see below.
it('re-scoped: falls back to ALL, not the first option, when every selected value is gone', () => {
const next = run(
model({ type: 'QUERY', multiSelect: true, showAllOption: true }),
['x', 'y'],
{ value: ['a', 'b'], allSelected: false },
VariableCycleReason.ValueCascade,
);
expect(next).toStrictEqual({ value: ['x', 'y'], allSelected: true });
});
@@ -102,20 +110,23 @@ describe('useAutoSelect', () => {
expect(next).toStrictEqual({ value: ['b'], allSelected: false });
});
it('keeps the still-valid subset of a multi-select when options re-scope', () => {
it('re-scoped: keeps the still-valid subset of a multi-select', () => {
const next = run(
model({ type: 'QUERY', multiSelect: true }),
['a', 'b', 'd'],
{ value: ['a', 'b', 'c'], allSelected: false },
VariableCycleReason.ValueCascade,
);
expect(next).toStrictEqual({ value: ['a', 'b'], allSelected: false });
});
it('re-defaults a multi-select when none of the selected values remain', () => {
const next = run(model({ type: 'QUERY', multiSelect: true }), ['x', 'y'], {
value: ['a', 'b'],
allSelected: false,
});
it('re-scoped: re-defaults a multi-select when none of the selected values remain', () => {
const next = run(
model({ type: 'QUERY', multiSelect: true }),
['x', 'y'],
{ value: ['a', 'b'], allSelected: false },
VariableCycleReason.ValueCascade,
);
expect(next).toStrictEqual({ value: ['x'], allSelected: false });
});
@@ -151,4 +162,45 @@ describe('useAutoSelect', () => {
});
expect(next).toBeUndefined();
});
describe('by cycle reason', () => {
const service = model({
name: 'service',
type: 'DYNAMIC',
multiSelect: true,
showAllOption: true,
dynamicAttribute: 'service.name',
});
const gone: VariableSelection = { value: ['frontend'], allSelected: false };
it('keeps the selection when a full cycle refetched the options', () => {
// The new window has no data for the selected service — no reason to widen to ALL.
const next = run(
service,
['backend', 'cart'],
gone,
VariableCycleReason.FullCycle,
);
expect(next).toBeUndefined();
});
it('re-scopes the selection when a value cascade refetched the options', () => {
const next = run(
service,
['backend', 'cart'],
gone,
VariableCycleReason.ValueCascade,
);
expect(next).toStrictEqual({ value: null, allSelected: true });
});
it('reconciles a variable with no cycle of its own (custom definition change)', () => {
const next = run(
model({ name: 'env', type: 'CUSTOM', multiSelect: true }),
['staging', 'prod'],
{ value: ['dev'], allSelected: false },
);
expect(next).toStrictEqual({ value: ['staging'], allSelected: false });
});
});
});

View File

@@ -13,11 +13,11 @@ jest.mock('nuqs', () => ({
useQueryState: (): unknown => [null, jest.fn()],
}));
const mockGlobalTime = { minTime: 1, maxTime: 2, selectedTime: '5m' };
jest.mock('react-redux', () => ({
useSelector: (selector: (state: unknown) => unknown): unknown =>
selector({
globalTime: { minTime: 1, maxTime: 2, selectedTime: '5m' },
}),
selector({ globalTime: mockGlobalTime }),
}));
jest.mock('../../DashboardSettings/Variables/variableAdapters', () => ({
@@ -150,3 +150,57 @@ describe('useVariableSelection — setSelection', () => {
expect(svcCycleId()).toBe(before + 1);
});
});
describe('useVariableSelection — what a time-range change enqueues', () => {
// Longer than FETCH_CYCLE_DEBOUNCE_MS, which the hook keeps private.
const PAST_DEBOUNCE = 400;
function reasons(): Record<string, string> {
return useDashboardStore.getState().variableCycleReasons;
}
beforeEach(() => {
jest.useFakeTimers();
mockGlobalTime.selectedTime = '5m';
useDashboardStore.setState({
variableValues: {},
variableFetchStates: {},
variableLastUpdated: {},
variableCycleIds: {},
variableCycleReasons: {},
variableResolvedEmpty: {},
variableFetchContext: null,
lastFetchAllKey: null,
});
});
afterEach(() => {
jest.useRealTimers();
});
// The tag is what stops the reconcile re-defaulting a user's selection.
it('tags every variable as a full cycle, overriding an earlier cascade tag', () => {
const { result, rerender } = renderHook(() =>
useVariableSelection(dashboard),
);
act(() => {
jest.advanceTimersByTime(PAST_DEBOUNCE);
});
expect(reasons()).toStrictEqual({ env: 'full-cycle', svc: 'full-cycle' });
// A value change re-scopes the dependent's options: it may drop what no longer applies.
act(() => {
result.current.setSelection('env', { value: ['prod'], allSelected: false });
});
expect(reasons().svc).toBe('value-cascade');
mockGlobalTime.selectedTime = '30m';
rerender();
act(() => {
jest.advanceTimersByTime(PAST_DEBOUNCE);
});
expect(reasons()).toStrictEqual({ env: 'full-cycle', svc: 'full-cycle' });
});
});

View File

@@ -6,6 +6,7 @@ import { DashboardDetailEvents } from 'pages/DashboardPageV2/constants/events';
import type { VariableSelection } from '../../selectionTypes';
import { areSelectionsEqual } from '../../utils/resolveVariableSelection';
import { selectionFromCommittedValues } from '../../utils/selectionUtils';
import OverflowValuesTooltip from './OverflowValuesTooltip';
import styles from '../../VariablesBar.module.scss';
@@ -75,13 +76,23 @@ function ValueSelector({
options.every((option) => draft.includes(option));
const commit = (values: string[]): void => {
// CustomMultiSelect emits the full value set when ALL is picked.
const isAll =
showAllOption &&
options.length > 0 &&
options.every((option) => values.includes(option));
const next: VariableSelection =
values.length === 0 ? emptyFallback : { value: values, allSelected: isAll };
// A close that left the list as it opened commits nothing — else a pick covering
// every option this window offers would be promoted to a standing ALL.
if (
areSelectionsEqual(
{ value: values, allSelected: false },
{ value: committedValues, allSelected: false },
)
) {
return;
}
const next = selectionFromCommittedValues({
values,
options,
showAllOption,
emptyFallback,
});
// Closing without actually changing the selection must not re-fire onChange —
// that would needlessly re-cascade to dependent variables/panels.

View File

@@ -1,6 +1,11 @@
import { useEffect } from 'react';
import type { VariableFormModel } from '../../DashboardSettings/Variables/variableFormModel';
import {
selectVariableCycleReason,
VariableCycleReason,
} from '../../store/slices/variableFetchSlice';
import { useDashboardStore } from '../../store/useDashboardStore';
import { reconcileWithOptions } from '../utils/resolveVariableSelection';
import type { VariableSelection } from '../selectionTypes';
@@ -9,6 +14,9 @@ import type { VariableSelection } from '../selectionTypes';
* `onAutoSelect` only when the value must change. The reconcile rule lives in
* {@link reconcileWithOptions} (shared with seed + payload defaulting) so the bar
* and the panel query can never disagree about a variable's default.
*
* Only a value cascade may re-default the selection; a full cycle (time range,
* reload) leaves the user's pick alone. Types with no cycle of their own reconcile.
*/
export function useAutoSelect(
variable: VariableFormModel,
@@ -16,8 +24,14 @@ export function useAutoSelect(
selection: VariableSelection,
onAutoSelect: (selection: VariableSelection) => void,
): void {
const cycleReason = useDashboardStore(
selectVariableCycleReason(variable.name),
);
useEffect(() => {
const next = reconcileWithOptions(variable, selection, options);
const next = reconcileWithOptions(variable, selection, options, {
preserveSelection: cycleReason === VariableCycleReason.FullCycle,
});
if (next) {
onAutoSelect(next);
}

View File

@@ -10,6 +10,11 @@ export interface VariableSelection {
value: SelectedVariableValue;
/** True when every option is selected ("ALL"); for dynamic vars value may be null. */
allSelected: boolean;
/**
* Entries of `value` the user typed rather than picked. Never in any option list,
* so the reconcile keeps them instead of reading them as invalid.
*/
customValues?: string[];
}
/** Selected values for a dashboard's variables, keyed by variable name. */

View File

@@ -134,12 +134,23 @@ export function resolveDefaultSelection(
return { value: model.multiSelect ? [] : '', allSelected: false };
}
interface ReconcileOptions {
/**
* Set when no other variable caused this refetch (time-range change, reload): the
* selection then outranks the options and is kept as-is. Leave false for a
* dependency cascade, where a selection that no longer applies must give way.
*/
preserveSelection?: boolean;
}
/**
* Reconciles a variable's current selection against its freshly-fetched options.
* Returns the next selection, or null when nothing should change (a valid pick is
* left untouched — local-first). Behaviour, in order:
* - materialize ALL to the full option set (query/custom);
* - keep a still-valid multi-select subset, dropping only invalid entries;
* - keep a multi-select selection outright when `preserveSelection` is set;
* - keep a still-valid multi-select subset, dropping only entries the list no longer
* offers and the user did not type in (`customValues`);
* - otherwise auto-pick the default (or first option) so dependent variables and
* panels always resolve against a usable value.
*/
@@ -147,6 +158,7 @@ export function reconcileWithOptions(
model: VariableFormModel,
current: VariableSelection,
options: string[],
{ preserveSelection = false }: ReconcileOptions = {},
): VariableSelection | null {
if (options.length === 0) {
return null;
@@ -161,13 +173,31 @@ export function reconcileWithOptions(
Array.isArray(current.value) &&
current.value.length > 0
) {
const valid = current.value.map(String).filter((c) => options.includes(c));
// A pick this window has no data for is still the user's filter; re-defaulting it
// here is what widened a single pick to ALL on every time-range change.
if (preserveSelection) {
return null;
}
// A typed value is in no option list, so it is never "no longer offered".
const custom = new Set(current.customValues ?? []);
const valid = current.value
.map(String)
.filter((c) => options.includes(c) || custom.has(c));
if (valid.length === current.value.length) {
return null;
}
return valid.length > 0
? { value: valid, allSelected: false }
: fillDefault(model, options);
if (valid.length === 0) {
return fillDefault(model, options);
}
const customValues = valid.filter((v) => custom.has(v));
return {
value: valid,
allSelected: false,
...(customValues.length > 0 && { customValues }),
};
}
if (!model.multiSelect) {

View File

@@ -47,6 +47,43 @@ export function hasUsableValue(
return value !== '' && value !== null && value !== undefined;
}
interface CommittedValues {
values: string[];
options: string[];
showAllOption: boolean;
emptyFallback: VariableSelection;
}
/**
* The selection a multi-select commit resolves to. Options are known only here, so
* this is where a value the list never offered is recorded as typed in.
*/
export function selectionFromCommittedValues({
values,
options,
showAllOption,
emptyFallback,
}: CommittedValues): VariableSelection {
if (values.length === 0) {
return emptyFallback;
}
const customValues = values.filter((value) => !options.includes(value));
// ALL re-materializes to the option set, so a set carrying a typed value is not ALL
// — the next refetch would expand it back and drop what the user typed.
const allSelected =
showAllOption &&
options.length > 0 &&
customValues.length === 0 &&
options.every((option) => values.includes(option));
return {
value: values,
allSelected,
...(customValues.length > 0 && { customValues }),
};
}
/** Flatten the selection map into the `{ name: value }` payload a query expects. */
export function selectionToPayload(
selection: VariableSelectionMap,

View File

@@ -34,6 +34,7 @@ function reset(names: string[], context: VariableFetchContext): void {
variableFetchStates: {},
variableLastUpdated: {},
variableCycleIds: {},
variableCycleReasons: {},
variableFetchContext: null,
});
store().initVariableFetch(names, context);
@@ -133,6 +134,33 @@ describe('variableFetchSlice', () => {
expect(states().q1).toBe('error');
expect(states().q2).toBe('idle');
});
// The reason is what tells the post-fetch reconcile whether it may re-default a
// selection: a full cycle must not, a value cascade must.
it('tags a full cycle, then re-tags only the cascaded variables', () => {
store().enqueueFetchAll();
expect(store().variableCycleReasons).toStrictEqual({
q1: 'full-cycle',
q2: 'full-cycle',
d1: 'full-cycle',
d2: 'full-cycle',
});
resolve('q1');
store().enqueueDescendants('q1');
expect(store().variableCycleReasons).toStrictEqual({
q1: 'full-cycle',
q2: 'value-cascade',
d1: 'full-cycle',
d2: 'full-cycle',
});
});
it('drops the reason for a variable that no longer exists', () => {
store().enqueueFetchAll();
store().initVariableFetch(['q1'], context);
expect(store().variableCycleReasons).toStrictEqual({ q1: 'full-cycle' });
});
});
describe('variableFetchSlice — query depends on a dynamic', () => {

View File

@@ -9,6 +9,7 @@ import {
type FetchMaps,
isVariableInActiveFetchState,
resolveFetchState,
VariableCycleReason,
VariableFetchState,
} from './variableFetchSlice.utils';
@@ -30,7 +31,10 @@ function queryParentsHaveValues(
);
}
export { VariableFetchState } from './variableFetchSlice.utils';
export {
VariableCycleReason,
VariableFetchState,
} from './variableFetchSlice.utils';
/**
* Runtime fetch orchestration for dashboard variables — native port of V1's
@@ -45,6 +49,8 @@ export interface VariableFetchSlice {
variableFetchStates: Record<string, VariableFetchState>;
variableLastUpdated: Record<string, number>;
variableCycleIds: Record<string, number>;
/** Why each variable's current cycle was enqueued, read by the post-fetch reconcile. */
variableCycleReasons: Record<string, VariableCycleReason>;
/**
* Whether a QUERY/DYNAMIC variable settled its fetch with zero options (so it
* will never get a value). Lets a dependent panel fall through to "no data"
@@ -106,6 +112,7 @@ export const createVariableFetchSlice: StateCreator<
variableFetchStates: {},
variableLastUpdated: {},
variableCycleIds: {},
variableCycleReasons: {},
variableResolvedEmpty: {},
variableFetchContext: null,
lastFetchAllKey: null,
@@ -115,6 +122,7 @@ export const createVariableFetchSlice: StateCreator<
variableFetchStates: {},
variableLastUpdated: {},
variableCycleIds: {},
variableCycleReasons: {},
variableResolvedEmpty: {},
variableFetchContext: null,
lastFetchAllKey: null,
@@ -132,6 +140,7 @@ export const createVariableFetchSlice: StateCreator<
initVariableFetch: (names, context): void => {
const maps = cloneMaps(get());
const resolvedEmpty = { ...get().variableResolvedEmpty };
const reasons = { ...get().variableCycleReasons };
names.forEach((name) => {
if (!maps.states[name]) {
maps.states[name] = VariableFetchState.Idle;
@@ -144,12 +153,14 @@ export const createVariableFetchSlice: StateCreator<
delete maps.lastUpdated[name];
delete maps.cycleIds[name];
delete resolvedEmpty[name];
delete reasons[name];
}
});
set({
variableFetchStates: maps.states,
variableLastUpdated: maps.lastUpdated,
variableCycleIds: maps.cycleIds,
variableCycleReasons: reasons,
variableResolvedEmpty: resolvedEmpty,
variableFetchContext: context,
});
@@ -171,6 +182,11 @@ export const createVariableFetchSlice: StateCreator<
dynamicVariableOrder,
} = variableFetchContext;
const maps = cloneMaps(get());
const reasons = { ...get().variableCycleReasons };
const bump = (name: string): void => {
maps.cycleIds[name] = (maps.cycleIds[name] || 0) + 1;
reasons[name] = VariableCycleReason.FullCycle;
};
// Query variables wait only for their QUERY parents. A DYNAMIC parent does not
// gate: its option fetch feeds only its own dropdown, while its selected value
@@ -178,7 +194,7 @@ export const createVariableFetchSlice: StateCreator<
// dependent query substitutes it immediately and refetches via the cascade if
// it later changes. Text/custom parents resolve synchronously, so nothing waits.
queryVariableOrder.forEach((name) => {
maps.cycleIds[name] = (maps.cycleIds[name] || 0) + 1;
bump(name);
const parents = dependencyData.parentGraph[name] || [];
const hasQueryParents = parents.some((p) => variableTypes[p] === 'QUERY');
maps.states[name] = hasQueryParents
@@ -192,7 +208,7 @@ export const createVariableFetchSlice: StateCreator<
const orderedQuery = new Set(queryVariableOrder);
Object.keys(variableTypes).forEach((name) => {
if (variableTypes[name] === 'QUERY' && !orderedQuery.has(name)) {
maps.cycleIds[name] = (maps.cycleIds[name] || 0) + 1;
bump(name);
maps.states[name] = resolveFetchState(maps, name);
}
});
@@ -203,7 +219,7 @@ export const createVariableFetchSlice: StateCreator<
// populate fast even when query variables are slow; a sibling selection change
// later refetches them via `enqueueDescendantsBatch`.
dynamicVariableOrder.forEach((name) => {
maps.cycleIds[name] = (maps.cycleIds[name] || 0) + 1;
bump(name);
maps.states[name] = resolveFetchState(maps, name);
});
@@ -211,6 +227,7 @@ export const createVariableFetchSlice: StateCreator<
variableFetchStates: maps.states,
variableLastUpdated: maps.lastUpdated,
variableCycleIds: maps.cycleIds,
variableCycleReasons: reasons,
lastFetchAllKey: key ?? get().lastFetchAllKey,
});
},
@@ -290,6 +307,11 @@ export const createVariableFetchSlice: StateCreator<
const { dependencyData, variableTypes, dynamicVariableOrder } =
variableFetchContext;
const maps = cloneMaps(get());
const reasons = { ...get().variableCycleReasons };
const bump = (name: string): void => {
maps.cycleIds[name] = (maps.cycleIds[name] || 0) + 1;
reasons[name] = VariableCycleReason.ValueCascade;
};
const changed = new Set(names);
// Callers commit values before this runs, so the gate sees the new parent values.
const selection = selectVariableValues(get().dashboardId)(get());
@@ -305,7 +327,7 @@ export const createVariableFetchSlice: StateCreator<
});
});
queryDescendants.forEach((desc) => {
maps.cycleIds[desc] = (maps.cycleIds[desc] || 0) + 1;
bump(desc);
maps.states[desc] = queryParentsHaveValues(
desc,
variableFetchContext,
@@ -322,7 +344,7 @@ export const createVariableFetchSlice: StateCreator<
dynamicVariableOrder
.filter((dynName) => !changed.has(dynName))
.forEach((dynName) => {
maps.cycleIds[dynName] = (maps.cycleIds[dynName] || 0) + 1;
bump(dynName);
maps.states[dynName] = resolveFetchState(maps, dynName);
});
}
@@ -331,6 +353,7 @@ export const createVariableFetchSlice: StateCreator<
variableFetchStates: maps.states,
variableLastUpdated: maps.lastUpdated,
variableCycleIds: maps.cycleIds,
variableCycleReasons: reasons,
});
},
});
@@ -347,6 +370,12 @@ export const selectVariableCycleId =
(state: DashboardStore): number =>
state.variableCycleIds[name] ?? 0;
/** Selector: why a variable's cycle was enqueued. Undefined for types that never fetch. */
export const selectVariableCycleReason =
(name: string) =>
(state: DashboardStore): VariableCycleReason | undefined =>
state.variableCycleReasons[name];
/** Selector: whether a variable has completed at least one fetch. */
export const selectVariableFetchedOnce =
(name: string) =>

View File

@@ -7,6 +7,14 @@ export enum VariableFetchState {
Error = 'error',
}
/** Why a cycle was started — only a cascade may re-default a user's selection. */
export enum VariableCycleReason {
/** `enqueueFetchAll`: load, time-range or variable-order change. */
FullCycle = 'full-cycle',
/** `enqueueDescendantsBatch`: a parent or sibling variable's value changed. */
ValueCascade = 'value-cascade',
}
/** Mutable clones a fetch action works over before committing back in one `set`. */
export interface FetchMaps {
states: Record<string, VariableFetchState>;

View File

@@ -1,9 +0,0 @@
export interface Props {
token: string;
password: string;
}
export interface PayloadProps {
data: string;
status: string;
}

View File

@@ -245,11 +245,13 @@ func (module *module) PatchV2(ctx context.Context, orgID valuer.UUID, id valuer.
}
func (module *module) DeleteV2(ctx context.Context, orgID valuer.UUID, id valuer.UUID) error {
existing, err := module.GetV2(ctx, orgID, id)
// Read the storable, not the decoded v2 dashboard: deleting must work even
// when the stored data is corrupt or never migrated off the v1 schema.
storable, err := module.store.Get(ctx, orgID, id)
if err != nil {
return err
}
if err := existing.ErrIfNotDeletable(); err != nil {
if err := storable.ErrIfNotDeletable(); err != nil {
return err
}

View File

@@ -993,8 +993,8 @@ func TestBuild_TraceList_MultiVariantGateKey(t *testing.T) {
assert.Contains(t, got, "mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_number, 'gen_ai.tool.name')")
}
// A `trace.`-prefixed aggregate in the filter box and the same condition in the
// explicit Having box build the same query; output-only aggregates are rejected.
// `trace.` marks a trace-level aggregate; `tracefield.` routes trace-level too but is
// not a rewritable alias, so the HAVING rewriter rejects it.
func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
b := newTestBuilder(t)
build := func(q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) (*qbtypes.Statement, error) {
@@ -1002,14 +1002,19 @@ func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTrace, q, nil)
}
viaTrace, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
_, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"}})
require.NoError(t, err)
viaHaving, err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Having: &qbtypes.Having{Expression: "trace.output_tokens > 1000"}})
require.NoError(t, err)
assert.Equal(t, viaTrace.Query, viaHaving.Query)
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
assert.Contains(t, err.Error(), "Invalid references in `Having` expression: [tracefield.output_tokens]")
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Having: &qbtypes.Having{Expression: "tracefield.output_tokens > 1000"}})
require.Error(t, err)
assert.Contains(t, err.Error(), "Invalid references in `Having` expression: [tracefield.output_tokens]")
_, err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Filter: &qbtypes.Filter{Expression: "trace.span_count > 3"}})
@@ -1017,8 +1022,7 @@ func TestBuild_TraceList_TraceContextPrefix(t *testing.T) {
assert.Contains(t, err.Error(), "cannot be used")
}
// Query variables in a trace-level condition resolve like span filters: bound args,
// list/IN handling, dynamic __all__ dropping the condition.
// Query variables in a trace-level condition are substituted into the HAVING.
func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
b := newTestBuilder(t)
build := func(expr string, vars map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
@@ -1030,18 +1034,17 @@ func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
}, vars)
}
// scalar variable -> bound arg via the filter pipeline
// scalar variable -> literal in HAVING
stmt, err := build("trace.output_tokens > $threshold",
map[string]qbtypes.VariableItem{"threshold": {Value: 700}})
require.NoError(t, err)
assert.Contains(t, stmt.Query, "HAVING output_tokens > ?")
assert.Contains(t, stmt.Args, float64(700))
assert.Contains(t, stmt.Query, "HAVING output_tokens > 700")
// list variable with IN
stmt, err = build("trace.llm_call_count IN $counts",
map[string]qbtypes.VariableItem{"counts": {Value: []any{1, 2}}})
require.NoError(t, err)
assert.Contains(t, stmt.Query, "HAVING llm_call_count IN (?, ?)")
assert.Contains(t, stmt.Query, "HAVING llm_call_count IN")
// dynamic __all__ -> condition dropped, no HAVING at all
stmt, err = build("trace.output_tokens > $threshold",
@@ -1049,7 +1052,7 @@ func TestBuild_TraceList_VariableInAggregateFilter(t *testing.T) {
require.NoError(t, err)
assert.NotContains(t, stmt.Query, "HAVING")
// unresolved variable -> rejected, though only as an unknown aggregate today
// unresolved variable -> rejected, not compared as a literal
_, err = build("trace.output_tokens > $missing", map[string]qbtypes.VariableItem{"other": {Value: 1}})
require.Error(t, err)
}

View File

@@ -1,805 +0,0 @@
package aistatementbuilder
import (
"context"
"testing"
"time"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Build tests for scalar / time-series through the gen_ai scope; the
// rewriteTraceAggregation unit tests live in scopedtracesstatementbuilder.
// The goldens build up one dimension at a time: base → span filter → trace filter
// → group by → everything combined; then the time-series variants.
// The empty base case never reaches the builder: request validation rejects a
// scalar / time-series builder_ai_query with no aggregations, so the builder
// assumes at least one (internal calls are validated upstream).
func TestBuild_Aggregation_NoAggregations_RejectedByRequestValidation(t *testing.T) {
for _, rt := range []qbtypes.RequestType{qbtypes.RequestTypeScalar, qbtypes.RequestTypeTimeSeries} {
req := qbtypes.QueryRangeRequest{
Start: testStartMs,
End: testEndMs,
RequestType: rt,
CompositeQuery: qbtypes.CompositeQuery{
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilderAI,
Spec: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Name: "A",
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
},
}},
},
}
require.ErrorContains(t, req.Validate(), "at least one aggregation is required", rt.StringValue())
}
}
// Base scalar over per-trace values: one window-clipped per-trace scan, outer avg
// across traces. Traces without token spans yield NULL, which avg skips.
func TestBuild_FullSQL_Scalar_TraceAgg(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span-level filter: resolved through the standard filter pipeline and ANDed into
// the per-trace scan's WHERE, next to the gate mask.
func TestBuild_FullSQL_Scalar_SpanFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini'"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Trace-level filter: qualification first — __qualified selects the trace ids whose
// whole-window value passes, then the per-trace scan is constrained to them.
func TestBuild_FullSQL_Scalar_TraceFilter(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING output_tokens > 1000
),
__scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Group by a span attribute: selected (stringified) and grouped in the per-trace
// scan, then grouped again in the outer aggregation. A trace spanning two models
// contributes one per-trace row per model.
func TestBuild_FullSQL_Scalar_GroupBy(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model
)
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, avg(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Grouping by an intrinsic: the positional alias keeps `toString(name) AS name` (a cyclic
// alias) from forming, and an order key on the dimension resolves to that alias.
func TestBuild_FullSQL_Scalar_GroupByIntrinsic(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "name"}}},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "name"}}, Direction: qbtypes.OrderDirectionAsc}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
toString(multiIf(name <> '', toString(name), NULL)) AS __GROUP_BY_KEY_0_name,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, __GROUP_BY_KEY_0_name
)
SELECT __GROUP_BY_KEY_0_name, avg(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY __GROUP_BY_KEY_0_name
ORDER BY __GROUP_BY_KEY_0_name asc
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Everything combined: span + trace filter parts split (WHERE + __qualified), group
// by, two aggregations, HAVING on the alias (rewritten to __result_0), explicit
// order and limit.
func TestBuild_FullSQL_Scalar_FullCombo(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{
{Expression: "avg(trace.output_tokens)", Alias: "avg_out"},
{Expression: "count(trace.trace_id)"},
},
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND trace.total_tokens > 100"},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}}},
Having: &qbtypes.Having{Expression: "avg_out > 50"},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "avg_out"}}, Direction: qbtypes.OrderDirectionDesc}},
Limit: 5,
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING total_tokens > 100
),
__scoped_traces AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model
)
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, avg(output_tokens) AS __result_0, count(trace_id) AS __result_1
FROM __scoped_traces
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model
HAVING __result_0 > 50
ORDER BY __result_0 desc
LIMIT 5
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Time series: the per-trace scan buckets by span time, the outer aggregation per bucket.
func TestBuild_FullSQL_TimeSeries_TraceAgg(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces AS (
SELECT trace_id,
toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, ts
)
SELECT ts, avg(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY ts
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Grouped, limited time series: groups are ranked on whole-window per-trace values
// (__scoped_traces_total, no ts bucketing → exact for non-composable aggregates like
// avg), __limit_cte keeps the top-N by the requested order, and the bucketed main
// scan is pruned to those groups before aggregating. The alias HAVING applies to the
// outer aggregation; the aggregation order key ranks __limit_cte, the series
// themselves order by ts.
func TestBuild_FullSQL_TimeSeries_GroupLimit(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "sum(trace.output_tokens)", Alias: "total_out"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}}},
Having: &qbtypes.Having{Expression: "total_out > 500"},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "total_out"}}, Direction: qbtypes.OrderDirectionDesc}},
Limit: 3,
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __scoped_traces_total AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model
),
__limit_cte AS (
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, sum(output_tokens) AS __result_0
FROM __scoped_traces_total
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model
ORDER BY __result_0 desc
LIMIT 3
),
__scoped_traces AS (
SELECT trace_id,
toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL))) GLOBAL IN (SELECT __GROUP_BY_KEY_0_gen_ai.request.model FROM __limit_cte)
GROUP BY trace_id, ts, __GROUP_BY_KEY_0_gen_ai.request.model
)
SELECT ts, __GROUP_BY_KEY_0_gen_ai.request.model, sum(output_tokens) AS __result_0
FROM __scoped_traces
GROUP BY ts, __GROUP_BY_KEY_0_gen_ai.request.model
HAVING __result_0 > 500
ORDER BY ts desc
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Span-level scalar with a trace-level filter: delegated to the trace builder,
// constrained by the __trace_scope qualification (the delegate's own scalar shape,
// hence no SETTINGS suffix).
func TestBuild_FullSQL_Scalar_SpanAgg_TraceScoped(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "sum(gen_ai.usage.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __trace_scope AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING output_tokens > 1000
)
SELECT sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS __result_0
FROM signoz_traces.distributed_signoz_index_v3
WHERE trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
ORDER BY __result_0 DESC
`, stmt)
}
// Grouped, limited time series with two group keys and a split filter: the top-N
// prune is a 2-tuple GLOBAL IN, the qualification and span predicate apply to the
// ranking scan and the main scan alike, and with no explicit order the ranking
// defaults to __result_0 DESC.
func TestBuild_FullSQL_TimeSeries_GroupLimit_MultiKey(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{
{Expression: "sum(trace.output_tokens)"},
{Expression: "count(trace.trace_id)"},
},
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND trace.total_tokens > 100"},
GroupBy: []qbtypes.GroupByKey{
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.request.model"}},
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "gen_ai.user.id"}},
},
Limit: 2,
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.input_tokens'), toFloat64(attributes_number['gen_ai.usage.input_tokens']), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING total_tokens > 100
),
__scoped_traces_total AS (
SELECT trace_id,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
toString(multiIf(mapContains(attributes_string, 'gen_ai.user.id'), attributes_string['gen_ai.user.id'], NULL)) AS __GROUP_BY_KEY_1_gen_ai.user.id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
),
__limit_cte AS (
SELECT __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id, sum(output_tokens) AS __result_0, count(trace_id) AS __result_1
FROM __scoped_traces_total
GROUP BY __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
ORDER BY __result_0 DESC
LIMIT 2
),
__scoped_traces AS (
SELECT trace_id,
toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts,
toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)) AS __GROUP_BY_KEY_0_gen_ai.request.model,
toString(multiIf(mapContains(attributes_string, 'gen_ai.user.id'), attributes_string['gen_ai.user.id'], NULL)) AS __GROUP_BY_KEY_1_gen_ai.user.id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND (attributes_string['gen_ai.request.model'] = 'gpt-4o-mini' AND mapContains(attributes_string, 'gen_ai.request.model'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
AND (toString(multiIf(mapContains(attributes_string, 'gen_ai.request.model'), attributes_string['gen_ai.request.model'], NULL)), toString(multiIf(mapContains(attributes_string, 'gen_ai.user.id'), attributes_string['gen_ai.user.id'], NULL))) GLOBAL IN (SELECT __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id FROM __limit_cte)
GROUP BY trace_id, ts, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
)
SELECT ts, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id, sum(output_tokens) AS __result_0, count(trace_id) AS __result_1
FROM __scoped_traces
GROUP BY ts, __GROUP_BY_KEY_0_gen_ai.request.model, __GROUP_BY_KEY_1_gen_ai.user.id
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// A time-series limit without group-by has nothing to rank: it is ignored, matching
// the trace builder — the query equals its unlimited form.
func TestBuild_TimeSeries_LimitWithoutGroupByIgnored(t *testing.T) {
b := newTestBuilder(t)
build := func(limit int) *qbtypes.Statement {
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 60 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Limit: limit,
}, nil)
require.NoError(t, err)
return stmt
}
assert.Equal(t, build(0).Query, build(5).Query)
}
// ---------------------------------------------------------------------------
// Behavior / branch tests not covered by the goldens above
// ---------------------------------------------------------------------------
// Mixing span- and trace-level aggregations across one query is rejected.
func TestBuild_Aggregation_MixedDomainsRejected(t *testing.T) {
b := newTestBuilder(t)
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{
{Expression: "avg(trace.output_tokens)"},
{Expression: "sum(gen_ai.usage.output_tokens)"},
},
}, nil)
require.ErrorContains(t, err, "cannot be mixed")
}
// Output-only aggregates are rejected in trace-level filters on the aggregation
// path too (the raw and trace-list paths are covered elsewhere).
func TestBuild_Aggregation_OutputOnlyFilterRejected(t *testing.T) {
b := newTestBuilder(t)
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "count()"}},
Filter: &qbtypes.Filter{Expression: "trace.span_count > 3"},
}, nil)
require.ErrorContains(t, err, `aggregate "span_count" cannot be used`)
}
// Trace-level columns are rejected as group-by keys with a targeted builder error;
// order keys never reach the builder — request validation only admits group keys and
// aggregation aliases/expressions — and ordering by the alias stays valid.
func TestBuild_Aggregation_GroupByOrderValidation(t *testing.T) {
b := newTestBuilder(t)
ctx := context.Background()
_, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
GroupBy: []qbtypes.GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace.llm_call_count"}}},
}, nil)
require.ErrorContains(t, err, `grouping by trace-level aggregate "trace.llm_call_count" is not supported`)
req := qbtypes.QueryRangeRequest{
Start: testStartMs,
End: testEndMs,
RequestType: qbtypes.RequestTypeScalar,
CompositeQuery: qbtypes.CompositeQuery{
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilderAI,
Spec: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Name: "A",
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace.total_tokens"}}, Direction: qbtypes.OrderDirectionDesc}},
},
}},
},
}
require.ErrorContains(t, req.Validate(), "invalid order by key")
_, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)", Alias: "avg_out"}},
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "avg_out"}}, Direction: qbtypes.OrderDirectionAsc}},
}, nil)
require.NoError(t, err)
}
// Variables in trace-level conditions resolve through the standard pipeline as bound
// args; a dynamic __all__ drops the condition entirely; an unresolved $var is only
// rejected as an unknown aggregate today (a targeted variable error is a separate
// concern).
func TestBuild_FullSQL_Aggregation_VariablesInTraceFilter(t *testing.T) {
b := newTestBuilder(t)
ctx := context.Background()
q := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > $threshold"},
}
stmt, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q,
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.TextBoxVariableType, Value: float64(1000)}})
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING output_tokens > 1000
),
__scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
// an unresolved $var is only rejected as an unknown aggregate today; a targeted
// "unknown variable" error is a separate concern
_, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q, nil)
require.ErrorContains(t, err, `aggregate "$threshold" cannot be used`)
// __all__ drops the condition: the query equals its unfiltered form
stmt, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q,
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.DynamicVariableType, Value: "__all__"}})
require.NoError(t, err)
unfiltered := q
unfiltered.Filter = nil
want, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, unfiltered, nil)
require.NoError(t, err)
assert.Equal(t, want.Query, stmt.Query)
// list variables render as IN with bound args; the scan selects only trace_id
// since no aggregation touches a per-trace column
stmt, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "count(trace.trace_id)"}},
Filter: &qbtypes.Filter{Expression: "trace.llm_call_count IN $counts"},
}, map[string]qbtypes.VariableItem{
"counts": {Type: qbtypes.QueryVariableType, Value: []any{float64(1), float64(2)}},
})
require.NoError(t, err)
assertSQLEqual(t, `
WITH __qualified AS (
SELECT trace_id,
countIf(mapContains(attributes_string, 'gen_ai.request.model')) AS llm_call_count
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
GROUP BY trace_id
HAVING llm_call_count IN (1, 2)
),
__scoped_traces AS (
SELECT trace_id
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT count(trace_id) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Resource conditions on the native path: the __resource_filter CTE prunes the
// qualification scan and the per-trace scan by fingerprint.
func TestBuild_FullSQL_Aggregation_ResourceFilter_Native(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "avg(trace.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "service.name = 'api' AND trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __resource_filter AS (
SELECT fingerprint
FROM signoz_traces.distributed_traces_v3_resource
WHERE (simpleJSONExtractString(labels, 'service.name') = 'api' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"api%')
AND seen_at_ts_bucket_start >= 1747945619
AND seen_at_ts_bucket_start <= 1747983448
GROUP BY fingerprint
),
__qualified AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
GROUP BY trace_id
HAVING output_tokens > 1000
),
__scoped_traces AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
AND trace_id GLOBAL IN (SELECT trace_id FROM __qualified)
GROUP BY trace_id
)
SELECT avg(output_tokens) AS __result_0
FROM __scoped_traces
ORDER BY __result_0 DESC
SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000
`, stmt)
}
// Resource conditions on the delegated path: the standalone __trace_scope inlines its
// fingerprint subquery (it is built without the delegate's CTEs), while the delegate
// keeps its own __resource_filter CTE and inline resource predicate.
func TestBuild_FullSQL_Aggregation_ResourceFilter_Delegated(t *testing.T) {
b := newTestBuilder(t)
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar,
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "sum(gen_ai.usage.output_tokens)"}},
Filter: &qbtypes.Filter{Expression: "service.name = 'api' AND trace.output_tokens > 1000"},
}, nil)
require.NoError(t, err)
assertSQLEqual(t, `
WITH __resource_filter AS (
SELECT fingerprint
FROM signoz_traces.distributed_traces_v3_resource
WHERE ((simpleJSONExtractString(labels, 'service.name') = 'api' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"api%'))
AND seen_at_ts_bucket_start >= 1747945619
AND seen_at_ts_bucket_start <= 1747983448
GROUP BY fingerprint
),
__trace_scope AS (
SELECT trace_id,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND (mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))
AND resource_fingerprint GLOBAL IN (SELECT fingerprint FROM (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = 'api' AND labels LIKE '%service.name%' AND labels LIKE '%service.name":"api%') AND seen_at_ts_bucket_start >= 1747945619 AND seen_at_ts_bucket_start <= 1747983448 GROUP BY fingerprint))
GROUP BY trace_id
HAVING output_tokens > 1000
)
SELECT sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS __result_0
FROM signoz_traces.distributed_signoz_index_v3
WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)
AND trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)
AND (((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AND ((multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = 'api' AND multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL)))
AND timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
ORDER BY __result_0 DESC
`, stmt)
}
// rate() over a trace-level column divides by the window (scalar) / step (series).
// Note the AggreFuncMap semantics: it counts per-trace rows per second — it does not
// sum the column.
func TestBuild_Aggregation_RateDividesByInterval(t *testing.T) {
b := newTestBuilder(t)
ctx := context.Background()
q := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
Aggregations: []qbtypes.TraceAggregation{{Expression: "rate(trace.llm_call_count)"}},
}
stmt, err := b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeScalar, q, nil)
require.NoError(t, err)
assert.Contains(t, stmt.Query, "count(llm_call_count)/36029 AS __result_0") // (end-start) seconds
q.StepInterval = qbtypes.Step{Duration: 60 * time.Second}
stmt, err = b.Build(ctx, valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeTimeSeries, q, nil)
require.NoError(t, err)
assert.Contains(t, stmt.Query, "count(llm_call_count)/60 AS __result_0")
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"log/slog"
"sort"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
@@ -18,6 +19,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
qbvariables "github.com/SigNoz/signoz/pkg/variables"
"github.com/huandu/go-sqlbuilder"
)
@@ -113,8 +115,6 @@ func (b *scopedTraceStatementBuilder) Build(
return b.buildTraceListQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), query, variables)
case qbtypes.RequestTypeRaw:
return b.buildDelegated(ctx, orgID, start, end, requestType, query, variables)
case qbtypes.RequestTypeScalar, qbtypes.RequestTypeTimeSeries:
return b.buildAggregation(ctx, orgID, start, end, requestType, query, variables)
default:
return nil, ErrUnsupportedRequestType
}
@@ -143,62 +143,6 @@ func (b *scopedTraceStatementBuilder) buildDelegated(
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
// traceScopedStatementBuilder is the delegate's optional capability of constraining a
// query to a set of trace ids (implemented by the traces statement builder).
type traceScopedStatementBuilder interface {
BuildTraceScoped(ctx context.Context, orgID valuer.UUID, start, end uint64, requestType qbtypes.RequestType, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], variables map[string]qbtypes.VariableItem, traceScope *qbtypes.Statement) (*qbtypes.Statement, error)
}
// buildDelegatedAggregation serves span-level scalar/time-series: the gate is ANDed
// into the filter's span-level part and the query delegates to the standard trace
// builder; a trace-level part becomes a qualification the delegate constrains
// trace_id by.
func (b *scopedTraceStatementBuilder) buildDelegatedAggregation(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
var spanExpr, traceExpr string
var err error
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
spanExpr, traceExpr, err = querybuilder.SplitFilterForAggregates(query.Filter.Expression, b.aggregateAliasSet())
if err != nil {
return nil, err
}
}
gate := b.scope.FilterExpression
expr := gate
if strings.TrimSpace(spanExpr) != "" {
expr = fmt.Sprintf("(%s) AND (%s)", gate, spanExpr)
}
// shallow copy; only Filter is replaced, caller's query untouched
gated := query
gated.Filter = &qbtypes.Filter{Expression: expr}
if strings.TrimSpace(traceExpr) == "" {
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
scoped, ok := b.traceStmtBuilder.(traceScopedStatementBuilder)
if !ok {
return nil, errors.NewInternalf(errors.CodeInternal, "trace statement builder does not support trace-scoped queries")
}
scope, err := b.buildQualifiedStatement(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), traceExpr, query, variables)
if err != nil {
return nil, err
}
if scope == nil {
// every trace-level condition was dropped by variable resolution
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
}
return scoped.BuildTraceScoped(ctx, orgID, start, end, requestType, gated, variables, scope)
}
// buildTraceListQuery wires the CTE pipeline (start/end are nanoseconds):
// matched (windowed, mask-pruned top-N trace_ids) → ranked (their [start,end] from
// the summary table) → buckets (ts_bucket_start prune) → enrichment (every per-trace
@@ -240,17 +184,22 @@ func (b *scopedTraceStatementBuilder) buildTraceListQuery(
if err != nil {
return nil, err
}
orderableSet := orderableAliasSet(resolved)
resourceFrag, resourceArgs, resourcePred, err := b.maybeAttachResourceFilter(ctx, orgID, query, start, end, variables)
if err != nil {
return nil, err
}
fp, err := b.splitFilter(ctx, orgID, query, b.aggregateAliasSet(), start, end, variables, matchedSB)
fp, err := b.splitFilter(ctx, orgID, query, b.aggregateAliasSet(), orderableSet, start, end, variables, matchedSB)
if err != nil {
return nil, err
}
matchedFrag, matchedArgs := b.buildMatchedCTE(matchedSB, start, end, startBucket, endBucket, resolved, orders, maskExpr, fp, resourcePred, limit, query.Offset)
matchedFrag, matchedArgs, err := b.buildMatchedCTE(matchedSB, start, end, startBucket, endBucket, resolved, orders, orderableSet, maskExpr, fp, resourcePred, limit, query.Offset)
if err != nil {
return nil, err
}
rankedFrag, rankedArgs := b.buildRankedCTE(start, end)
adj := querybuilder.BucketAdjustment // 30-min bucket width in seconds
@@ -431,27 +380,27 @@ func (b *scopedTraceStatementBuilder) resolveListOrders(order []qbtypes.OrderBy,
return orders, nil
}
// filterParts is the user filter split into a span-level predicate and the resolved
// trace-level HAVING (nil when there is none).
// filterParts is the user filter split into a span-level predicate and a trace-level
// HAVING expression.
type filterParts struct {
spanPred string
hasSpanFilter bool
having *traceHaving
havingExpr string
warnings []string
warningsURL string
}
// splitFilter splits query.Filter into a span-level predicate and a trace-level
// HAVING (explicit query.Having ANDed on before resolution); args bind into sb.
func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID valuer.UUID, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], classifySet map[string]struct{}, start, end uint64, variables map[string]qbtypes.VariableItem, sb *sqlbuilder.SelectBuilder) (filterParts, error) {
// splitFilter splits query.Filter into a span-level predicate (args bound into sb)
// and a trace-level HAVING (explicit query.Having ANDed on), then validates the
// trace-level part against the matched-pass aggregates.
func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID valuer.UUID, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], classifySet, orderableSet map[string]struct{}, start, end uint64, variables map[string]qbtypes.VariableItem, sb *sqlbuilder.SelectBuilder) (filterParts, error) {
var fp filterParts
havingExpr := ""
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
spanExpr, traceExpr, err := querybuilder.SplitFilterForAggregates(query.Filter.Expression, classifySet)
if err != nil {
return fp, err
}
havingExpr = traceExpr
fp.havingExpr = traceExpr
if strings.TrimSpace(spanExpr) != "" {
pred, warnings, url, err := b.resolveSpanPredicate(ctx, orgID, start, end, spanExpr, variables, sb)
if err != nil {
@@ -466,17 +415,23 @@ func (b *scopedTraceStatementBuilder) splitFilter(ctx context.Context, orgID val
}
}
if query.Having != nil && strings.TrimSpace(query.Having.Expression) != "" {
if havingExpr != "" {
havingExpr = fmt.Sprintf("(%s) AND (%s)", havingExpr, query.Having.Expression)
if fp.havingExpr != "" {
fp.havingExpr = fmt.Sprintf("(%s) AND (%s)", fp.havingExpr, query.Having.Expression)
} else {
havingExpr = query.Having.Expression
fp.havingExpr = query.Having.Expression
}
}
having, err := b.resolveTraceHaving(ctx, havingExpr, variables, sb)
if err != nil {
// the HAVING is a plain text rewrite, so substitute variables here
if strings.TrimSpace(fp.havingExpr) != "" && len(variables) > 0 {
replaced, err := qbvariables.ReplaceVariablesInExpression(fp.havingExpr, variables)
if err != nil {
return fp, err
}
fp.havingExpr = replaced
}
if err := validateAggregateFilter(fp.havingExpr, orderableSet); err != nil {
return fp, err
}
fp.having = having
return fp, nil
}
@@ -518,8 +473,8 @@ func (b *scopedTraceStatementBuilder) resolveSpanPredicate(ctx context.Context,
// span filter + HAVING + ORDER BY + LIMIT/OFFSET, selecting only the aliases ORDER BY
// / HAVING reference. Expressions carry $n markers bound to sb, so each can appear
// several times and every occurrence resolves to the same arg.
func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuilder, start, end, startBucket, endBucket uint64, resolved []resolvedColumn, orders []listOrder, maskExpr string, fp filterParts, resourcePred string, limit, offset int) (string, []any) {
needed := neededMatchedAliases(orders, fp.having)
func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuilder, start, end, startBucket, endBucket uint64, resolved []resolvedColumn, orders []listOrder, orderableSet map[string]struct{}, maskExpr string, fp filterParts, resourcePred string, limit, offset int) (string, []any, error) {
needed := neededMatchedAliases(orders, fp.havingExpr, orderableSet)
selects := []string{"trace_id"}
for _, rc := range resolved {
if _, ok := needed[rc.alias]; !ok {
@@ -556,8 +511,22 @@ func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuild
having = append(having, "countIf("+maskExpr+") > 0")
having = append(having, "countIf("+fp.spanPred+") > 0")
}
if fp.having != nil {
having = append(having, fp.having.pred)
if strings.TrimSpace(fp.havingExpr) != "" {
// the rewriter matches raw key text, so map the trace. form alongside the bare name
columnMap := make(map[string]string, len(orderableSet)*2)
for a := range orderableSet {
columnMap[a] = quoteAlias(a)
columnMap[telemetrytypes.FieldContextTrace.StringValue()+"."+a] = quoteAlias(a)
}
hv, err := querybuilder.NewHavingExpressionRewriter().Rewrite(fp.havingExpr, columnMap)
if err != nil {
return "", nil, err
}
if hv != "" {
// escape user text so a literal $ isn't read as an arg marker; the countIf
// entries hold live $n markers and must stay unescaped
having = append(having, sqlbuilder.Escape(hv))
}
}
if len(having) > 0 {
sb.Having(strings.Join(having, " AND "))
@@ -570,7 +539,7 @@ func (b *scopedTraceStatementBuilder) buildMatchedCTE(sb *sqlbuilder.SelectBuild
}
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("matched AS (%s)", sql), args
return fmt.Sprintf("matched AS (%s)", sql), args, nil
}
// buildRankedCTE builds `ranked`: [start,end] bounds per matched trace from the
@@ -611,9 +580,8 @@ func (b *scopedTraceStatementBuilder) buildEnrichmentSelect(sb *sqlbuilder.Selec
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
// aggregateAliasSet recognises trace-level keys, display-only aliases included so one gets
// a targeted error rather than falling through as a span attribute; orderableColumnSet is
// what a predicate may actually use. SpanLevel columns are filtered span-level, so skip them.
// aggregateAliasSet is every trace-level column alias, used to classify filter keys;
// SpanLevel columns are filtered span-level, so skip them.
func (b *scopedTraceStatementBuilder) aggregateAliasSet() map[string]struct{} {
set := make(map[string]struct{}, len(b.scope.Columns))
for _, c := range b.scope.Columns {
@@ -624,36 +592,59 @@ func (b *scopedTraceStatementBuilder) aggregateAliasSet() map[string]struct{} {
return set
}
// orderableAliasSet is the subset of aliases computable in the matched pass.
func orderableAliasSet(resolved []resolvedColumn) map[string]struct{} {
set := make(map[string]struct{})
for _, rc := range resolved {
if rc.orderable {
set[rc.alias] = struct{}{}
}
}
return set
}
// neededMatchedAliases is the minimal alias set the matched pass must select: those
// in ORDER BY plus those the resolved trace-level HAVING touches.
func neededMatchedAliases(orders []listOrder, having *traceHaving) map[string]struct{} {
// in ORDER BY plus those in the aggregate HAVING.
func neededMatchedAliases(orders []listOrder, havingExpr string, orderableSet map[string]struct{}) map[string]struct{} {
needed := make(map[string]struct{})
for _, o := range orders {
needed[o.alias] = struct{}{}
}
if having != nil {
for name := range having.used {
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; ok {
needed[name] = struct{}{}
}
}
return needed
}
// validateAggregateFilter rejects trace-level filters on aggregates not computable in
// the matched pass (e.g. span_count) with a targeted top-level error; inside the
// where-clause visitor it would surface only as a detail of a combined error. Only
// unspecified- and trace-context selectors name aggregates.
// traceAggregateNames extracts the aggregate names a trace-level HAVING references;
// only unspecified- and trace-context selectors name aggregates.
func traceAggregateNames(havingExpr string) []string {
var names []string
for _, sel := range querybuilder.QueryStringToKeysSelectors(havingExpr) {
if sel.FieldContext == telemetrytypes.FieldContextUnspecified || sel.FieldContext == telemetrytypes.FieldContextTrace {
names = append(names, sel.Name)
}
}
return names
}
// validateAggregateFilter rejects a trace-level filter referencing an aggregate not
// computable in the matched pass.
func validateAggregateFilter(havingExpr string, orderableSet map[string]struct{}) error {
if strings.TrimSpace(havingExpr) == "" {
return nil
}
for _, sel := range querybuilder.QueryStringToKeysSelectors(havingExpr) {
if sel.FieldContext != telemetrytypes.FieldContextUnspecified && sel.FieldContext != telemetrytypes.FieldContextTrace {
continue
}
if _, ok := orderableSet[sel.Name]; !ok {
allowed := make([]string, 0, len(orderableSet))
for a := range orderableSet {
allowed = append(allowed, a)
}
sort.Strings(allowed)
for _, name := range traceAggregateNames(havingExpr) {
if _, ok := orderableSet[name]; !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregate %q cannot be used in a trace-level filter; filterable aggregates: %s", sel.Name, strings.Join(sortedAliases(orderableSet), ", "))
"aggregate %q cannot be used in the trace-list filter; filterable aggregates: %s", name, strings.Join(allowed, ", "))
}
}
return nil

View File

@@ -1,808 +0,0 @@
package scopedtracesstatementbuilder
import (
"context"
"fmt"
"sort"
"strings"
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
// Scalar / time-series for scoped-trace queries. The `trace.` prefix picks the
// domain per expression: span-level (bare keys) delegates to the standard trace
// builder with the gate ANDed in; trace-level aggregates window-clipped per-trace
// values through the native pipeline (buildTraceAggregationQuery):
//
// __qualified traces whose whole-window aggregates satisfy the trace-level
// │ filter part; present only when the filter has one.
// ▼
// __scoped_traces per-trace values: windowed, mask-pruned GROUP BY trace_id
// │ (+ ts bucket for time series, + group-by columns). Columns
// ▼ off a trace slice are NULL and skipped by outer aggregates.
// main outer aggregation over the per-trace rows → __result_i.
//
// These per-trace values are window-clipped and span-filtered, unlike the list's
// enrichment pass over every span of the whole trace, so a column reads differently in each.
// traceAggregation is one aggregation rewritten to run over the per-trace scan.
type traceAggregation struct {
expr string // rewritten SQL over the per-trace column aliases
used map[string]struct{} // per-trace aliases referenced
isRate bool
}
// buildAggregation routes scalar/time-series requests by aggregation domain.
func (b *scopedTraceStatementBuilder) buildAggregation(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
traceAggs, err := b.classifyAggregations(query.Aggregations)
if err != nil {
return nil, err
}
if err := b.validateGroupBy(query); err != nil {
return nil, err
}
if len(traceAggs) == 0 {
return b.buildDelegatedAggregation(ctx, orgID, start, end, requestType, query, variables)
}
return b.buildTraceAggregationQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), requestType, query, variables, traceAggs)
}
// classifyAggregations returns the rewritten trace-domain aggregations, nil when all
// are span-domain; mixing the two domains is rejected.
func (b *scopedTraceStatementBuilder) classifyAggregations(aggs []qbtypes.TraceAggregation) ([]traceAggregation, error) {
// permission, not recognition: unknown names are reported against exactly this set
traceCols := b.orderableColumnSet()
var out []traceAggregation
spanCount := 0
for _, agg := range aggs {
ta, isTrace, err := rewriteTraceAggregation(agg.Expression, traceCols)
if err != nil {
return nil, err
}
if isTrace {
out = append(out, *ta)
} else {
spanCount++
}
}
if len(out) > 0 && spanCount > 0 {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"span-level and trace-level (trace.) aggregations cannot be mixed in one query")
}
return out, nil
}
// orderableColumnSet is what a trace-level aggregation or filter predicate may use;
// recognising a key as trace-level is aggregateAliasSet's job.
func (b *scopedTraceStatementBuilder) orderableColumnSet() map[string]struct{} {
set := make(map[string]struct{})
for _, c := range b.scope.Columns {
if c.Orderable {
set[c.Alias] = struct{}{}
}
}
return set
}
// validateGroupBy rejects trace-level columns as group-by keys with a targeted error
// (not the field mapper's generic "field not found"). Order keys need no check here:
// request validation only admits group keys and aggregation aliases/expressions.
func (b *scopedTraceStatementBuilder) validateGroupBy(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) error {
// recognition, not permission: a display-only alias must be named here to be rejected
// rather than reaching the field mapper as a span attribute
aliases := b.aggregateAliasSet()
for _, gb := range query.GroupBy {
key := gb.TelemetryFieldKey
key.Normalize()
// a bare name may be a span column sharing the alias (duration_nano, timestamp)
if key.FieldContext != telemetrytypes.FieldContextTrace {
continue
}
if _, ok := aliases[key.Name]; ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"grouping by trace-level aggregate %q is not supported; group by span attributes instead (e.g. service.name)", gb.Name)
}
}
return nil
}
// rewriteTraceAggregation rewrites an aggregation over trace.-prefixed columns to run
// on the per-trace scan (trace.output_tokens → output_tokens, functions mapped via
// AggreFuncMap); a pure span-level expression returns isTrace=false for the delegate.
func rewriteTraceAggregation(expr string, traceCols map[string]struct{}) (*traceAggregation, bool, error) {
p := chparser.NewParser("SELECT " + expr)
stmts, err := p.ParseStmts()
if err != nil {
return nil, false, errors.WrapInvalidInputf(err, errors.CodeInvalidInput, "failed to parse aggregation expression %q", expr)
}
if len(stmts) == 0 {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid aggregation expression %q", expr)
}
sel, ok := stmts[0].(*chparser.SelectQuery)
if !ok || len(sel.SelectItems) == 0 {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid aggregation expression %q", expr)
}
v := &traceAggVisitor{traceCols: traceCols, used: make(map[string]struct{})}
if err := sel.SelectItems[0].Accept(v); err != nil {
return nil, false, err
}
if !v.hasTrace {
return nil, false, nil
}
if v.hasSpan {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregation %q mixes trace-level (trace.) and span-level columns; use one domain per aggregation", expr)
}
// the interval divides the rendered expression as a whole, so a second aggregation
// alongside the rate would be divided too
if v.isRate && v.aggCount > 1 {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregation %q combines a rate with another aggregation; the rate interval would divide both, so give each its own aggregation", expr)
}
return &traceAggregation{expr: chparser.Format(sel.SelectItems[0]), used: v.used, isRate: v.isRate}, true, nil
}
// traceAggVisitor classifies column references and rewrites trace.-prefixed ones in
// place; the ancestor stack tells a column identifier from a path segment, function
// name, or alias, and rejects trace. columns inside *If combinators.
type traceAggVisitor struct {
chparser.DefaultASTVisitor
traceCols map[string]struct{}
used map[string]struct{}
stack []chparser.Expr
aggCount int
hasTrace bool
hasSpan bool
isRate bool
}
func (v *traceAggVisitor) Enter(expr chparser.Expr) { v.stack = append(v.stack, expr) }
func (v *traceAggVisitor) Leave(expr chparser.Expr) { v.stack = v.stack[:len(v.stack)-1] }
// parent is the node enclosing the one currently being visited (the visited node
// itself is the stack top).
func (v *traceAggVisitor) parent() chparser.Expr {
if len(v.stack) < 2 {
return nil
}
return v.stack[len(v.stack)-2]
}
// enclosingCombinator returns the name of a surrounding *If-combinator function, if any.
func (v *traceAggVisitor) enclosingCombinator() (string, bool) {
for _, e := range v.stack {
fn, ok := e.(*chparser.FunctionExpr)
if !ok {
continue
}
if agg, known := querybuilder.AggreFuncMap[valuer.NewString(strings.ToLower(fn.Name.Name))]; known && agg.FuncCombinator {
return fn.Name.Name, true
}
}
return "", false
}
// enclosingAggregate walks the ancestor stack; AggreFuncMap holds only aggregates and
// VisitFunctionExpr rejects any name missing from it, so a known name is enough.
func (v *traceAggVisitor) enclosingAggregate() bool {
for _, e := range v.stack {
fn, ok := e.(*chparser.FunctionExpr)
if !ok {
continue
}
if _, known := querybuilder.AggreFuncMap[valuer.NewString(strings.ToLower(fn.Name.Name))]; known {
return true
}
}
return false
}
// VisitPath classifies a dotted reference (trace.output_tokens); trace-level ones are
// rewritten in place to the bare per-trace alias.
func (v *traceAggVisitor) VisitPath(p *chparser.Path) error {
col, isTrace := traceColumnFromPath(p)
if !isTrace {
v.hasSpan = true
return nil
}
if err := v.acceptTraceColumn(chparser.Format(p), col); err != nil {
return err
}
p.Fields = p.Fields[len(p.Fields)-1:]
p.Fields[0].Name = col
return nil
}
// VisitIdent classifies a plain identifier (a backquoted `trace.output_tokens` is
// trace-level); path segments, function names, and aliases are structural, not columns.
func (v *traceAggVisitor) VisitIdent(i *chparser.Ident) error {
switch parent := v.parent().(type) {
case *chparser.Path:
return nil // segments are classified whole by VisitPath
case *chparser.FunctionExpr:
if parent.Name == i {
return nil
}
case *chparser.ColumnExpr:
if parent.Alias == i {
return nil
}
}
// the parser hands us one identifier, so the prefix is the only text to cut here
col, isTrace := strings.CutPrefix(i.Name, telemetrytypes.FieldContextTrace.StringValue()+".")
if !isTrace || col == "" {
v.hasSpan = true
return nil
}
if err := v.acceptTraceColumn(i.Name, col); err != nil {
return err
}
i.Name = col
return nil
}
// acceptTraceColumn validates one trace-level column reference and records it.
func (v *traceAggVisitor) acceptTraceColumn(ref, col string) error {
if name, in := v.enclosingCombinator(); in {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"%q over trace-level (trace.) columns is not supported; put the trace-level condition in the filter expression instead", name)
}
// trace_id is always selected by the per-trace scan (count(trace.trace_id)
// counts traces); everything else must be a scope column.
if col != "trace_id" {
if _, known := v.traceCols[col]; !known {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"unknown trace-level aggregation column %q; usable columns: %s", ref, strings.Join(sortedAliases(v.traceCols), ", "))
}
v.used[col] = struct{}{}
}
// ungrouped, a bare per-trace column would make the outer SELECT emit one row per
// trace instead of one aggregated row
if !v.enclosingAggregate() {
return errors.NewInvalidInputf(errors.CodeInvalidInput,
"trace-level column %q must be inside an aggregation function (e.g. avg(%s))", ref, ref)
}
v.hasTrace = true
return nil
}
// VisitFunctionExpr validates and maps the function name. Children were already
// visited (post-order), so classification is complete for this subtree.
func (v *traceAggVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
name := strings.ToLower(fn.Name.Name)
aggFunc, ok := querybuilder.AggreFuncMap[valuer.NewString(name)]
if !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unrecognized function: %s", name)
}
if fn.Params != nil && fn.Params.Items != nil && len(fn.Params.Items.Items) > 0 && aggFunc.FuncCombinator {
// combinator predicates over span columns stay span-level (countIf(has_error=true))
v.hasSpan = true
return nil
}
fn.Name.Name = aggFunc.FuncName
v.aggCount++
if aggFunc.Rate {
v.isRate = true
}
return nil
}
// traceColumnFromPath returns the per-trace column a dotted reference names
// (trace.output_tokens -> output_tokens, trace.a.b -> a.b).
func traceColumnFromPath(p *chparser.Path) (string, bool) {
if len(p.Fields) < 2 || p.Fields[0].Name != telemetrytypes.FieldContextTrace.StringValue() {
return "", false
}
segments := make([]string, 0, len(p.Fields)-1)
for _, f := range p.Fields[1:] {
segments = append(segments, f.Name)
}
return strings.Join(segments, "."), true
}
func sortedAliases(set map[string]struct{}) []string {
out := make([]string, 0, len(set))
for a := range set {
out = append(out, a)
}
sort.Strings(out)
return out
}
// ---------------------------------------------------------------------------
// Qualification + per-trace scan
// ---------------------------------------------------------------------------
// buildQualifiedStatement builds the delegate's __trace_scope: trace ids whose
// window-clipped aggregates satisfy the trace-level filter, resource-pruned inline
// (the caller embeds it standalone). start/end are ns; nil when every condition was
// dropped by variable resolution.
func (b *scopedTraceStatementBuilder) buildQualifiedStatement(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
traceExpr string,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
keys, err := b.fetchKeys(ctx, orgID)
if err != nil {
return nil, err
}
sb := sqlbuilder.NewSelectBuilder()
maskExpr, resolved, err := b.resolveFor(ctx, orgID, start, end, keys, sb)
if err != nil {
return nil, err
}
having, err := b.resolveTraceHaving(ctx, traceExpr, variables, sb)
if err != nil {
return nil, err
}
if having == nil {
return nil, nil //nolint:nilnil
}
var resourcePred string
// nil when the filter has no resource-attribute conditions
if stmt, err := b.resourceFilterStmtBuilder.Build(ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables); err != nil {
return nil, err
} else if stmt != nil {
inlined, err := embedExpr(sb, stmt.Query, stmt.Args)
if err != nil {
return nil, err
}
resourcePred = fmt.Sprintf("resource_fingerprint GLOBAL IN (SELECT fingerprint FROM (%s))", inlined)
}
sql, args := b.buildPerTraceScan(sb, start, end, resolved, maskExpr, perTraceScanOpts{
needed: having.used,
havingPred: having.pred,
resourcePred: resourcePred,
})
return &qbtypes.Statement{Query: sql, Args: args}, nil
}
// embedExpr inlines a pre-built statement into sb, replacing each `?` with a builder
// Var; a count mismatch would silently shift args into the wrong slots, so error out.
func embedExpr(sb *sqlbuilder.SelectBuilder, expr string, args []any) (string, error) {
if n := strings.Count(expr, "?"); n != len(args) {
return "", errors.NewInternalf(errors.CodeInternal,
"scoped trace builder: %d placeholders != %d args embedding %q", n, len(args), expr)
}
var out strings.Builder
ai := 0
for i := 0; i < len(expr); i++ {
if expr[i] == '?' {
out.WriteString(sb.Var(args[ai]))
ai++
continue
}
out.WriteByte(expr[i])
}
return out.String(), nil
}
// groupColumn is a resolved span-attribute group-by column (arg-free expression).
type groupColumn struct {
alias string
expr string
}
// groupByColumnAlias prefixes the i-th group-by dimension so the alias cannot shadow the
// span column its expression reads; the querier (stripKeyAlias) strips it back off.
func groupByColumnAlias(i int, name string) string {
return fmt.Sprintf("__GROUP_BY_KEY_%d_%s", i, name)
}
// orderColumn is the SQL identifier a non-aggregation order key sorts by: the
// positional alias when the key names a group-by dimension, else the key itself.
func orderColumn(orderKey string, groupBy []qbtypes.GroupByKey) string {
for i := range groupBy {
if groupBy[i].Name == orderKey {
return groupByColumnAlias(i, groupBy[i].Name)
}
}
return orderKey
}
// perTraceScanOpts parametrize one windowed, mask-pruned GROUP BY trace_id scan.
// All expressions are already resolved against the scan's builder.
type perTraceScanOpts struct {
stepSeconds int64 // >0 → bucket per-trace values by time (ts column)
groupCols []groupColumn
needed map[string]struct{} // per-trace aliases to select
spanPred string // resolved span-level filter, ANDed per span
resourcePred string // resource-fingerprint prune (CTE reference or inline subquery)
qualified bool // constrain to __qualified
limitPred string // top-N group prune (GLOBAL IN __limit_cte)
havingPred string // resolved HAVING predicate over the selected aliases
}
// buildPerTraceScan renders the scan: window + gate mask (+ span filter, resource
// prune, qualification), grouped by trace_id (+ ts bucket, group-by columns).
func (b *scopedTraceStatementBuilder) buildPerTraceScan(sb *sqlbuilder.SelectBuilder, start, end uint64, resolved []resolvedColumn, maskExpr string, o perTraceScanOpts) (string, []any) {
startBucket := start/querybuilder.NsToSeconds - querybuilder.BucketAdjustment
endBucket := end / querybuilder.NsToSeconds
selects := []string{"trace_id"}
if o.stepSeconds > 0 {
selects = append(selects, fmt.Sprintf("toStartOfInterval(timestamp, INTERVAL %d SECOND) AS ts", o.stepSeconds))
}
for _, gc := range o.groupCols {
selects = append(selects, fmt.Sprintf("toString(%s) AS `%s`", gc.expr, gc.alias))
}
for _, rc := range resolved {
if _, ok := o.needed[rc.alias]; !ok {
continue
}
selects = append(selects, rc.expr+" AS "+quoteAlias(rc.alias))
}
sb.Select(selects...)
sb.From(fmt.Sprintf("%s.%s", tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName))
where := []string{
sb.GE("timestamp", fmt.Sprintf("%d", start)),
sb.L("timestamp", fmt.Sprintf("%d", end)),
sb.GE("ts_bucket_start", startBucket),
sb.LE("ts_bucket_start", endBucket),
maskExpr,
}
if strings.TrimSpace(o.spanPred) != "" {
where = append(where, o.spanPred)
}
if o.resourcePred != "" {
where = append(where, o.resourcePred)
}
if o.qualified {
where = append(where, "trace_id GLOBAL IN (SELECT trace_id FROM __qualified)")
}
if o.limitPred != "" {
where = append(where, o.limitPred)
}
sb.Where(where...)
groupBy := []string{"trace_id"}
if o.stepSeconds > 0 {
groupBy = append(groupBy, "ts")
}
for _, gc := range o.groupCols {
groupBy = append(groupBy, "`"+gc.alias+"`")
}
sb.GroupBy(groupBy...)
if strings.TrimSpace(o.havingPred) != "" {
sb.Having(o.havingPred)
}
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
// resolveGroupColumns resolves span-attribute group-by keys through the field mapper
// (metadata-aware), for selection inside the per-trace scan.
func (b *scopedTraceStatementBuilder) resolveGroupColumns(ctx context.Context, orgID valuer.UUID, start, end uint64, groupBy []qbtypes.GroupByKey) ([]groupColumn, error) {
if len(groupBy) == 0 {
return nil, nil
}
selectors := make([]*telemetrytypes.FieldKeySelector, 0, len(groupBy))
for i := range groupBy {
selectors = append(selectors, &telemetrytypes.FieldKeySelector{
Name: groupBy[i].Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: groupBy[i].FieldContext,
FieldDataType: groupBy[i].FieldDataType,
SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact,
})
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
if err != nil {
return nil, err
}
out := make([]groupColumn, 0, len(groupBy))
for i := range groupBy {
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &groupBy[i].TelemetryFieldKey, telemetrytypes.FieldDataTypeString, keys)
if err != nil {
return nil, err
}
out = append(out, groupColumn{alias: groupByColumnAlias(i, groupBy[i].Name), expr: sqlbuilder.Escape(expr)})
}
return out, nil
}
// ---------------------------------------------------------------------------
// Native trace-domain aggregation query
// ---------------------------------------------------------------------------
// scanContext is one per-scan resolution: a fresh builder with the mask, columns,
// span predicate, and optionally the trace-level HAVING resolved against it.
type scanContext struct {
sb *sqlbuilder.SelectBuilder
maskExpr string
resolved []resolvedColumn
spanPred string
having *traceHaving
warnings []string
warnURL string
}
// newScanContext resolves everything a per-trace scan embeds against a fresh builder.
func (b *scopedTraceStatementBuilder) newScanContext(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
spanExpr, traceExpr string,
variables map[string]qbtypes.VariableItem,
) (*scanContext, error) {
sc := &scanContext{sb: sqlbuilder.NewSelectBuilder()}
var err error
sc.maskExpr, sc.resolved, err = b.resolveFor(ctx, orgID, start, end, keys, sc.sb)
if err != nil {
return nil, err
}
if strings.TrimSpace(spanExpr) != "" {
pred, warns, url, err := b.resolveSpanPredicate(ctx, orgID, start, end, spanExpr, variables, sc.sb)
if err != nil {
return nil, err
}
sc.spanPred, sc.warnings, sc.warnURL = pred, warns, url
}
if strings.TrimSpace(traceExpr) != "" {
sc.having, err = b.resolveTraceHaving(ctx, traceExpr, variables, sc.sb)
if err != nil {
return nil, err
}
}
return sc, nil
}
// buildTraceAggregationQuery builds the native pipeline (see the file comment).
// start/end are ns.
func (b *scopedTraceStatementBuilder) buildTraceAggregationQuery(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
traceAggs []traceAggregation,
) (*qbtypes.Statement, error) {
keys, err := b.fetchKeys(ctx, orgID)
if err != nil {
return nil, err
}
var spanExpr, traceExpr string
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
// the broad set so a condition on a display-only alias still lands in the
// trace-level part, where resolveTraceHaving rejects it by name
spanExpr, traceExpr, err = querybuilder.SplitFilterForAggregates(query.Filter.Expression, b.aggregateAliasSet())
if err != nil {
return nil, err
}
}
resourceFrag, resourceArgs, resourcePred, err := b.maybeAttachResourceFilter(ctx, orgID, query, start, end, variables)
if err != nil {
return nil, err
}
var cteFragments []string
var cteArgs [][]any
if resourceFrag != "" {
cteFragments = append(cteFragments, resourceFrag)
cteArgs = append(cteArgs, resourceArgs)
}
// __qualified: its own scan resolution, HAVING = the trace-level filter part
qualified := false
if strings.TrimSpace(traceExpr) != "" {
qsc, err := b.newScanContext(ctx, orgID, start, end, keys, "", traceExpr, variables)
if err != nil {
return nil, err
}
if qsc.having != nil {
qsql, qargs := b.buildPerTraceScan(qsc.sb, start, end, qsc.resolved, qsc.maskExpr, perTraceScanOpts{
needed: qsc.having.used,
havingPred: qsc.having.pred,
resourcePred: resourcePred,
})
cteFragments = append(cteFragments, fmt.Sprintf("__qualified AS (%s)", qsql))
cteArgs = append(cteArgs, qargs)
qualified = true
}
}
groupCols, err := b.resolveGroupColumns(ctx, orgID, start, end, query.GroupBy)
if err != nil {
return nil, err
}
groupNames := make([]string, 0, len(groupCols))
for _, gc := range groupCols {
groupNames = append(groupNames, "`"+gc.alias+"`")
}
needed := make(map[string]struct{})
for _, ta := range traceAggs {
for a := range ta.used {
needed[a] = struct{}{}
}
}
stepSeconds := int64(0)
rateInterval := (end - start) / querybuilder.NsToSeconds
if requestType == qbtypes.RequestTypeTimeSeries {
stepSeconds = int64(query.StepInterval.Seconds())
rateInterval = uint64(stepSeconds)
}
// outer aggregation over the per-trace rows
sb := sqlbuilder.NewSelectBuilder()
selects := []string{}
if stepSeconds > 0 {
selects = append(selects, "ts")
}
selects = append(selects, groupNames...)
for i, ta := range traceAggs {
selects = append(selects, fmt.Sprintf("%s AS __result_%d", ta.rendered(rateInterval), i))
}
sb.Select(selects...)
sb.From("__scoped_traces")
// grouped, limited time series → rank groups on whole-window per-trace values
// (exact for non-composable aggregates) and prune the main scan to the top-N.
limitPred := ""
if requestType == qbtypes.RequestTypeTimeSeries && query.Limit > 0 && len(groupCols) > 0 {
tsc, err := b.newScanContext(ctx, orgID, start, end, keys, spanExpr, "", variables)
if err != nil {
return nil, err
}
totalSQL, totalArgs := b.buildPerTraceScan(tsc.sb, start, end, tsc.resolved, tsc.maskExpr, perTraceScanOpts{
groupCols: groupCols,
needed: needed,
spanPred: tsc.spanPred,
resourcePred: resourcePred,
qualified: qualified,
})
cteFragments = append(cteFragments, fmt.Sprintf("__scoped_traces_total AS (%s)", totalSQL))
cteArgs = append(cteArgs, totalArgs)
limitSQL, limitArgs := outerLimitSQL(query, traceAggs, groupNames, (end-start)/querybuilder.NsToSeconds)
cteFragments = append(cteFragments, fmt.Sprintf("__limit_cte AS (%s)", limitSQL))
cteArgs = append(cteArgs, limitArgs)
exprs := make([]string, 0, len(groupCols))
for _, gc := range groupCols {
exprs = append(exprs, "toString("+gc.expr+")")
}
limitPred = fmt.Sprintf("(%s) GLOBAL IN (SELECT %s FROM __limit_cte)",
strings.Join(exprs, ", "), strings.Join(groupNames, ", "))
}
msc, err := b.newScanContext(ctx, orgID, start, end, keys, spanExpr, "", variables)
if err != nil {
return nil, err
}
perTraceSQL, perTraceArgs := b.buildPerTraceScan(msc.sb, start, end, msc.resolved, msc.maskExpr, perTraceScanOpts{
stepSeconds: stepSeconds,
groupCols: groupCols,
needed: needed,
spanPred: msc.spanPred,
resourcePred: resourcePred,
qualified: qualified,
limitPred: limitPred,
})
cteFragments = append(cteFragments, fmt.Sprintf("__scoped_traces AS (%s)", perTraceSQL))
cteArgs = append(cteArgs, perTraceArgs)
groupBys := []string{}
if stepSeconds > 0 {
groupBys = append(groupBys, "ts")
}
groupBys = append(groupBys, groupNames...)
if len(groupBys) > 0 {
sb.GroupBy(groupBys...)
}
if query.Having != nil && strings.TrimSpace(query.Having.Expression) != "" {
rewritten, err := querybuilder.NewHavingExpressionRewriter().RewriteForTraces(query.Having.Expression, query.Aggregations)
if err != nil {
return nil, err
}
sb.Having(sqlbuilder.Escape(rewritten))
}
if requestType == qbtypes.RequestTypeTimeSeries {
if len(query.Order) != 0 {
for _, orderBy := range query.Order {
if _, ok := traceAggOrderIndex(orderBy, query); !ok {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderColumn(orderBy.Key.Name, query.GroupBy), orderBy.Direction.StringValue()))
}
}
sb.OrderBy("ts desc")
}
} else {
for _, orderBy := range query.Order {
if idx, ok := traceAggOrderIndex(orderBy, query); ok {
sb.OrderBy(fmt.Sprintf("__result_%d %s", idx, orderBy.Direction.StringValue()))
} else {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderColumn(orderBy.Key.Name, query.GroupBy), orderBy.Direction.StringValue()))
}
}
if len(query.Order) == 0 {
sb.OrderBy("__result_0 DESC")
}
if query.Limit > 0 {
sb.Limit(query.Limit)
}
}
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
finalSQL := querybuilder.CombineCTEs(cteFragments) + mainSQL + " SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000"
finalArgs := querybuilder.PrependArgs(cteArgs, mainArgs)
return &qbtypes.Statement{
Query: finalSQL,
Args: finalArgs,
Warnings: msc.warnings,
WarningsDocURL: msc.warnURL,
}, nil
}
// rendered returns the outer aggregation SQL, dividing rate aggregations by the
// interval (step for time series, window length for scalar); the divisor applies to the
// whole expression, which holds only because a rate must be the sole aggregation.
func (ta traceAggregation) rendered(rateInterval uint64) string {
if ta.isRate {
return fmt.Sprintf("%s/%d", ta.expr, rateInterval)
}
return ta.expr
}
// outerLimitSQL renders the top-N group selection for a grouped, limited time series:
// outer aggregations over whole-window per-trace values, ranked and limited.
func outerLimitSQL(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], traceAggs []traceAggregation, groupNames []string, windowSeconds uint64) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
selects := append([]string{}, groupNames...)
for i, ta := range traceAggs {
selects = append(selects, fmt.Sprintf("%s AS __result_%d", ta.rendered(windowSeconds), i))
}
sb.Select(selects...)
sb.From("__scoped_traces_total")
sb.GroupBy(groupNames...)
for _, orderBy := range query.Order {
if idx, ok := traceAggOrderIndex(orderBy, query); ok {
sb.OrderBy(fmt.Sprintf("__result_%d %s", idx, orderBy.Direction.StringValue()))
} else {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderColumn(orderBy.Key.Name, query.GroupBy), orderBy.Direction.StringValue()))
}
}
if len(query.Order) == 0 {
sb.OrderBy("__result_0 DESC")
}
sb.Limit(query.Limit)
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
// traceAggOrderIndex reports whether an order key refers to the i-th aggregation
// (by alias, expression, or index), mirroring the trace builder.
func traceAggOrderIndex(k qbtypes.OrderBy, q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) (int, bool) {
for i, agg := range q.Aggregations {
if k.Key.Name == agg.Alias ||
k.Key.Name == agg.Expression ||
k.Key.Name == fmt.Sprintf("%d", i) {
return i, true
}
}
return 0, false
}

View File

@@ -1,75 +0,0 @@
package scopedtracesstatementbuilder
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestRewriteTraceAggregation(t *testing.T) {
cols := map[string]struct{}{
"input_tokens": {}, "output_tokens": {}, "total_tokens": {}, "llm_call_count": {}, "max_llm_latency_ns": {},
}
cases := []struct {
name string
expr string
isTrace bool
want string // rewritten expr, only checked when isTrace
used []string
wantErr string
}{
{name: "avg trace col", expr: "avg(trace.output_tokens)", isTrace: true, want: "avg(output_tokens)", used: []string{"output_tokens"}},
{name: "sum trace col", expr: "sum(trace.total_tokens)", isTrace: true, want: "sum(total_tokens)", used: []string{"total_tokens"}},
{name: "count traces", expr: "count(trace.trace_id)", isTrace: true, want: "count(trace_id)"},
{name: "p90 trace col", expr: "p90(trace.max_llm_latency_ns)", isTrace: true, want: "quantile(0.90)(max_llm_latency_ns)", used: []string{"max_llm_latency_ns"}},
{name: "arithmetic between trace cols", expr: "avg(trace.output_tokens + trace.input_tokens)", isTrace: true, want: "avg(output_tokens + input_tokens)", used: []string{"output_tokens", "input_tokens"}},
{name: "arithmetic with constant", expr: "sum(trace.output_tokens * 1.5)", isTrace: true, want: "sum(output_tokens * 1.5)", used: []string{"output_tokens"}},
{name: "ratio of two aggregations", expr: "sum(trace.output_tokens)/count(trace.trace_id)", isTrace: true, want: "sum(output_tokens) / count(trace_id)", used: []string{"output_tokens"}},
{name: "backquoted trace col", expr: "avg(`trace.output_tokens`)", isTrace: true, want: "avg(`output_tokens`)", used: []string{"output_tokens"}},
{name: "bare count is span-level", expr: "count()", isTrace: false},
{name: "span attribute is span-level", expr: "sum(gen_ai.usage.output_tokens)", isTrace: false},
{name: "countIf span predicate is span-level", expr: "countIf(has_error = true)", isTrace: false},
{name: "mixed domains in one expression", expr: "sum(trace.output_tokens) + sum(gen_ai.usage.input_tokens)", wantErr: "mixes trace-level"},
{name: "mixed domains in one function", expr: "sum(trace.output_tokens + gen_ai.usage.input_tokens)", wantErr: "mixes trace-level"},
{name: "output-only column rejected", expr: "avg(trace.span_count)", wantErr: "unknown trace-level aggregation column"},
{name: "unknown column rejected", expr: "avg(trace.bogus)", wantErr: "unknown trace-level aggregation column"},
// a dotted column keeps every segment after the prefix, so it is reported whole
{name: "multi segment column rejected by full name", expr: "avg(trace.service.name)", wantErr: `"trace.service.name"`},
{name: "bare trace identifier is span-level", expr: "avg(trace)", isTrace: false},
{name: "countIf over trace col rejected", expr: "countIf(trace.output_tokens > 1000)", wantErr: "not supported"},
{name: "bare trace col rejected", expr: "trace.output_tokens", wantErr: "must be inside an aggregation function"},
{name: "backquoted bare trace col rejected", expr: "`trace.output_tokens`", wantErr: "must be inside an aggregation function"},
{name: "bare trace_id rejected", expr: "trace.trace_id", wantErr: "must be inside an aggregation function"},
{name: "arithmetic outside an aggregation rejected", expr: "trace.output_tokens + trace.input_tokens", wantErr: "must be inside an aggregation function"},
{name: "trace col beside an aggregation rejected", expr: "sum(trace.output_tokens) + trace.input_tokens", wantErr: "must be inside an aggregation function"},
{name: "aggregation scaled by a constant", expr: "sum(trace.output_tokens) * 2", isTrace: true, want: "sum(output_tokens) * 2", used: []string{"output_tokens"}},
{name: "rate over traces", expr: "rate(trace.trace_id)", isTrace: true, want: "count(trace_id)"},
{name: "rate_sum trace col", expr: "rate_sum(trace.output_tokens)", isTrace: true, want: "sum(output_tokens)", used: []string{"output_tokens"}},
// the interval divides the whole rendered expression, so a second aggregation
// alongside a rate would be divided too
{name: "rate mixed with another aggregation rejected", expr: "rate(trace.trace_id) + avg(trace.output_tokens)", wantErr: "combines a rate with another aggregation"},
{name: "ratio of two rates rejected", expr: "rate_sum(trace.output_tokens)/rate_sum(trace.input_tokens)", wantErr: "combines a rate with another aggregation"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
ta, isTrace, err := rewriteTraceAggregation(tc.expr, cols)
if tc.wantErr != "" {
require.ErrorContains(t, err, tc.wantErr)
return
}
require.NoError(t, err)
assert.Equal(t, tc.isTrace, isTrace)
if !tc.isTrace {
return
}
assert.Equal(t, tc.want, ta.expr)
for _, u := range tc.used {
assert.Contains(t, ta.used, u)
}
assert.Len(t, ta.used, len(tc.used))
})
}
}

View File

@@ -1,146 +0,0 @@
package scopedtracesstatementbuilder
import (
"context"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
qbvariables "github.com/SigNoz/signoz/pkg/variables"
"github.com/huandu/go-sqlbuilder"
)
// traceHaving is the resolved trace-level filter part: a HAVING predicate over the
// per-trace aliases plus the aliases it references (so scans select only those).
type traceHaving struct {
pred string
used map[string]struct{}
}
// resolveTraceHaving resolves a trace-level filter through the standard pipeline
// (variable replacement, then PrepareWhereClause against the per-trace aliases), so
// operators, bound args, and __all__ behave exactly as in span filters. Returns nil
// when the expression is empty or every condition was dropped; args bind into sb.
func (b *scopedTraceStatementBuilder) resolveTraceHaving(ctx context.Context, expr string, variables map[string]qbtypes.VariableItem, sb *sqlbuilder.SelectBuilder) (*traceHaving, error) {
if strings.TrimSpace(expr) == "" {
return nil, nil //nolint:nilnil
}
// variables are replaced before validation so their literals are not mistaken for
// aggregate names; an unresolved $var is left in place and fails validation below
// (as an unknown aggregate — targeted variable errors are a separate concern)
if len(variables) > 0 {
replaced, err := qbvariables.ReplaceVariablesInExpression(expr, variables)
if err != nil {
return nil, err
}
expr = replaced
if strings.TrimSpace(expr) == "" {
return nil, nil //nolint:nilnil
}
}
allowed := b.orderableColumnSet()
// upfront targeted errors; the visitor folds them into a combined "Found N errors"
if err := validateAggregateFilter(expr, allowed); err != nil {
return nil, err
}
// both spellings resolve here: the key parser strips the trace. prefix into
// FieldContextTrace, which matches this entry's context
fieldKeys := make(map[string][]*telemetrytypes.TelemetryFieldKey, len(allowed))
for alias := range allowed {
key := &telemetrytypes.TelemetryFieldKey{Name: alias, FieldContext: telemetrytypes.FieldContextTrace}
fieldKeys[alias] = []*telemetrytypes.TelemetryFieldKey{key}
}
cb := &aliasConditionBuilder{allowed: allowed, used: make(map[string]struct{})}
prepared, err := querybuilder.PrepareWhereClause(expr, querybuilder.FilterExprVisitorOpts{
Context: ctx,
Logger: b.logger,
ConditionBuilder: cb,
FieldKeys: fieldKeys,
Variables: variables,
Builder: sb,
})
if err != nil {
return nil, err
}
if prepared.IsEmpty() {
return nil, nil //nolint:nilnil
}
return &traceHaving{pred: prepared.Expr, used: cb.used}, nil
}
// aliasConditionBuilder renders filter conditions directly against the per-trace
// aliases, recording the ones it touches; a key resolving to no alias is an error.
type aliasConditionBuilder struct {
allowed map[string]struct{}
used map[string]struct{}
}
var _ qbtypes.ConditionBuilder = (*aliasConditionBuilder)(nil)
func (c *aliasConditionBuilder) ConditionFor(
_ context.Context,
_ valuer.UUID,
_, _ uint64,
key *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
_ qbtypes.ConditionBuilderOptions,
op qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
matching := keys[key.Name]
if len(matching) == 0 {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"aggregate %q cannot be used in a trace-level filter; filterable aggregates: %s",
key.Name, strings.Join(sortedAliases(c.allowed), ", "))
}
alias := matching[0].Name
c.used[alias] = struct{}{}
col := quoteAlias(alias)
var cond string
switch op {
case qbtypes.FilterOperatorEqual:
cond = sb.E(col, value)
case qbtypes.FilterOperatorNotEqual:
cond = sb.NE(col, value)
case qbtypes.FilterOperatorGreaterThan:
cond = sb.G(col, value)
case qbtypes.FilterOperatorGreaterThanOrEq:
cond = sb.GE(col, value)
case qbtypes.FilterOperatorLessThan:
cond = sb.L(col, value)
case qbtypes.FilterOperatorLessThanOrEq:
cond = sb.LE(col, value)
case qbtypes.FilterOperatorIn, qbtypes.FilterOperatorNotIn:
values, ok := value.([]any)
if !ok {
values = []any{value}
}
if op == qbtypes.FilterOperatorIn {
cond = sb.In(col, values...)
} else {
cond = sb.NotIn(col, values...)
}
case qbtypes.FilterOperatorBetween, qbtypes.FilterOperatorNotBetween:
values, ok := value.([]any)
if !ok || len(values) != 2 {
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"between on trace-level aggregate %q requires exactly two values", alias)
}
if op == qbtypes.FilterOperatorBetween {
cond = sb.Between(col, values[0], values[1])
} else {
cond = sb.NotBetween(col, values[0], values[1])
}
default:
return nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput,
"trace-level aggregate %q supports only comparison operators (=, !=, <, <=, >, >=, in, between)", alias)
}
return []string{cond}, nil, nil
}

View File

@@ -32,9 +32,6 @@ type traceQueryStatementBuilder struct {
resourceFilterResolver *resourcefilter.ResourceFingerprintResolver[qbtypes.TraceAggregation]
aggExprRewriter qbtypes.AggExprRewriter
skipResourceFingerprintEnabled bool
// traceScope, set only on the per-call copy made by BuildTraceScoped, constrains
// queries to spans whose trace_id is in the __trace_scope CTE.
traceScope *qbtypes.Statement
}
var _ qbtypes.StatementBuilder[qbtypes.TraceAggregation] = (*traceQueryStatementBuilder)(nil)
@@ -98,33 +95,6 @@ func NewTraceQueryStatementBuilder(
}
}
// BuildTraceScoped is Build constrained to trace_ids selected by traceScope; the
// receiver is copied so the shared builder stays stateless.
func (b *traceQueryStatementBuilder) BuildTraceScoped(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
variables map[string]qbtypes.VariableItem,
traceScope *qbtypes.Statement,
) (*qbtypes.Statement, error) {
scoped := *b
scoped.traceScope = traceScope
return scoped.Build(ctx, orgID, start, end, requestType, query, variables)
}
// attachTraceScope adds the trace-scope condition to sb and returns the CTE fragment
// + args to prepend; both empty when no scope is set.
func (b *traceQueryStatementBuilder) attachTraceScope(sb *sqlbuilder.SelectBuilder) (string, []any) {
if b.traceScope == nil {
return "", nil
}
sb.Where("trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)")
return fmt.Sprintf("__trace_scope AS (%s)", b.traceScope.Query), b.traceScope.Args
}
// Build builds a SQL query for traces based on the given parameters.
func (b *traceQueryStatementBuilder) Build(
ctx context.Context,
@@ -549,11 +519,6 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
cteArgs = append(cteArgs, args)
}
if scopeFrag, scopeArgs := b.attachTraceScope(sb); scopeFrag != "" {
cteFragments = append(cteFragments, scopeFrag)
cteArgs = append(cteArgs, scopeArgs)
}
sb.SelectMore(fmt.Sprintf(
"toStartOfInterval(timestamp, INTERVAL %d SECOND) AS ts",
int64(query.StepInterval.Seconds()),
@@ -714,13 +679,6 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
cteArgs = append(cteArgs, args)
}
// skipResourceCTE means this scalar is embedded as a CTE of a time-series query,
// which has already emitted the __trace_scope fragment — add only the condition.
if scopeFrag, scopeArgs := b.attachTraceScope(sb); scopeFrag != "" && !skipResourceCTE {
cteFragments = append(cteFragments, scopeFrag)
cteArgs = append(cteArgs, scopeArgs)
}
allAggChArgs := []any{}
fieldNames := make([]string, 0, len(query.GroupBy))

View File

@@ -201,6 +201,18 @@ func (storableDashboardData *StorableDashboardData) GetWidgetIds() []string {
return widgetIds
}
// ErrIfNotDeletable gates deletion on the columns alone, never on Data, so a
// dashboard whose data is corrupt or stuck on the v1 schema stays deletable.
func (storable StorableDashboard) ErrIfNotDeletable() error {
if storable.Locked {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "cannot delete a locked dashboard, please unlock the dashboard to delete")
}
if !storable.Source.isUserDeletable() {
return errors.Newf(errors.TypeInvalidInput, ErrCodeDashboardImmutable, "%s dashboards cannot be deleted", storable.Source)
}
return nil
}
func (dashboard *Dashboard) ErrIfNotMutable() error {
if dashboard.Source == SourceIntegration {
return errors.Newf(errors.TypeInvalidInput, ErrCodeDashboardImmutable, "integration dashboards cannot be modified")

View File

@@ -4,6 +4,7 @@ import (
"context"
"testing"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
)
@@ -81,3 +82,64 @@ func TestCanUpdate_MultipleDeletions_ByDiff(t *testing.T) {
})
}
}
func TestStorableDashboardErrIfNotDeletable(t *testing.T) {
testCases := []struct {
subtestName string
locked bool
source Source
data StorableDashboardData
expectDeletable bool
}{
{
subtestName: "user dashboard on the v2 schema",
source: SourceUser,
data: StorableDashboardData{"metadata": map[string]any{"schemaVersion": SchemaVersion}},
expectDeletable: true,
},
{
subtestName: "user dashboard still on the v1 schema",
source: SourceUser,
data: StorableDashboardData{"widgets": makeTestWidgets("a")},
expectDeletable: true,
},
{
subtestName: "user dashboard with unreadable data",
source: SourceUser,
data: StorableDashboardData{"metadata": "not-an-object"},
expectDeletable: true,
},
{
subtestName: "locked user dashboard",
locked: true,
source: SourceUser,
data: StorableDashboardData{"widgets": makeTestWidgets("a")},
expectDeletable: false,
},
{
subtestName: "system dashboard",
source: SourceSystem,
data: StorableDashboardData{"widgets": makeTestWidgets("a")},
expectDeletable: false,
},
{
subtestName: "integration dashboard",
source: SourceIntegration,
data: StorableDashboardData{"widgets": makeTestWidgets("a")},
expectDeletable: false,
},
}
for _, tc := range testCases {
t.Run(tc.subtestName, func(t *testing.T) {
storable := StorableDashboard{
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
OrgID: valuer.GenerateUUID(),
Locked: tc.locked,
Source: tc.source,
Data: tc.data,
}
assert.Equal(t, tc.expectDeletable, storable.ErrIfNotDeletable() == nil)
})
}
}

View File

@@ -129,16 +129,6 @@ func (d *DashboardV2) LockUnlock(lock bool, isAdmin bool, updatedBy string) erro
return nil
}
func (d *DashboardV2) ErrIfNotDeletable() error {
if d.Locked {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "cannot delete a locked dashboard, please unlock the dashboard to delete")
}
if !d.Source.isUserDeletable() {
return errors.Newf(errors.TypeInvalidInput, ErrCodeDashboardImmutable, "%s dashboards cannot be deleted", d.Source)
}
return nil
}
func (d *DashboardV2) ErrIfNotClonable() error {
if !d.Source.isClonable() {
return errors.Newf(errors.TypeInvalidInput, ErrCodeDashboardImmutable, "%s dashboards cannot be cloned", d.Source)

View File

@@ -0,0 +1,78 @@
package telemetrytypes
import "strings"
// LogicalField is one queryable field. Its Name is the spelling that the
// request used. Its Members are the physical keys that store the field.
// LogicalField is the output type of name resolution: resolution changes a
// referenced name into logical fields, and compilers make SQL from them.
//
// A []*LogicalField shows ambiguity. Ambiguity means that possibly different
// fields have the same name. Each logical field in the slice gets its own
// condition. The operator tells the compiler how to connect the conditions.
//
// One LogicalField with more than one member shows a semantic-convention
// family. A family is one field that has more than one spelling. The members
// are in current-first order. The compiler merges the members into one
// expression, and the current name wins.
//
// Members always has one entry or more. A field that is not a family has
// exactly one member. The members point to the metadata map entries. Do not
// change the members.
type LogicalField struct {
// Name is the spelling that the request used. Aliases, series labels,
// and warnings use this spelling. Because of this, the response shows
// the same spelling as the request.
Name string
// Signal, FieldContext, and FieldDataType are the identity that all
// members share. Members with a different signal, field context, or
// data type are parts of different logical fields.
Signal Signal
FieldContext FieldContext
FieldDataType FieldDataType
// Members are the physical keys that store this field, in current-first
// order. Each member has its own physical data (Materialized,
// Evolutions, JSONPlan, ...). A per-member accessor does not need data
// from the other members.
Members []*TelemetryFieldKey
}
// SingleLogicalField makes a logical field that has one physical key.
func SingleLogicalField(name string, key *TelemetryFieldKey) *LogicalField {
return &LogicalField{
Name: name,
Signal: key.Signal,
FieldContext: key.FieldContext,
FieldDataType: key.FieldDataType,
Members: []*TelemetryFieldKey{key},
}
}
// Single returns the only member of a single-member field. A decision that
// uses only the shared identity can also use Single on a family. This is
// safe because all members have the same signal, context, and data type.
func (l *LogicalField) Single() *TelemetryFieldKey {
return l.Members[0]
}
// IsFamily returns true when the field has more than one physical member.
func (l *LogicalField) IsFamily() bool {
return len(l.Members) > 1
}
// String implements fmt.Stringer. A single-member field prints as its
// member. Because of this, a message made from the field and a message made
// from the key are the same. A family prints its shared identity and its
// member spellings.
func (l *LogicalField) String() string {
if len(l.Members) == 1 {
return l.Members[0].String()
}
names := make([]string, 0, len(l.Members))
for _, member := range l.Members {
names = append(names, member.Name)
}
return l.Name + "(" + l.FieldContext.StringValue() + ", " + l.FieldDataType.StringValue() + ", members: " + strings.Join(names, ", ") + ")"
}

View File

@@ -0,0 +1,50 @@
package telemetrytypes
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestSingleLogicalFieldSharesIdentityAndAliasesKey(t *testing.T) {
key := &TelemetryFieldKey{
Name: "service.name",
Signal: SignalTraces,
FieldContext: FieldContextResource,
FieldDataType: FieldDataTypeString,
}
logical := SingleLogicalField("resource.service.name", key)
assert.Equal(t, "resource.service.name", logical.Name, "the identity is the spelling that the request used, not the stored spelling")
assert.Equal(t, key.Signal, logical.Signal)
assert.Equal(t, key.FieldContext, logical.FieldContext)
assert.Equal(t, key.FieldDataType, logical.FieldDataType)
assert.False(t, logical.IsFamily())
assert.Same(t, key, logical.Single(), "the member points to the key; there is no copy")
}
func TestStringDelegatesForSingleMember(t *testing.T) {
key := &TelemetryFieldKey{
Name: "service.name",
FieldContext: FieldContextResource,
FieldDataType: FieldDataTypeString,
}
assert.Equal(t, key.String(), SingleLogicalField(key.Name, key).String(),
"a message made from a single-member field must be the same as a message made from the key")
}
func TestStringListsFamilyMembers(t *testing.T) {
logical := &LogicalField{
Name: "deployment.environment.name",
Signal: SignalTraces,
FieldContext: FieldContextResource,
FieldDataType: FieldDataTypeString,
Members: []*TelemetryFieldKey{
{Name: "deployment.environment.name"},
{Name: "deployment.environment"},
},
}
assert.True(t, logical.IsFamily())
assert.Equal(t, "deployment.environment.name(resource, string, members: deployment.environment.name, deployment.environment)", logical.String())
}

View File

@@ -26,14 +26,14 @@ class ProviderAccountSpec:
provider: str
# params for the account created by default.
initial_params: dict
# params for the config an update (PUT) test sends.
updated_params: dict
# params -> the provider-keyed `config` block for a POST/PUT body.
build_config: Callable[[dict], dict]
# params -> the full config block the API is expected to return under
# config[provider] on GET/list. This may differ from what build_config sends:
# e.g. AWS accepts deploymentRegion on POST but the API does not echo it back.
expected_config: Callable[[dict], dict]
# only the suites that exercise updates need to supply it.
updated_params: dict = field(default_factory=dict)
# id shown in parametrized test names; defaults to the provider slug.
id: str = field(default="")
@@ -42,6 +42,29 @@ class ProviderAccountSpec:
object.__setattr__(self, "id", self.provider)
# Per-provider service shape.
@dataclass(frozen=True)
class ProviderServiceSpec:
provider: str
service_id: str
# GCP ships every service with supportedSignals.logs false, so a logs block
# is neither required on write nor persisted.
supports_logs: bool
account_config: dict
# id shown in parametrized test names; defaults to the provider slug.
id: str = field(default="")
def __post_init__(self) -> None:
if not self.id:
object.__setattr__(self, "id", self.provider)
def build_service_config(self, metrics_enabled: bool, logs_enabled: bool | None = None) -> dict:
config: dict = {"metrics": {"enabled": metrics_enabled}}
if self.supports_logs:
config["logs"] = {"enabled": metrics_enabled if logs_enabled is None else logs_enabled}
return {self.provider: config}
@pytest.fixture(scope="function")
def deprecated_create_cloud_integration_account(
request: pytest.FixtureRequest,

View File

@@ -32,10 +32,10 @@ def ai_trace(
*,
now: datetime,
service: str,
user: str,
in_tokens: int | None,
out_tokens: int,
user: str = "user",
cost: float = 0.1,
cost: float,
model: str = "gpt-4o-mini",
environment: str = "production",
) -> list[Traces]:
@@ -74,28 +74,6 @@ def ai_trace(
]
def tool_only_trace(*, now: datetime, service: str) -> list[Traces]:
"""Root + one tool span: passes the gen_ai gate but has NO LLM span."""
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
resources = {"service.name": service}
return [
root_span(now=now, trace_id=trace_id, span_id=root_id, resources=resources, duration_s=2),
Traces(
timestamp=now - timedelta(seconds=4),
duration=timedelta(seconds=0.5),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="execute_tool",
kind=TracesKind.SPAN_KIND_INTERNAL,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"gen_ai.tool.name": "get_weather", "gen_ai.tool.type": "function"},
),
]
def ai_trace_mixed_spans(*, now: datetime, service: str, user: str) -> list[Traces]:
"""Root + LLM + tool + agent spans; only the LLM span carries gen_ai.request.model."""
trace_id = TraceIdGenerator.trace_id()

File diff suppressed because it is too large Load Diff

View File

@@ -1,14 +1,52 @@
from collections.abc import Callable
from http import HTTPStatus
import pytest
import requests
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD, add_license
from fixtures.cloudintegrations import ProviderAccountSpec
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
AWS_ACCOUNT_SPEC = ProviderAccountSpec(
provider="aws",
initial_params={"deployment_region": "us-east-1", "regions": ["us-east-1", "us-west-2"]},
build_config=lambda p: {"aws": {"deploymentRegion": p["deployment_region"], "regions": p["regions"]}},
expected_config=lambda p: {"regions": p["regions"]},
)
GCP_ACCOUNT_SPEC = ProviderAccountSpec(
provider="gcp",
initial_params={
"deployment_project_id": "signoz-test-project",
"deployment_region": "us-central1",
"project_ids": ["signoz-test-project"],
},
build_config=lambda p: {
"gcp": {
"deploymentProjectId": p["deployment_project_id"],
"deploymentRegion": p["deployment_region"],
"projectIds": p["project_ids"],
}
},
expected_config=lambda p: {
"deploymentProjectId": p["deployment_project_id"],
"deploymentRegion": p["deployment_region"],
"projectIds": p["project_ids"],
},
)
PROVIDER_ACCOUNT_SPECS = [AWS_ACCOUNT_SPEC, GCP_ACCOUNT_SPEC]
provider_spec = pytest.mark.parametrize(
"spec",
PROVIDER_ACCOUNT_SPECS,
ids=[s.id for s in PROVIDER_ACCOUNT_SPECS],
)
def test_apply_license(
signoz: types.SigNoz,
@@ -20,19 +58,19 @@ def test_apply_license(
add_license(signoz, make_http_mocks, get_token)
@provider_spec
def test_create_account(
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderAccountSpec,
) -> None:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
cloud_provider = "aws"
data = create_cloud_integration_account(
admin_token,
cloud_provider,
deployment_region="us-east-1",
regions=["us-east-1", "us-west-2"],
spec.provider,
config=spec.build_config(spec.initial_params),
)
assert "id" in data, "Response data should contain 'id' field"
@@ -40,12 +78,17 @@ def test_create_account(
assert "connectionArtifact" in data, "Response data should contain 'connectionArtifact' field"
artifact = data["connectionArtifact"]
assert "aws" in artifact, "connectionArtifact should contain 'aws' field"
assert "connectionUrl" in artifact["aws"], "connectionArtifact.aws should contain 'connectionUrl'"
connection_url = artifact["aws"]["connectionUrl"]
assert "console.aws.amazon.com/cloudformation" in connection_url, "connectionUrl should be an AWS CloudFormation URL"
assert "region=us-east-1" in connection_url, "connectionUrl should contain the deployment region"
if spec.provider == "aws":
assert "aws" in artifact, "connectionArtifact should contain 'aws' field"
assert "connectionUrl" in artifact["aws"], "connectionArtifact.aws should contain 'connectionUrl'"
connection_url = artifact["aws"]["connectionUrl"]
assert "console.aws.amazon.com/cloudformation" in connection_url, "connectionUrl should be an AWS CloudFormation URL"
assert f"region={spec.initial_params['deployment_region']}" in connection_url, "connectionUrl should contain the deployment region"
else:
# GCP is a manual flow: no one-click install artifact.
assert artifact.get("gcp") is None, f"GCP should not return a connection artifact, got: {artifact}"
def test_create_account_unsupported_provider(
@@ -76,3 +119,36 @@ def test_create_account_unsupported_provider(
response_data = response.json()
assert "error" in response_data, "Response should contain 'error' field"
def test_create_gcp_account_without_project_ids(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
"""GCP account config requires at least one project ID to monitor."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/cloud_integrations/gcp/accounts"),
headers={"Authorization": f"Bearer {admin_token}"},
json={
"config": {
"gcp": {
"deploymentProjectId": "signoz-test-project",
"deploymentRegion": "us-central1",
"projectIds": [],
}
},
"credentials": {
"sigNozApiURL": "https://test.signoz.cloud",
"sigNozApiKey": "test-key",
"ingestionUrl": "https://ingest.test.signoz.cloud",
"ingestionKey": "test-ingestion-key",
},
},
timeout=10,
)
assert response.status_code == HTTPStatus.BAD_REQUEST, f"Expected 400 for empty projectIds, got {response.status_code}: {response.text}"
assert "error" in response.json(), "Response should contain 'error' field"

View File

@@ -2,14 +2,53 @@ import uuid
from collections.abc import Callable
from http import HTTPStatus
import pytest
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD, add_license
from fixtures.cloudintegrations import simulate_agent_checkin
from fixtures.cloudintegrations import (
ProviderAccountSpec,
simulate_agent_checkin,
)
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
CLOUD_PROVIDER = "aws"
AWS_ACCOUNT_SPEC = ProviderAccountSpec(
provider="aws",
initial_params={"deployment_region": "us-east-1", "regions": ["us-east-1"]},
build_config=lambda p: {"aws": {"deploymentRegion": p["deployment_region"], "regions": p["regions"]}},
expected_config=lambda p: {"regions": p["regions"]},
)
GCP_ACCOUNT_SPEC = ProviderAccountSpec(
provider="gcp",
initial_params={
"deployment_project_id": "signoz-test-project",
"deployment_region": "us-central1",
"project_ids": ["signoz-test-project"],
},
build_config=lambda p: {
"gcp": {
"deploymentProjectId": p["deployment_project_id"],
"deploymentRegion": p["deployment_region"],
"projectIds": p["project_ids"],
}
},
expected_config=lambda p: {
"deploymentProjectId": p["deployment_project_id"],
"deploymentRegion": p["deployment_region"],
"projectIds": p["project_ids"],
},
)
PROVIDER_ACCOUNT_SPECS = [AWS_ACCOUNT_SPEC, GCP_ACCOUNT_SPEC]
provider_spec = pytest.mark.parametrize(
"spec",
PROVIDER_ACCOUNT_SPECS,
ids=[s.id for s in PROVIDER_ACCOUNT_SPECS],
)
def test_apply_license(
@@ -22,22 +61,28 @@ def test_apply_license(
add_license(signoz, make_http_mocks, get_token)
@provider_spec
def test_agent_check_in(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderAccountSpec,
) -> None:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER, regions=["us-east-1"])
account = create_cloud_integration_account(
admin_token,
spec.provider,
config=spec.build_config(spec.initial_params),
)
account_id = account["id"]
provider_account_id = str(uuid.uuid4())
response = simulate_agent_checkin(
signoz,
admin_token,
CLOUD_PROVIDER,
spec.provider,
account_id,
provider_account_id,
data={"version": "v0.0.8"},
@@ -47,57 +92,63 @@ def test_agent_check_in(
data = response.json()["data"]
# New camelCase fields
assert data["cloudIntegrationId"] == account_id, "cloudIntegrationId should match"
assert data["providerAccountId"] == provider_account_id, "providerAccountId should match"
assert "integrationConfig" in data, "Response should contain 'integrationConfig'"
assert data["removedAt"] is None, "removedAt should be null for a live account"
# Backward-compat snake_case fields
assert data["account_id"] == account_id, "account_id (compat) should match"
assert data["cloud_account_id"] == provider_account_id, "cloud_account_id (compat) should match"
assert "integration_config" in data, "Response should contain 'integration_config' (compat)"
assert "removed_at" in data, "Response should contain 'removed_at' (compat)"
if spec.provider == "aws":
# Backward compat for agents deployed before the camelCase response; AWS only.
assert data["account_id"] == account_id, "account_id (compat) should match"
assert data["cloud_account_id"] == provider_account_id, "cloud_account_id (compat) should match"
assert "integration_config" in data, "Response should contain 'integration_config' (compat)"
assert "removed_at" in data, "Response should contain 'removed_at' (compat)"
# integrationConfig should reflect the configured regions
integration_config = data["integrationConfig"]
assert "aws" in integration_config, "integrationConfig should contain 'aws' block"
assert integration_config["aws"]["enabledRegions"] == ["us-east-1"], "enabledRegions should match account config"
integration_config = data["integrationConfig"]
assert "aws" in integration_config, "integrationConfig should contain 'aws' block"
assert integration_config["aws"]["enabledRegions"] == spec.initial_params["regions"], "enabledRegions should match account config"
else:
# GCP is a manual flow: the agent carries its own configuration.
assert data["integrationConfig"].get("gcp") is None, f"GCP should not return an integration config, got: {data['integrationConfig']}"
@provider_spec
def test_agent_check_in_account_not_found(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
spec: ProviderAccountSpec,
) -> None:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
fake_id = str(uuid.uuid4())
response = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, fake_id, str(uuid.uuid4()))
response = simulate_agent_checkin(signoz, admin_token, spec.provider, fake_id, str(uuid.uuid4()))
assert response.status_code == HTTPStatus.NOT_FOUND, f"Expected 404, got {response.status_code}: {response.text}"
@provider_spec
def test_duplicate_cloud_account_checkins(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderAccountSpec,
) -> None:
"""Test that two different accounts cannot check in with the same providerAccountId."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account1 = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account2 = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account1 = create_cloud_integration_account(admin_token, spec.provider, config=spec.build_config(spec.initial_params))
account2 = create_cloud_integration_account(admin_token, spec.provider, config=spec.build_config(spec.initial_params))
assert account1["id"] != account2["id"], "Two accounts should have different IDs"
same_provider_account_id = str(uuid.uuid4())
# First check-in: account1 claims the provider account ID
response = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account1["id"], same_provider_account_id)
response = simulate_agent_checkin(signoz, admin_token, spec.provider, account1["id"], same_provider_account_id)
assert response.status_code == HTTPStatus.OK, f"Expected 200 for first check-in, got {response.status_code}: {response.text}"
# Second check-in: account2 tries to claim the same provider account ID → 409
response = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account2["id"], same_provider_account_id)
response = simulate_agent_checkin(signoz, admin_token, spec.provider, account2["id"], same_provider_account_id)
assert response.status_code == HTTPStatus.CONFLICT, f"Expected 409 for duplicate providerAccountId, got {response.status_code}: {response.text}"

View File

@@ -2,18 +2,47 @@ import uuid
from collections.abc import Callable
from http import HTTPStatus
import pytest
import requests
from sqlalchemy import bindparam, sql
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD, add_license
from fixtures.cloudintegrations import simulate_agent_checkin
from fixtures.cloudintegrations import (
ProviderServiceSpec,
simulate_agent_checkin,
)
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
CLOUD_PROVIDER = "aws"
SERVICE_ID = "rds"
AWS_SERVICE_SPEC = ProviderServiceSpec(
provider="aws",
service_id="rds",
supports_logs=True,
account_config={"aws": {"deploymentRegion": "us-east-1", "regions": ["us-east-1"]}},
)
GCP_SERVICE_SPEC = ProviderServiceSpec(
provider="gcp",
service_id="cloudsql_postgres",
supports_logs=False,
account_config={
"gcp": {
"deploymentProjectId": "signoz-test-project",
"deploymentRegion": "us-central1",
"projectIds": ["signoz-test-project"],
}
},
)
PROVIDER_SERVICE_SPECS = [AWS_SERVICE_SPEC, GCP_SERVICE_SPEC]
provider_spec = pytest.mark.parametrize(
"spec",
PROVIDER_SERVICE_SPECS,
ids=[s.id for s in PROVIDER_SERVICE_SPECS],
)
def test_apply_license(
@@ -26,16 +55,18 @@ def test_apply_license(
add_license(signoz, make_http_mocks, get_token)
@provider_spec
def test_list_services_without_account(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
spec: ProviderServiceSpec,
) -> None:
"""List the cloud provider's supported services"""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/services"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/services"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -53,35 +84,37 @@ def test_list_services_without_account(
assert "icon" in service, "Service should have 'icon' field"
assert "enabled" in service, "Service should have 'enabled' field"
EC2_SERVICE_ID = "ec2"
listed_ids = {s["id"] for s in data["services"]}
assert spec.service_id in listed_ids, f"'{spec.service_id}' should be listed for {spec.provider}"
@provider_spec
def test_list_account_services(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""ListAccountServicesMetadata reflects enabled state after enabling a service."""
"""ListAccountServicesMetadata reflects enabled state per service."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
put_response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{EC2_SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": True}, "logs": {"enabled": True}}}},
json={"config": spec.build_service_config(True)},
timeout=10,
)
assert put_response.status_code == HTTPStatus.NO_CONTENT, f"Enable ec2 failed: {put_response.status_code}: {put_response.text}"
assert put_response.status_code == HTTPStatus.NO_CONTENT, f"Enable {spec.service_id} failed: {put_response.status_code}: {put_response.text}"
list_response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -92,21 +125,28 @@ def test_list_account_services(
assert isinstance(data["services"], list), "services should be a list"
assert len(data["services"]) > 0, "services list should be non-empty"
ec2_service = next((s for s in data["services"] if s["id"] == EC2_SERVICE_ID), None)
assert ec2_service is not None, f"EC2 service '{EC2_SERVICE_ID}' not found in services list"
assert ec2_service["enabled"] is True, f"EC2 service should be enabled, got: {ec2_service['enabled']}"
enabled_service = next((s for s in data["services"] if s["id"] == spec.service_id), None)
assert enabled_service is not None, f"Service '{spec.service_id}' not found in services list"
assert enabled_service["enabled"] is True, f"Service should be enabled, got: {enabled_service['enabled']}"
# The listing must report state per service, not blanket-enable or echo the write.
untouched_service = next((s for s in data["services"] if s["id"] != spec.service_id), None)
assert untouched_service is not None, "Expected more than one service in the listing"
assert untouched_service["enabled"] is False, f"Service '{untouched_service['id']}' was never enabled, got: {untouched_service['enabled']}"
@provider_spec
def test_get_service_details_without_account(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
spec: ProviderServiceSpec,
) -> None:
"""Get full service definition without specifying an account."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -114,31 +154,36 @@ def test_get_service_details_without_account(
assert response.status_code == HTTPStatus.OK, f"Expected 200, got {response.status_code}"
data = response.json()["data"]
assert data["id"] == SERVICE_ID, f"id should be '{SERVICE_ID}'"
assert data["id"] == spec.service_id, f"id should be '{spec.service_id}'"
assert "title" in data, "Service should have 'title'"
assert "overview" in data, "Service should have 'overview' (markdown)"
assert "assets" in data, "Service should have 'assets'"
assert isinstance(data["assets"]["dashboards"], list), "assets.dashboards should be a list"
assert data["cloudIntegrationService"] is None, "cloudIntegrationService should be null without account context"
assert data["supportedSignals"]["metrics"] is True, "metrics should be a supported signal"
assert data["supportedSignals"]["logs"] is spec.supports_logs, f"logs support should be {spec.supports_logs} for {spec.provider}"
@provider_spec
def test_get_account_service(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""Get service for a specific account — all disabled by default."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -146,20 +191,22 @@ def test_get_account_service(
assert response.status_code == HTTPStatus.OK, f"Expected 200, got {response.status_code}"
data = response.json()["data"]
assert data["id"] == SERVICE_ID, f"id should be '{SERVICE_ID}'"
assert data["id"] == spec.service_id, f"id should be '{spec.service_id}'"
assert data["cloudIntegrationService"] is None, "cloudIntegrationService should be null before any config is set"
@provider_spec
def test_get_service_not_found(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
spec: ProviderServiceSpec,
) -> None:
"""Get a non-existent service ID returns 400 (invalid service ID is a bad request)."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/services/non-existent-service"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/services/non-existent-service"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -167,32 +214,34 @@ def test_get_service_not_found(
assert response.status_code == HTTPStatus.BAD_REQUEST, f"Expected 400, got {response.status_code}"
@provider_spec
def test_update_service_config(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""Enable a service and verify the config is persisted via GET."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
put_response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": True}, "logs": {"enabled": True}}}},
json={"config": spec.build_service_config(True)},
timeout=10,
)
assert put_response.status_code == HTTPStatus.NO_CONTENT, f"Expected 204, got {put_response.status_code}: {put_response.text}"
get_response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -201,33 +250,39 @@ def test_update_service_config(
data = get_response.json()["data"]
svc = data["cloudIntegrationService"]
assert svc is not None, "cloudIntegrationService should be non-null after UpdateService"
assert svc["config"]["aws"]["metrics"]["enabled"] is True, "metrics should be enabled"
assert svc["config"]["aws"]["logs"]["enabled"] is True, "logs should be enabled"
assert svc["config"][spec.provider]["metrics"]["enabled"] is True, "metrics should be enabled"
assert svc["cloudIntegrationId"] == account_id, "cloudIntegrationId should match the account"
if spec.supports_logs:
assert svc["config"][spec.provider]["logs"]["enabled"] is True, "logs should be enabled"
else:
assert svc["config"][spec.provider].get("logs") is None, f"logs should not be stored for {spec.provider}, got: {svc['config'][spec.provider]}"
@provider_spec
def test_update_service_config_disable(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""Enable then disable a service — config change is persisted."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
endpoint = signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}")
endpoint = signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}")
# Enable
r = requests.put(
endpoint,
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": True}, "logs": {"enabled": True}}}},
json={"config": spec.build_service_config(True)},
timeout=10,
)
assert r.status_code == HTTPStatus.NO_CONTENT, f"Enable failed: {r.status_code}: {r.text}"
@@ -236,13 +291,13 @@ def test_update_service_config_disable(
r = requests.put(
endpoint,
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": False}, "logs": {"enabled": False}}}},
json={"config": spec.build_service_config(False)},
timeout=10,
)
assert r.status_code == HTTPStatus.NO_CONTENT, f"Disable failed: {r.status_code}: {r.text}"
get_response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -250,28 +305,57 @@ def test_update_service_config_disable(
assert get_response.status_code == HTTPStatus.OK
svc = get_response.json()["data"]["cloudIntegrationService"]
assert svc is not None, "cloudIntegrationService should still be present after disable"
assert svc["config"]["aws"]["metrics"]["enabled"] is False, "metrics should be disabled"
assert svc["config"]["aws"]["logs"]["enabled"] is False, "logs should be disabled"
assert svc["config"][spec.provider]["metrics"]["enabled"] is False, "metrics should be disabled"
if spec.supports_logs:
assert svc["config"][spec.provider]["logs"]["enabled"] is False, "logs should be disabled"
@provider_spec
def test_update_service_account_not_found(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
spec: ProviderServiceSpec,
) -> None:
"""PUT with a non-existent account UUID returns 404."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{uuid.uuid4()}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{uuid.uuid4()}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": True}}}},
json={"config": spec.build_service_config(True)},
timeout=10,
)
assert response.status_code == HTTPStatus.NOT_FOUND, f"Expected 404, got {response.status_code}"
def test_update_gcp_service_without_metrics_config(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
) -> None:
"""GCP services support metrics only, so a config omitting metrics is rejected."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, "gcp", config=GCP_SERVICE_SPEC.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, "gcp", account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/gcp/accounts/{account_id}/services/{GCP_SERVICE_SPEC.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"gcp": {"logs": {"enabled": True}}}},
timeout=10,
)
assert response.status_code == HTTPStatus.BAD_REQUEST, f"Expected 400 when metrics config is missing, got {response.status_code}: {response.text}"
def test_list_services_unsupported_provider(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
@@ -289,30 +373,32 @@ def test_list_services_unsupported_provider(
assert response.status_code == HTTPStatus.BAD_REQUEST, f"Expected 400, got {response.status_code}"
@provider_spec
def test_list_services_account_removed(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""List services for a deleted account returns 404."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
delete_response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
assert delete_response.status_code == HTTPStatus.NO_CONTENT, f"Expected 204 on delete, got {delete_response.status_code}"
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -320,30 +406,32 @@ def test_list_services_account_removed(
assert response.status_code == HTTPStatus.NOT_FOUND, f"Expected 404, got {response.status_code}"
@provider_spec
def test_get_service_details_account_removed(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""Get service details for a deleted account returns 404."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
delete_response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
assert delete_response.status_code == HTTPStatus.NO_CONTENT, f"Expected 204 on delete, got {delete_response.status_code}"
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -351,64 +439,68 @@ def test_get_service_details_account_removed(
assert response.status_code == HTTPStatus.NOT_FOUND, f"Expected 404, got {response.status_code}"
@provider_spec
def test_update_service_account_removed(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""PUT service config for a deleted account returns 404."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
delete_response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
assert delete_response.status_code == HTTPStatus.NO_CONTENT, f"Expected 204 on delete, got {delete_response.status_code}"
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": True}}}},
json={"config": spec.build_service_config(True)},
timeout=10,
)
assert response.status_code == HTTPStatus.NOT_FOUND, f"Expected 404, got {response.status_code}"
@provider_spec
def test_enable_metrics_provisions_dashboards(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""Enabling metrics provisions dashboards visible in GetService and present in the DB."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
put_response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": True}, "logs": {"enabled": False}}}},
json={"config": spec.build_service_config(True, logs_enabled=False)},
timeout=10,
)
assert put_response.status_code == HTTPStatus.NO_CONTENT, f"Expected 204, got {put_response.status_code}: {put_response.text}"
# Assertion 1: GetService returns provisioned dashboard UUIDs
get_svc_response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -417,7 +509,7 @@ def test_enable_metrics_provisions_dashboards(
data = get_svc_response.json()["data"]
svc = data["cloudIntegrationService"]
assert svc is not None, "cloudIntegrationService should be non-null after enabling metrics"
assert svc["config"]["aws"]["metrics"]["enabled"] is True
assert svc["config"][spec.provider]["metrics"]["enabled"] is True
dashboards_in_service = data["assets"]["dashboards"]
assert isinstance(dashboards_in_service, list) and len(dashboards_in_service) > 0, "assets.dashboards should be non-empty after enabling metrics"
@@ -445,35 +537,37 @@ def test_enable_metrics_provisions_dashboards(
assert provisioned_ids == db_ids, f"Dashboards {provisioned_ids - db_ids} are missing from the DB"
@provider_spec
def test_disable_metrics_deprovisions_dashboards(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
spec: ProviderServiceSpec,
) -> None:
"""Disabling metrics removes provisioned dashboards from both GetService and the dashboards list."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER)
account = create_cloud_integration_account(admin_token, spec.provider, config=spec.account_config)
account_id = account["id"]
checkin = simulate_agent_checkin(signoz, admin_token, CLOUD_PROVIDER, account_id, str(uuid.uuid4()))
checkin = simulate_agent_checkin(signoz, admin_token, spec.provider, account_id, str(uuid.uuid4()))
assert checkin.status_code == HTTPStatus.OK, f"Check-in failed: {checkin.text}"
endpoint = signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}")
endpoint = signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}")
# Enable metrics to provision dashboards first
enable_response = requests.put(
endpoint,
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": True}, "logs": {"enabled": False}}}},
json={"config": spec.build_service_config(True, logs_enabled=False)},
timeout=10,
)
assert enable_response.status_code == HTTPStatus.NO_CONTENT, f"Enable failed: {enable_response.status_code}: {enable_response.text}"
# Capture the provisioned dashboard IDs before disabling
get_svc_response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)
@@ -485,14 +579,14 @@ def test_disable_metrics_deprovisions_dashboards(
disable_response = requests.put(
endpoint,
headers={"Authorization": f"Bearer {admin_token}"},
json={"config": {"aws": {"metrics": {"enabled": False}, "logs": {"enabled": False}}}},
json={"config": spec.build_service_config(False)},
timeout=10,
)
assert disable_response.status_code == HTTPStatus.NO_CONTENT, f"Disable failed: {disable_response.status_code}: {disable_response.text}"
# Assertion 1: GetService no longer returns UUID dashboard IDs
get_svc_after = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{CLOUD_PROVIDER}/accounts/{account_id}/services/{SERVICE_ID}"),
signoz.self.host_configs["8080"].get(f"/api/v1/cloud_integrations/{spec.provider}/accounts/{account_id}/services/{spec.service_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=10,
)

View File

@@ -72,8 +72,8 @@ def test_ai_list_having_aggregate_filter(
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""One filter box splits into WHERE + HAVING; bare and `trace.` spellings behave
identically; an output-only aggregate is rejected."""
"""Span + aggregate condition in one filter box splits into WHERE + HAVING; bare
and `trace.` spellings behave identically; an output-only aggregate is rejected."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-having"
@@ -322,8 +322,9 @@ def test_ai_list_nested_group_span_or_and_aggregate(
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A nested (span OR span) group ANDed with an aggregate must not flatten: span
predicates go to WHERE, the aggregate to HAVING."""
"""service.name = X AND (has_error = true OR gen_ai.request.model = 'gpt-4o') AND
total_tokens > 100: the nested OR group must not flatten, span predicates go to
WHERE, the aggregate to HAVING."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-nested"

View File

@@ -1,588 +0,0 @@
"""
builder_ai_query scalar / time-series aggregations. The `trace.` prefix picks the
domain per expression: bare keys aggregate over gen_ai spans, trace.* over
window-clipped per-trace values; a trace-level filter condition qualifies whole
traces on both domains. Tests isolate their data via unique service.name.
"""
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
import pytest
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.querier import (
Aggregation,
BuilderQuery,
OrderBy,
RequestType,
TelemetryFieldKey,
get_scalar_columns,
get_scalar_table_data,
make_query_request,
)
from fixtures.querierai import ai_trace, query_window, tool_only_trace
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
def scalar_query(
service: str,
expression: str,
*,
filter_extra: str = "",
group_by: list[TelemetryFieldKey] | None = None,
alias: str | None = None,
having: str | None = None,
order: list[OrderBy] | None = None,
limit: int | None = None,
) -> dict:
filter_expression = f"service.name = '{service}'"
if filter_extra:
filter_expression += f" AND {filter_extra}"
return BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=filter_expression,
aggregations=[Aggregation(expression=expression, alias=alias)],
group_by=group_by,
having_expression=having,
order=order,
limit=limit,
).to_dict()
def scalar_value(signoz: types.SigNoz, token: str, start_ms: int, end_ms: int, service: str, expression: str, filter_extra: str = "") -> float:
"""Run one single-aggregation scalar query and return its value."""
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[scalar_query(service, expression, filter_extra=filter_extra)],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, f"{expression}: {resp.text}"
data = get_scalar_table_data(resp.json())
assert len(data) == 1, f"{expression}: expected one row, got {data}"
return float(data[0][-1])
def series_values(response_json: dict) -> list[list[float]]:
"""Per-series lists of bucket values (bucket order as returned)."""
series = response_json["data"]["data"]["results"][0]["aggregations"][0]["series"]
return [[v["value"] for v in ser["values"]] for ser in series]
def test_ai_scalar_trace_level_aggregations(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Trace-level scalars over per-trace values: out-tokens 100/300 give avg=200 and
count=2, while the span-level count() sees the two LLM spans (roots gated out)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-scalar"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def value(expression: str) -> float:
return scalar_value(signoz, token, start_ms, end_ms, service, expression)
assert value("avg(trace.output_tokens)") == pytest.approx(200)
assert value("count(trace.trace_id)") == 2
assert value("max(trace.total_tokens)") == pytest.approx(330)
assert value("p50(trace.output_tokens)") == pytest.approx(200) # AggreFuncMap -> quantile(0.50)
# arithmetic inside one function and between functions
assert value("avg(trace.output_tokens + trace.input_tokens)") == pytest.approx(220)
assert value("sum(trace.output_tokens)/count(trace.trace_id)") == pytest.approx(200)
# span-level domain still works through the same request type
assert value("count()") == 2 # the two LLM spans; roots are not gen_ai
assert value("sum(gen_ai.usage.output_tokens)") == pytest.approx(400)
# multiple trace-level aggregations in one query -> one column per aggregation
multi = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="avg(trace.output_tokens)"), Aggregation(expression="count(trace.trace_id)")],
)
resp = make_query_request(signoz, token, start_ms, end_ms, [multi.to_dict()], request_type=RequestType.SCALAR)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and [float(v) for v in data[0]] == [pytest.approx(200), 2], data
def test_ai_scalar_trace_level_filter_qualifies_traces(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""`trace.output_tokens > 100` qualifies whole traces before aggregation: with
out-tokens 100/300 only the 300 trace survives, on both aggregation domains."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-qualify"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
for expression in (
"sum(trace.output_tokens)", # native trace-domain path
"sum(gen_ai.usage.output_tokens)", # delegated span-domain path (__trace_scope)
):
got = scalar_value(signoz, token, start_ms, end_ms, service, expression, filter_extra="trace.output_tokens > 100")
assert got == pytest.approx(300), expression
# the qualification also constrains delegated (span-domain) time series
ts = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}' AND trace.output_tokens > 100",
aggregations=[Aggregation(expression="sum(gen_ai.usage.output_tokens)")],
step_interval=60,
)
resp = make_query_request(signoz, token, start_ms, end_ms, [ts.to_dict()], request_type=RequestType.TIME_SERIES)
assert resp.status_code == HTTPStatus.OK, resp.text
assert series_values(resp.json()) == [[pytest.approx(300)]]
def test_ai_scalar_group_by_model(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Trace-level aggregation grouped by a span attribute: per-model avg of per-trace tokens."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-groupby"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[scalar_query(service, "avg(trace.output_tokens)", group_by=[TelemetryFieldKey(name="gen_ai.request.model")])],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
by_model = {row[0]: float(row[-1]) for row in data}
assert by_model == {"gpt-4o": pytest.approx(200), "gpt-4o-mini": pytest.approx(50)}, data
def test_ai_scalar_group_by_intrinsic_span_column(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Grouping by an intrinsic must not alias the group column to the span column it reads
(`toString(name) AS name` is a cyclic alias ClickHouse rejects)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-groupby-intrinsic"
insert_traces(
ai_trace(now=now, service=service, in_tokens=10, out_tokens=100)
+ ai_trace(now=now, service=service, in_tokens=10, out_tokens=300)
+ tool_only_trace(now=now, service=service)
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
scalar_query(
service,
"count(trace.trace_id)",
group_by=[TelemetryFieldKey(name="name")],
order=[OrderBy(key=TelemetryFieldKey(name="name"), direction="asc")],
)
],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
columns = get_scalar_columns(resp.json())
assert columns[0]["name"] == "name", columns
data = get_scalar_table_data(resp.json())
# the root spans are gated out, so each trace groups under its gen_ai span name
assert [(row[0], int(row[-1])) for row in data] == [("chat gpt-4o-mini", 2), ("execute_tool", 1)], data
def test_ai_timeseries_trace_level_aggregation(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Time-series over per-trace values: all spans fall in one step bucket, avg=200."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-ts"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="avg(trace.output_tokens)")],
step_interval=60,
)
resp = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.TIME_SERIES)
assert resp.status_code == HTTPStatus.OK, resp.text
assert series_values(resp.json()) == [[pytest.approx(200)]]
def test_ai_timeseries_top_n_groups(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Grouped, limited time series ranks groups on whole-window per-trace values in
the requested order: gpt-4o sums to 400 vs gpt-4o-mini's 50, so limit=1 keeps
gpt-4o for the default/desc ranking and gpt-4o-mini when ranking asc."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-topn"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=100, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def top_series(order: list[OrderBy] | None) -> dict:
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="sum(trace.output_tokens)", alias="total_out")],
group_by=[TelemetryFieldKey(name="gen_ai.request.model")],
order=order,
step_interval=60,
limit=1,
)
resp = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.TIME_SERIES)
assert resp.status_code == HTTPStatus.OK, resp.text
series = resp.json()["data"]["data"]["results"][0]["aggregations"][0]["series"]
assert len(series) == 1, f"limit=1 must keep exactly one group, got {len(series)} series"
return series[0]
top = top_series(None) # default ranking: first aggregation desc
assert top["labels"][0]["value"] == "gpt-4o", top["labels"]
assert [v["value"] for v in top["values"]] == [pytest.approx(400)]
bottom = top_series([OrderBy(key=TelemetryFieldKey(name="total_out"), direction="asc")])
assert bottom["labels"][0]["value"] == "gpt-4o-mini", bottom["labels"]
assert [v["value"] for v in bottom["values"]] == [pytest.approx(50)]
def test_ai_timeseries_limit_without_group_by(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A time-series limit without group-by has nothing to rank and is ignored:
the single series comes back complete."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-limit-nogroup"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=10, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="avg(trace.output_tokens)")],
step_interval=60,
limit=1,
)
resp = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.TIME_SERIES)
assert resp.status_code == HTTPStatus.OK, resp.text
assert series_values(resp.json()) == [[pytest.approx(200)]]
def test_ai_scalar_group_order_limit(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Scalar limit is a plain top-N over the grouped rows: sums 400/50/10 with
order by the aggregation alias desc and limit=2 keep the two largest models."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-scalar-limit"
insert_traces(
ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o")
+ ai_trace(now=now, service=service, in_tokens=10, out_tokens=100, model="gpt-4o")
+ ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini")
+ ai_trace(now=now, service=service, in_tokens=10, out_tokens=10, model="gpt-4")
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
scalar_query(
service,
"sum(trace.output_tokens)",
group_by=[TelemetryFieldKey(name="gen_ai.request.model")],
alias="total_out",
order=[OrderBy(key=TelemetryFieldKey(name="total_out"), direction="desc")],
limit=2,
)
],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert [(row[0], float(row[-1])) for row in data] == [("gpt-4o", pytest.approx(400)), ("gpt-4o-mini", pytest.approx(50))], data
def test_ai_timeseries_span_time_bucketing(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Per-trace values are clipped per (bucket, trace): two LLM calls two minutes
apart contribute each call's tokens to its own bucket, not the total to both."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-buckets"
trace_id = TraceIdGenerator.trace_id()
root_id = TraceIdGenerator.span_id()
resources = {"service.name": service}
def llm(offset_s: float, out_tokens: int) -> Traces:
return Traces(
timestamp=now - timedelta(seconds=offset_s),
duration=timedelta(seconds=1),
trace_id=trace_id,
span_id=TraceIdGenerator.span_id(),
parent_span_id=root_id,
name="chat",
kind=TracesKind.SPAN_KIND_CLIENT,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"gen_ai.request.model": "gpt-4o-mini", "gen_ai.usage.output_tokens": out_tokens},
)
root = Traces(
timestamp=now - timedelta(seconds=130),
duration=timedelta(seconds=130),
trace_id=trace_id,
span_id=root_id,
parent_span_id="",
name="POST /api/chat",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=resources,
attributes={"http.request.method": "POST"},
)
insert_traces([root, llm(124, 100), llm(4, 300)])
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
query = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="avg(trace.output_tokens)")],
step_interval=60,
)
resp = make_query_request(signoz, token, start_ms, end_ms, [query.to_dict()], request_type=RequestType.TIME_SERIES)
assert resp.status_code == HTTPStatus.OK, resp.text
values = series_values(resp.json())
assert len(values) == 1, values
assert sorted(values[0]) == [pytest.approx(100), pytest.approx(300)], f"each call's tokens in its own bucket: {values}"
def test_ai_scalar_variables_in_trace_level_filter(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Variables resolve inside trace-level conditions with span-filter semantics;
an unresolvable $var is a 400 (today via aggregate validation — a targeted
unknown-variable error is a separate concern)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-vars"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + ai_trace(now=now, service=service, in_tokens=30, out_tokens=300))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
query = scalar_query(service, "sum(trace.output_tokens)", filter_extra="trace.output_tokens > $threshold")
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[query],
request_type=RequestType.SCALAR,
variables={"threshold": {"type": "text", "value": 100}},
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and float(data[0][-1]) == pytest.approx(300), data
resp = make_query_request(signoz, token, start_ms, end_ms, [query], request_type=RequestType.SCALAR)
assert resp.status_code == HTTPStatus.BAD_REQUEST, resp.text
# quotes in the message are JSON-escaped, so match the halves separately
assert "$threshold" in resp.text and "cannot be used in a trace-level filter" in resp.text, resp.text
# a dynamic variable resolved to __all__ drops the condition (both traces count)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[query],
request_type=RequestType.SCALAR,
variables={"threshold": {"type": "dynamic", "value": "__all__"}},
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and float(data[0][-1]) == pytest.approx(400), data
def test_ai_scalar_tool_only_trace_null_semantics(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""A tool-only trace (in the gate, no LLM span) follows plain SQL NULL semantics:
count(trace.trace_id) counts it (consistent with the trace list), avg over its
NULL tokens skips it, and `trace.llm_call_count > 0` is the explicit opt-out."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-toolonly"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100) + tool_only_trace(now=now, service=service))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def value(expression: str, filter_extra: str = "") -> float:
return scalar_value(signoz, token, start_ms, end_ms, service, expression, filter_extra)
assert value("count(trace.trace_id)") == 2, "tool-only trace is an AI trace and must be counted"
assert value("avg(trace.output_tokens)") == pytest.approx(100), "NULL tokens are skipped by avg"
assert value("avg(trace.tool_call_count)") == pytest.approx(0.5), "tool-only trace feeds tool aggregates (1 and 0 calls)"
assert value("count()") == 2, "span-level count sees the LLM and the tool span"
# filtering on LLM activity is explicit, not implicit
assert value("count(trace.trace_id)", filter_extra="trace.llm_call_count > 0") == 1
def test_ai_scalar_having_on_aggregation(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""The outer having filters aggregation results per group (by alias)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-having"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=300, model="gpt-4o") + ai_trace(now=now, service=service, in_tokens=10, out_tokens=50, model="gpt-4o-mini"))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
resp = make_query_request(
signoz,
token,
start_ms,
end_ms,
[
scalar_query(
service,
"avg(trace.output_tokens)",
group_by=[TelemetryFieldKey(name="gen_ai.request.model")],
alias="avg_out",
having="avg_out > 100",
)
],
request_type=RequestType.SCALAR,
)
assert resp.status_code == HTTPStatus.OK, resp.text
data = get_scalar_table_data(resp.json())
assert len(data) == 1 and data[0][0] == "gpt-4o", data
def test_ai_aggregation_rejections(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
) -> None:
"""Targeted 400s: mixed domains and group-by on a trace column come from the
builder; order-by is stopped earlier by request validation (only group keys and
aggregation aliases/expressions are admitted)."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
service = "ai-it-agg-reject"
insert_traces(ai_trace(now=now, service=service, in_tokens=10, out_tokens=100))
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms, end_ms = query_window(now)
def expect_bad_request(query: dict, message: str) -> None:
resp = make_query_request(signoz, token, start_ms, end_ms, [query], request_type=RequestType.SCALAR)
assert resp.status_code == HTTPStatus.BAD_REQUEST, resp.text
assert message in resp.text, resp.text
# span-level and trace-level aggregations cannot be mixed in one query
mixed = BuilderQuery(
signal="traces",
query_type="builder_ai_query",
name="A",
filter_expression=f"service.name = '{service}'",
aggregations=[Aggregation(expression="avg(trace.output_tokens)"), Aggregation(expression="count()")],
)
expect_bad_request(mixed.to_dict(), "cannot be mixed")
expect_bad_request(
scalar_query(service, "avg(trace.output_tokens)", group_by=[TelemetryFieldKey(name="trace.llm_call_count")]),
"grouping by trace-level aggregate",
)
# a bare per-trace column would emit one row per trace instead of one aggregated row
expect_bad_request(scalar_query(service, "trace.output_tokens"), "must be inside an aggregation function")
# the rate interval divides the whole expression, so it may not carry a second aggregation
expect_bad_request(scalar_query(service, "rate(trace.trace_id) + avg(trace.output_tokens)"), "combines a rate with another aggregation")
expect_bad_request(
scalar_query(service, "avg(trace.output_tokens)", order=[OrderBy(key=TelemetryFieldKey(name="trace.total_tokens"), direction="desc")]),
"invalid order by key",
)

View File

@@ -1,34 +0,0 @@
import pytest
from testcontainers.core.container import Network
from fixtures import types
from fixtures.signoz import create_signoz
@pytest.fixture(name="signoz", scope="package")
def signoz_ai_observability(
network: Network,
migrator: types.Operation, # pylint: disable=unused-argument
zeus: types.TestContainerDocker,
gateway: types.TestContainerDocker,
sqlstore: types.TestContainerSQL,
clickhouse: types.TestContainerClickhouse,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.SigNoz:
"""Package-scoped SigNoz with AI observability enabled: the flag gates the static
gen_ai key definitions (enrichWithGenAIKeys) — without it the gate keys only
resolve once a span carrying them has been ingested."""
return create_signoz(
network=network,
zeus=zeus,
gateway=gateway,
sqlstore=sqlstore,
clickhouse=clickhouse,
request=request,
pytestconfig=pytestconfig,
cache_key="signoz-ai-observability",
env_overrides={
"SIGNOZ_FLAGGER_CONFIG_BOOLEAN_ENABLE__AI__OBSERVABILITY": True,
},
)

View File

@@ -1,20 +1,14 @@
import os
from collections.abc import Callable
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
import pytest
import requests
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.logs import Logs
from fixtures.querier import get_column_data_from_response, make_query_request
TESTDATA_DIR = os.path.join(os.path.dirname(__file__), "..", "..", "testdata")
FILTER_EXPRESSIONS_FILE = os.path.join(TESTDATA_DIR, "filter_expressions_10000.txt")
@pytest.mark.parametrize(
"expression,expected_logs",
@@ -180,101 +174,3 @@ def test_not_filter_expression(
assert response.status_code == HTTPStatus.OK
assert response.json()["status"] == "success"
assert set(get_column_data_from_response(response.json(), "body")) == expected_logs
def test_filter_expressions_no_server_error(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
insert_logs,
get_token: Callable[[str, str], str],
) -> None:
"""
Reads every line from filter_expressions_10000.txt and fires it as a filter
expression against the logs query endpoint.
Expressions may be valid (200) or invalid (400) — both are acceptable.
A 500 means the server crashed on the input and is a test failure.
All failing expressions are collected before asserting so the full list is
visible in one run.
"""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
now = datetime.now(tz=UTC)
insert_logs(
[
Logs(
timestamp=now - timedelta(seconds=5),
body="alpha-log",
resources={
"f1": "v10",
"f2": "v20",
"f3": "v30",
},
attributes={
"f4": 40,
"f5": 50,
"f6": 60,
},
),
Logs(
timestamp=now - timedelta(seconds=3),
body="beta-log",
resources={
"f4": "v41",
"f5": "v51",
"f6": "v61",
},
attributes={
"f1": 11,
"f2": 21,
"f3": 31,
},
),
]
)
def _make_raw_logs_query(
signoz: types.SigNoz,
token: str,
filter_expression: str,
) -> requests.Response:
"""Helper to query raw logs with a filter expression over the last 30 seconds."""
now = datetime.now(tz=UTC)
return make_query_request(
signoz,
token,
start_ms=int((now - timedelta(seconds=30)).timestamp() * 1000),
end_ms=int(now.timestamp() * 1000),
request_type="raw",
queries=[
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "logs",
"disabled": False,
"limit": 100,
"offset": 0,
"filter": {"expression": filter_expression},
"order": [
{"key": {"name": "timestamp"}, "direction": "desc"},
{"key": {"name": "id"}, "direction": "desc"},
],
"having": {"expression": ""},
"aggregations": [{"expression": "count()"}],
},
}
],
)
failures: list[str] = []
with ThreadPoolExecutor(max_workers=40) as executor:
with open(FILTER_EXPRESSIONS_FILE, encoding="utf-8") as f:
futures = {executor.submit(_make_raw_logs_query, signoz, token, expr.rstrip("\n")): expr.rstrip("\n") for expr in f}
for future in as_completed(futures):
expr = futures[future]
if future.result().status_code == HTTPStatus.INTERNAL_SERVER_ERROR:
failures.append(expr)
assert len(failures) <= 0, f"{len(failures)} expression(s) caused HTTP 500:\n" + "\n".join(f" {expr!r}" for expr in failures)