Compare commits

...

142 Commits

Author SHA1 Message Date
Srikanth Chekuri
2ab83a0e96 Merge branch 'main' into e2e/alert_manager 2026-07-16 01:25:46 +05:30
Vinicius Lourenço
65fde71b72 feat(infrastructure-monitoring-v2): add counts cards (#12120)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat(entity-count): add base structure to show count

* feat(entity-count): add count for clusters

* feat(entity-count): add count for namespaces

* chore(infra-monitoring): move component to be under components folder

* fix(pr): address comments
2026-07-15 13:39:48 +00:00
Vinicius Lourenço
7f5f63b20a feat(infra-monitoring): add docs for every chart (#12037)
* feat(infra-monitoring): add docs for every chart

* fix(docs): add missing docs for each column/chart

* fix(infra-monitoring): keep referer

---------

Co-authored-by: Nikhil Mantri <nikhil.mantri1999@gmail.com>
2026-07-15 13:25:53 +00:00
Tushar Vats
63cfbe8bfb fix: convert key not found to warnings for traces (#12091) 2026-07-15 12:33:47 +00:00
Niladri Adhikary
821dc233bf fix(alertmanager): integration tests fixes (#11710)
* fix: integration tests

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

* test(alertmanager): add logs and traces e2e tests for custom templating

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

* fix: missing annotations

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

* fix: annotations and templating fixes

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

* chore: removed annotations

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

* fix: tests removed annotations

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

* fix: integration tests

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

* chore: py-lint run

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>

---------

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>
2026-06-15 11:21:39 +05:30
Srikanth Chekuri
05c3657ff9 Merge branch 'main' into e2e/alert_manager 2026-06-13 17:06:51 +05:30
Niladri Adhikary
9c066a3389 test(alertmanager): fix integration tests 2026-06-13 17:06:04 +05:30
srikanthccv
5cb04161c2 chore: stale changes 2026-05-25 23:07:21 +05:30
srikanthccv
1a23dbf4df Merge branch 'main' into e2e/alert_manager 2026-05-25 22:58:19 +05:30
Abhishek Kumar Singh
7ab9272a71 chore: moved alertmanager test to new package 2026-04-24 08:50:38 +05:30
Abhishek Kumar Singh
df35a8e0a4 chore: increased alert time to avoid flaky tests 2026-04-23 21:53:09 +05:30
Abhishek Kumar Singh
bee369309c chore: updated email test data with newline changes 2026-04-23 21:38:22 +05:30
Abhishek Kumar Singh
096602cf8b chore: supress line size linter for notification template 2026-04-23 21:32:55 +05:30
Abhishek Kumar Singh
96d423eca3 chore: updates test data based on new changes in templater 2026-04-23 21:29:18 +05:30
Abhishek Kumar Singh
64c736d549 chore: refactor as per new code 2026-04-23 21:03:01 +05:30
Abhishek Kumar Singh
04ee77d533 Merge branch 'chore/alert_templater_connecting_pieces' into e2e/alert_manager 2026-04-23 20:43:42 +05:30
Abhishek Kumar Singh
d40b0cb5da Merge branch 'main' into chore/alert_templater_connecting_pieces 2026-04-23 20:43:21 +05:30
Abhishek Kumar Singh
bcc17f1c0d Merge branch 'chore/alert_templater_connecting_pieces' into e2e/alert_manager 2026-04-23 20:41:06 +05:30
Abhishek Kumar Singh
05de5c6a17 Merge branch 'main' into chore/alert_templater_connecting_pieces 2026-04-23 20:22:02 +05:30
Abhishek Kumar Singh
ffe05af591 chore: updated email template based on new template struct 2026-04-23 20:21:20 +05:30
Abhishek Kumar Singh
64b45a36a8 Merge branch 'chore/alert_templater_connecting_pieces' into e2e/alert_manager 2026-04-23 16:56:38 +05:30
Abhishek Kumar Singh
75728c9918 chore: remove private annotations from pagerduty notifier 2026-04-23 16:55:53 +05:30
Abhishek Kumar Singh
b08578acb4 Merge branch 'chore/alert_templater_connecting_pieces' into e2e/alert_manager 2026-04-23 12:05:28 +05:30
Srikanth Chekuri
99208b89c1 Merge branch 'main' into chore/alert_templater_connecting_pieces 2026-04-23 03:05:05 +05:30
Abhishek Kumar Singh
8454cc8609 refactor: msteams skip logs and traces as factsset, slack code refactor 2026-04-22 20:02:15 +05:30
Abhishek Kumar Singh
bf9220596c chore: updated webhook notifier to send templated title and body in notification 2026-04-22 19:56:01 +05:30
Abhishek Kumar Singh
89fc758e7f chore: removed notification processor 2026-04-22 19:06:24 +05:30
Abhishek Kumar Singh
53f17ac362 refactor: changes as per internal review 2026-04-22 19:04:10 +05:30
Abhishek Kumar Singh
87e66b041d Merge branch 'feat/markdown_renderer' into chore/alert_templater_connecting_pieces 2026-04-20 12:34:33 +05:30
Abhishek Kumar Singh
36e3bad95d Merge branch 'main' into feat/markdown_renderer 2026-04-20 12:24:27 +05:30
Abhishek Kumar Singh
9b7a3a46ee refactor: changed markdown renderer from interface to package-level functions 2026-04-20 12:23:18 +05:30
Abhishek Kumar Singh
fadd421f9d Merge branch 'feat/markdown_renderer' into chore/alert_templater_connecting_pieces 2026-04-17 19:41:33 +05:30
Abhishek Kumar Singh
c36c18f79e Merge branch 'main' into feat/markdown_renderer 2026-04-17 19:37:36 +05:30
Abhishek Kumar Singh
5bb4079951 refactor: removed logger as markdown renderer dependency 2026-04-17 19:36:06 +05:30
Abhishek Kumar Singh
15a036904f chore: removed special handling for softline break 2026-04-17 19:32:09 +05:30
Abhishek Kumar Singh
af607bd249 Merge branch 'feat/alert_manager_template' into feat/markdown_renderer 2026-04-17 18:35:35 +05:30
Srikanth Chekuri
2eadc895a3 Merge branch 'main' into feat/alert_manager_template 2026-04-17 18:05:31 +05:30
Abhishek Kumar Singh
66e34c9b5e chore: lint issue 2026-04-17 17:50:05 +05:30
Abhishek Kumar Singh
799de1ece3 refactor: changes as per internal review 2026-04-17 17:15:07 +05:30
Abhishek Kumar Singh
c5c450c58c fix: concurrent rendering in markdown renderer 2026-04-16 18:20:25 +05:30
Abhishek Kumar Singh
dc67f8551f Merge branch 'feat/alert_manager_template' into feat/markdown_renderer 2026-04-16 17:59:31 +05:30
Abhishek Kumar Singh
c46c0e105a chore: removed notifier test files 2026-04-16 17:29:34 +05:30
Abhishek Kumar Singh
4aaa6ae5a1 fix: increase wait time for alertmanager server setup in notifier tests 2026-04-16 17:22:56 +05:30
Abhishek Kumar Singh
82abb9b113 chore: applied formatter + lint supressed for long config lines 2026-04-16 17:01:44 +05:30
Abhishek Kumar Singh
cc5a0b93ae Merge branch 'main' into feat/alert_manager_template 2026-04-16 16:46:44 +05:30
Abhishek Kumar Singh
e5d67f87eb chore: add alertmanager to integration CI workflow 2026-04-16 14:55:08 +05:30
Abhishek Kumar Singh
f2c56a9978 chore: maildev version bump 2026-04-16 14:52:35 +05:30
Abhishek Kumar Singh
15c593d797 chore: added maildev and notification channel to teardown 2026-04-16 14:44:35 +05:30
Abhishek Kumar Singh
9a47d3e553 Merge branch 'chore/alert_templater_connecting_pieces' into e2e/alert_manager 2026-04-16 14:29:22 +05:30
Abhishek Kumar Singh
b3fe077deb chore: remove static templates from pagerduty notifications 2026-04-16 14:28:27 +05:30
Abhishek Kumar Singh
14d30fa754 fix: email template directory for notification processor 2026-04-16 14:28:03 +05:30
Abhishek Kumar Singh
556bfe44d2 test: added e2e tests for notification content templating 2026-04-16 14:22:05 +05:30
Abhishek Kumar Singh
a9ab0bc480 chore: moved default notification channels to fixtures 2026-04-16 13:06:46 +05:30
Abhishek Kumar Singh
5eda220f88 test: alert manager supported notifier test 2026-04-14 20:25:16 +05:30
Abhishek Kumar Singh
a6ef54d6b9 Merge branch 'feat/markdown_renderer' into chore/alert_templater_connecting_pieces 2026-04-14 17:51:17 +05:30
Abhishek Kumar Singh
d1e332fb16 Merge branch 'feat/alert_manager_template' into feat/markdown_renderer 2026-04-14 17:51:05 +05:30
Abhishek Kumar Singh
c9f3e1ae26 Merge branch 'chore/am_custom_notifiers' into feat/alert_manager_template 2026-04-14 17:50:45 +05:30
Abhishek Kumar Singh
41ded342a1 Merge branch 'main' into chore/am_custom_notifiers 2026-04-14 17:50:27 +05:30
Abhishek Kumar Singh
7f22cb0442 chore: integrated slack mrkdwn renderer and added NoOp formatter 2026-04-14 17:46:33 +05:30
Abhishek Kumar Singh
6b77835050 feat: custom raw html renderer to escape <no value> 2026-04-14 17:44:57 +05:30
Abhishek Kumar Singh
909c3a80b1 feat: slack mrkdwn renderer 2026-04-14 17:44:15 +05:30
Abhishek Kumar Singh
42726747d8 Merge branch 'feat/alert_manager_template' into feat/markdown_renderer 2026-04-14 17:35:19 +05:30
Abhishek Kumar Singh
64ce90e418 fix: variables with symbols in template 2026-04-14 17:27:41 +05:30
Abhishek Kumar Singh
2fcffb7cdc feat: return single templating result from with flag for template type 2026-04-14 17:24:40 +05:30
Abhishek Kumar Singh
5ceb9255d1 chore: error logging + NoOp type definition 2026-04-14 16:28:54 +05:30
Abhishek Kumar Singh
1df7d75d43 feat: added Literal for CompareOperator and MatchType and expose from ruleManager 2026-04-14 15:18:21 +05:30
Abhishek Kumar Singh
1bbee9bc63 chore: fix linter and merge conflict issues 2026-04-14 14:59:55 +05:30
Abhishek Kumar Singh
581e7c8b19 Merge branch 'feat/markdown_renderer' into chore/alert_templater_connecting_pieces 2026-04-14 14:28:37 +05:30
Abhishek Kumar Singh
782eee23d2 Merge branch 'feat/alert_manager_template' into feat/markdown_renderer 2026-04-14 14:16:08 +05:30
Abhishek Kumar Singh
abc0d71c16 Merge branch 'chore/am_custom_notifiers' into feat/alert_manager_template 2026-04-13 22:13:25 +05:30
Abhishek Kumar Singh
2e2dd4c42b Merge branch 'main' into chore/am_custom_notifiers 2026-04-13 22:04:08 +05:30
Abhishek Kumar Singh
51621a3131 chore: added action links to email and slack notifiers 2026-04-06 20:44:24 +05:30
Abhishek Kumar Singh
0fd3979de5 chore: integration of custom templating in rule manager 2026-04-02 11:51:03 +05:30
Abhishek Kumar Singh
4f75075df0 feat: email rendering with custom template in notification processor 2026-03-31 20:28:02 +05:30
Abhishek Kumar Singh
b905d5cc5d feat: added no value extension to render <no value> in html 2026-03-31 20:16:32 +05:30
Abhishek Kumar Singh
6d1b9738b5 chore: lint fixes 2026-03-31 18:27:03 +05:30
Abhishek Kumar Singh
710cd8bdb3 Merge branch 'feat/markdown_renderer' into chore/alert_templater_connecting_pieces 2026-03-31 18:22:15 +05:30
Abhishek Kumar Singh
629929c6a6 Merge branch 'feat/alert_manager_template' into feat/markdown_renderer 2026-03-31 17:57:21 +05:30
Abhishek Kumar Singh
0ce76a94d6 Merge branch 'chore/am_custom_notifiers' into feat/alert_manager_template 2026-03-31 17:57:02 +05:30
Abhishek Kumar Singh
46ae74ced5 chore: updated email notifier from upstream 2026-03-31 11:52:08 +05:30
Abhishek Kumar Singh
2d8c1b7c86 chore: updated licenses for notifiers 2026-03-31 11:51:45 +05:30
Abhishek Kumar Singh
6602c8c523 refactor: lint fixes 2026-03-31 10:46:52 +05:30
Abhishek Kumar Singh
c22dbcbf74 refactor: review comments 2026-03-31 10:44:37 +05:30
Abhishek Kumar Singh
250bd9abeb Merge branch 'main' into chore/am_custom_notifiers 2026-03-31 10:15:39 +05:30
Abhishek Kumar Singh
605b218836 test: added test in notification procesor for no value 2026-03-30 19:53:49 +05:30
Abhishek Kumar Singh
99af679a62 fix: handled <no value> in templated response 2026-03-30 19:47:40 +05:30
Abhishek Kumar Singh
46123f925f fix: added handling for labels and annotations with . and - 2026-03-30 18:25:07 +05:30
Abhishek Kumar Singh
3e5e90f904 fix: webhook notifier update annotations before preparing data 2026-03-30 15:17:36 +05:30
Abhishek Kumar Singh
f8a614478c feat: updated slack notifier with slack mrkdwn format 2026-03-29 21:12:39 +05:30
Abhishek Kumar Singh
ffc54137ca test: add new test cases for Slack MRKDWN rendering 2026-03-29 20:08:08 +05:30
Abhishek Kumar Singh
34655db8cc test: simplify TestRenderSlackMrkdwn 2026-03-29 19:34:54 +05:30
Abhishek Kumar Singh
020140643c feat: added new format in markdown renderer 2026-03-29 19:22:28 +05:30
Abhishek Kumar Singh
6b8a4e4441 feat: slack mrkdwn renderer 2026-03-29 19:06:49 +05:30
Abhishek Kumar Singh
c345f579bb chore: updated alertmanagernotify package with updated notifier signature 2026-03-27 20:18:01 +05:30
Abhishek Kumar Singh
819c7e1103 feat: added notification processor in webhook notifier 2026-03-27 20:17:34 +05:30
Abhishek Kumar Singh
f0a1d07213 chore: added IsCustomTemplated helper function in result struct 2026-03-27 20:06:55 +05:30
Abhishek Kumar Singh
895e10b986 feat: added notification processor in pagerduty notifier 2026-03-27 20:05:04 +05:30
Abhishek Kumar Singh
78228b97ff feat: added notification processor in slack notifier 2026-03-27 19:37:13 +05:30
Abhishek Kumar Singh
826d763b89 feat: added notification processor in opsgenie notifier 2026-03-27 17:38:37 +05:30
Abhishek Kumar Singh
cb74acefc7 chore: msteams note 2026-03-27 17:00:24 +05:30
Abhishek Kumar Singh
eb79494e73 refactor: ms teams notifier 2026-03-27 16:58:52 +05:30
Abhishek Kumar Singh
28698d1af4 feat: update ms team notifier with notification processor 2026-03-27 16:45:33 +05:30
Abhishek Kumar Singh
be55cef462 feat: updated email notifier 2026-03-27 16:05:52 +05:30
Abhishek Kumar Singh
183e400280 chore: return isDefaultTemplated true even in case of blank default template 2026-03-26 19:14:01 +05:30
Abhishek Kumar Singh
5f0b43d975 chore: refactor notification processor and send processor in ReceiverIntegrations 2026-03-26 19:03:39 +05:30
Abhishek Kumar Singh
09adb8bef0 feat: alert notification processor 2026-03-26 16:04:28 +05:30
Abhishek Kumar Singh
77f5522e47 chore: return missing variables as sorted list 2026-03-26 16:03:41 +05:30
Abhishek Kumar Singh
c68154a031 feat: added no-op formatter in markdown rederer 2026-03-26 14:27:55 +05:30
Abhishek Kumar Singh
ec94a6555b refactor: alert manager templater 2026-03-25 20:01:12 +05:30
Abhishek Kumar Singh
f132dc28c3 chore: updated br with new line in test and logs added 2026-03-23 17:02:30 +05:30
Abhishek Kumar Singh
834df680f0 Merge branch 'feat/alert_manager_template' into feat/markdown_renderer 2026-03-23 16:48:26 +05:30
Abhishek Kumar Singh
48b9f15e18 feat: integrated slack blockit in markdownrenderer package and removed plaintext format 2026-03-23 16:45:29 +05:30
Abhishek Kumar Singh
55fa03fe7e test: added test for html rendering 2026-03-23 16:32:25 +05:30
Abhishek Kumar Singh
933717f309 feat: slack blockkit renderer using goldmark 2026-03-23 15:36:32 +05:30
Abhishek Kumar Singh
9ffc1203da chore: updated newline to markdown format 2026-03-19 22:50:24 +05:30
Abhishek Kumar Singh
205a78f0e6 feat: added basic html markdown templater 2026-03-17 20:17:57 +05:30
Abhishek Kumar Singh
79518b6823 Merge branch 'chore/am_custom_notifiers' into feat/alert_manager_template 2026-03-17 20:15:00 +05:30
Abhishek Kumar Singh
e6a9f49cec Merge branch 'main' into chore/am_custom_notifiers 2026-03-17 20:14:30 +05:30
Abhishek Kumar Singh
fd5fc40823 chore: updated comments 2026-03-16 18:19:03 +05:30
Abhishek Kumar Singh
db2e2a4617 chore: lint fix 2026-03-16 15:54:22 +05:30
Abhishek Kumar Singh
9368d3f393 refactor: comments and test improvements 2026-03-16 15:47:45 +05:30
Abhishek Kumar Singh
0c97ba36d6 refactor: test case and sb related changed 2026-03-16 15:12:23 +05:30
Abhishek Kumar Singh
2e1bdbc2fd chore: added test for missing function 2026-03-16 14:49:52 +05:30
Abhishek Kumar Singh
330737f779 chore: renamed the interface 2026-03-13 19:20:49 +05:30
Abhishek Kumar Singh
f0c531ae2b chore: lint fix 2026-03-13 19:11:34 +05:30
Abhishek Kumar Singh
54477ee786 feat: added support for and in templating 2026-03-13 19:09:39 +05:30
Abhishek Kumar Singh
d281f7b6a2 test: fix preprocessor test case 2026-03-13 17:31:28 +05:30
Abhishek Kumar Singh
378dc350ef refactor: added extractCommonKV instead of 2 different functions 2026-03-13 17:10:13 +05:30
Abhishek Kumar Singh
89c38ed9bc feat: converted alerttemplater to interface and updated tests 2026-03-13 17:02:26 +05:30
Abhishek Kumar Singh
04c4869b12 chore: added handling for missing variable used in template 2026-03-13 14:10:13 +05:30
Abhishek Kumar Singh
388a1184ca chore: fix lint issues 2026-03-12 21:51:13 +05:30
Abhishek Kumar Singh
03901b353b chore: hooked preProcess function in expandTitle and body, added labels and annotations in alertdata 2026-03-12 21:47:43 +05:30
Abhishek Kumar Singh
74441c74a8 feat: added preprocessor for alert templater 2026-03-12 20:54:28 +05:30
Abhishek Kumar Singh
93d332bef2 chore: exposed templates for alertmanager types 2026-03-12 18:52:40 +05:30
Abhishek Kumar Singh
1e730cae8c chore: added utils for using variables with $ notation 2026-03-12 16:34:43 +05:30
Abhishek Kumar Singh
01a09cf6d2 chore: updated test name + code for timeout errors 2026-03-12 10:22:42 +05:30
Abhishek Kumar Singh
403dddab85 feat: alert manager template to template title and notification body 2026-03-11 21:55:09 +05:30
Abhishek Kumar Singh
d07a833574 chore: added tracing to msteamsv2 notifier 2026-03-11 16:05:00 +05:30
Abhishek Kumar Singh
b39bec7245 Merge branch 'main' into chore/am_custom_notifiers 2026-03-10 22:37:24 +05:30
Abhishek Kumar Singh
6ff55c48be chore: fix email linter 2026-03-10 22:19:05 +05:30
Abhishek Kumar Singh
b15fa0f88f chore: lint fixs 2026-03-10 21:57:53 +05:30
Abhishek Kumar Singh
19fe4f860e chore: custom notifiers in alert manager 2026-03-10 13:20:04 +05:30
108 changed files with 3376 additions and 539 deletions

View File

@@ -39,6 +39,7 @@ jobs:
matrix:
suite:
- alerts
- alertmanager
- basepath
- callbackauthn
- cloudintegrations

View File

@@ -51,7 +51,7 @@ function ColumnHeader({
<a
href={`${DOCS_BASE_URL}${docPath}`}
target="_blank"
rel="noopener noreferrer"
rel="noopener"
onClick={(e): void => e.stopPropagation()}
>
Learn more.

View File

@@ -30,7 +30,7 @@ function EntityGroupHeader({
<a
href={`${DOCS_BASE_URL}${docPath}`}
target="_blank"
rel="noopener noreferrer"
rel="noopener"
onClick={(e): void => e.stopPropagation()}
>
Learn more.

View File

@@ -63,6 +63,10 @@ import LoadingContainer from '../LoadingContainer';
import '../EntityDetailsUtils/entityDetails.styles.scss';
import { parseAsString, useQueryState } from 'nuqs';
import {
EntityCountConfig,
EntityCountsSection,
} from './components/EntityCountsSection/EntityCountsSection';
const TimeRangeOffset = 1000000000;
@@ -72,6 +76,8 @@ export interface K8sDetailsMetadataConfig<T> {
render?: (value: string | number, entity: T) => React.ReactNode;
}
export type K8sDetailsCountConfig<T> = EntityCountConfig<T>;
export interface K8sDetailsFilters {
filter: { expression: string };
start: number;
@@ -92,9 +98,12 @@ export interface K8sBaseDetailsProps<T> {
getInitialLogTracesExpression: (entity: T) => string;
getInitialEventsExpression: (entity: T) => string;
metadataConfig: K8sDetailsMetadataConfig<T>[];
countsConfig?: K8sDetailsCountConfig<T>[];
getCountsFilterExpression?: (entity: T) => string;
entityWidgetInfo: {
title: string;
yAxisUnit: string;
docPath?: string;
}[];
getEntityQueryPayload: (
entity: T,
@@ -137,6 +146,8 @@ export default function K8sBaseDetails<T>({
getInitialLogTracesExpression,
getInitialEventsExpression,
metadataConfig,
countsConfig,
getCountsFilterExpression,
entityWidgetInfo,
getEntityQueryPayload,
queryKeyPrefix,
@@ -479,6 +490,19 @@ export default function K8sBaseDetails<T>({
})}
</div>
</div>
{countsConfig &&
countsConfig.length > 0 &&
selectedItem &&
getCountsFilterExpression && (
<EntityCountsSection
entity={entity}
countsConfig={countsConfig}
selectedItem={selectedItem}
filterExpression={getCountsFilterExpression(entity)}
closeDrawer={handleClose}
/>
)}
</div>
{!hideDetailViewTabs && (

View File

@@ -0,0 +1,41 @@
.countsContainer {
display: flex;
gap: var(--spacing-6);
margin-top: var(--spacing-8);
flex-wrap: wrap;
}
.countCard {
position: relative;
flex: 1;
min-width: 120px;
max-width: 180px;
border: 1px solid var(--l3-border);
border-radius: 4px;
display: flex;
flex-direction: column;
gap: var(--spacing-2);
padding: var(--spacing-6);
}
.countLabel {
color: var(--l2-foreground);
letter-spacing: var(--letter-spacing-wide);
text-transform: uppercase;
}
.countValue {
color: var(--l1-foreground);
font-family: var(--periscope-font-family-mono);
font-size: var(--font-size-xl);
}
.navigateButton {
position: absolute;
top: var(--spacing-4);
right: var(--spacing-4);
--button-padding: var(--spacing-1);
--button-height: 24px;
--button-width: 24px;
}

View File

@@ -0,0 +1,107 @@
import { Tooltip } from 'antd';
import { Button } from '@signozhq/ui/button';
import { Typography } from '@signozhq/ui/typography';
import { Compass } from '@signozhq/icons';
import { QueryParams } from 'constants/query';
import { initialQueriesMap } from 'constants/queryBuilder';
import ROUTES from 'constants/routes';
import { Link } from 'react-router-dom';
import { DataSource } from 'types/common/queryBuilder';
import { v4 as uuid } from 'uuid';
import {
INFRA_MONITORING_K8S_PARAMS_KEYS,
InfraMonitoringEntity,
} from '../../../constants';
import styles from './EntityCountsSection.module.scss';
export interface EntityCountConfig<T> {
label: string;
getValue: (entity: T) => number;
targetCategory: InfraMonitoringEntity;
}
interface EntityCountsSectionProps<T> {
entity: T;
countsConfig: EntityCountConfig<T>[];
selectedItem: string;
filterExpression: string;
closeDrawer: () => void;
}
export function EntityCountsSection<T>({
entity,
countsConfig,
selectedItem,
filterExpression,
closeDrawer,
}: EntityCountsSectionProps<T>): JSX.Element {
const buildNavigationUrl = (targetCategory: InfraMonitoringEntity): string => {
const defaultQuery = initialQueriesMap[DataSource.METRICS];
const compositeQuery = {
...defaultQuery,
id: uuid(),
builder: {
...defaultQuery.builder,
queryData: defaultQuery.builder.queryData.map((query) => ({
...query,
filter: { expression: filterExpression },
filters: { items: [], op: 'AND' as const },
})),
},
};
// TODO(H4ad): After https://github.com/SigNoz/signoz/pull/12038, inherit custom time of drawer to list
const urlParams = new URLSearchParams();
urlParams.set(INFRA_MONITORING_K8S_PARAMS_KEYS.CATEGORY, targetCategory);
urlParams.set(
QueryParams.compositeQuery,
encodeURIComponent(JSON.stringify(compositeQuery)),
);
return `${ROUTES.INFRASTRUCTURE_MONITORING_KUBERNETES}?${urlParams.toString()}`;
};
return (
<div className={styles.countsContainer}>
{countsConfig.map((config) => (
<div
key={config.label}
className={styles.countCard}
data-testid={`count-card-${config.label.toLowerCase().replace(/\s+/g, '-')}`}
>
<Typography.Text
color="muted"
size="small"
weight="medium"
className={styles.countLabel}
>
{config.label}
</Typography.Text>
<Typography.Text className={styles.countValue} size="xl" weight="semibold">
{config.getValue(entity) || '-'}
</Typography.Text>
<Link
to={buildNavigationUrl(config.targetCategory)}
onClick={closeDrawer}
data-testid={`navigate-${config.label.toLowerCase().replace(/\s+/g, '-')}`}
>
<Tooltip
title={`View ${config.label.toLowerCase()} of '${selectedItem}'`}
placement="top"
>
<Button
size="icon"
variant="ghost"
color="secondary"
className={styles.navigateButton}
prefix={<Compass size={14} />}
/>
</Tooltip>
</Link>
</div>
))}
</div>
);
}

View File

@@ -14,7 +14,9 @@ import { InfraMonitoringEntity } from '../constants';
import {
clusterWidgetInfo,
getClusterMetricsQueryPayload,
k8sClusterDetailsCountsConfig,
k8sClusterDetailsMetadataConfig,
k8sClusterGetCountsFilterExpression,
k8sClusterGetEntityName,
k8sClusterGetSelectedItemExpression,
k8sClusterInitialEventsExpression,
@@ -136,6 +138,8 @@ function K8sClustersList({
getInitialLogTracesExpression={k8sClusterInitialLogTracesExpression}
getInitialEventsExpression={k8sClusterInitialEventsExpression}
metadataConfig={k8sClusterDetailsMetadataConfig}
countsConfig={k8sClusterDetailsCountsConfig}
getCountsFilterExpression={k8sClusterGetCountsFilterExpression}
entityWidgetInfo={clusterWidgetInfo}
getEntityQueryPayload={getClusterMetricsQueryPayload}
queryKeyPrefix="cluster"

View File

@@ -6,9 +6,15 @@ import { EQueryType } from 'types/common/dashboard';
import { DataSource, ReduceOperators } from 'types/common/queryBuilder';
import { v4 } from 'uuid';
import { K8sDetailsMetadataConfig } from '../Base/K8sBaseDetails';
import {
K8sDetailsCountConfig,
K8sDetailsMetadataConfig,
} from '../Base/K8sBaseDetails';
import { formatValueForExpression } from 'components/QueryBuilderV2/utils';
import { INFRA_MONITORING_ATTR_KEYS } from '../constants';
import {
INFRA_MONITORING_ATTR_KEYS,
InfraMonitoringEntity,
} from '../constants';
import { SelectedItemParams } from '../hooks';
import {
buildEventsExpression,
@@ -23,6 +29,40 @@ export const k8sClusterGetSelectedItemExpression = (
export const k8sClusterDetailsMetadataConfig: K8sDetailsMetadataConfig<InframonitoringtypesClusterRecordDTO>[] =
[{ label: 'Cluster Name', getValue: (p): string => p.clusterName || '' }];
export const k8sClusterDetailsCountsConfig: K8sDetailsCountConfig<InframonitoringtypesClusterRecordDTO>[] =
[
{
label: 'Namespaces',
getValue: (p): number => p.counts?.namespaces ?? 0,
targetCategory: InfraMonitoringEntity.NAMESPACES,
},
{
label: 'Nodes',
getValue: (p): number => p.counts?.nodes ?? 0,
targetCategory: InfraMonitoringEntity.NODES,
},
{
label: 'Deployments',
getValue: (p): number => p.counts?.deployments ?? 0,
targetCategory: InfraMonitoringEntity.DEPLOYMENTS,
},
{
label: 'StatefulSets',
getValue: (p): number => p.counts?.statefulSets ?? 0,
targetCategory: InfraMonitoringEntity.STATEFULSETS,
},
{
label: 'DaemonSets',
getValue: (p): number => p.counts?.daemonSets ?? 0,
targetCategory: InfraMonitoringEntity.DAEMONSETS,
},
{
label: 'Jobs',
getValue: (p): number => p.counts?.jobs ?? 0,
targetCategory: InfraMonitoringEntity.JOBS,
},
];
export const k8sClusterInitialEventsExpression = (
item: InframonitoringtypesClusterRecordDTO,
): string =>
@@ -43,38 +83,54 @@ export const k8sClusterGetEntityName = (
item: InframonitoringtypesClusterRecordDTO,
): string => item.clusterName || '';
export const k8sClusterGetCountsFilterExpression = (
item: InframonitoringtypesClusterRecordDTO,
): string =>
`k8s.cluster.name = ${formatValueForExpression(item.clusterName ?? '')}`;
export const clusterWidgetInfo = [
{
title: 'CPU Usage, allocatable',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/clusters/#cpu-usage-allocatable',
},
{
title: 'Memory Usage, allocatable',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/clusters/#memory-usage-allocatable',
},
{
title: 'Ready Nodes',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#ready-nodes',
},
{
title: 'NotReady Nodes',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#notready-nodes',
},
{
title: 'Deployments available and desired',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/clusters/#deployments-available-and-desired',
},
{
title: 'Statefulset pods',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#statefulset-pods',
},
{
title: 'Daemonset nodes',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#daemonset-nodes',
},
{
title: 'Jobs',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#jobs',
},
];

View File

@@ -71,18 +71,25 @@ export const daemonSetWidgetInfo = [
{
title: 'CPU usage, request, limits',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/daemonsets/#cpu-usage-request-limits',
},
{
title: 'Memory usage, request, limits',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/daemonsets/#memory-usage-request-limits',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/daemonsets/#network-io',
},
{
title: 'Network errors count',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/daemonsets/#network-errors-count',
},
];

View File

@@ -140,10 +140,10 @@ export const k8sDaemonSetsColumnsConfig: DaemonSetTableColumnConfig[] = [
},
},
{
id: 'node_status',
id: 'scheduled_nodes',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/daemonsets#node-status">
Node Status
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/daemonsets#scheduled-nodes">
Scheduled Nodes
</ColumnHeader>
),
accessorFn: (row): number => row.currentNodes,

View File

@@ -71,18 +71,25 @@ export const deploymentWidgetInfo = [
{
title: 'CPU usage, request, limits',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/deployments/#cpu-usage-request-limits',
},
{
title: 'Memory usage, request, limits',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/deployments/#memory-usage-request-limits',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/deployments/#network-io',
},
{
title: 'Network error count',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/deployments/#network-error-count',
},
];

View File

@@ -133,10 +133,10 @@ export const k8sDeploymentsColumnsConfig: TableColumnDef<InframonitoringtypesDep
},
},
{
id: 'replica_status',
id: 'pod_replicas',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/deployments#replica-status">
Replica Status
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/deployments#pod-replicas">
Pod Replicas
</ColumnHeader>
),
accessorFn: (row): number => row.availablePods,

View File

@@ -0,0 +1,32 @@
.chartHeader {
display: inline-flex;
align-items: center;
gap: var(--spacing-2);
}
.infoIcon {
display: inline-flex;
align-items: center;
color: var(--l2-foreground);
cursor: pointer;
&:hover {
color: var(--text-slate-primary);
}
}
.metricsExplorerLink {
display: inline-flex;
align-items: center;
color: var(--l2-foreground);
transition: opacity 0.2s;
&:hover {
color: var(--l3-foreground);
}
}
.chartHeaderLabel {
font-size: var(--periscope-font-size-base);
color: var(--l2-foreground);
}

View File

@@ -0,0 +1,83 @@
import { Link } from 'react-router-dom';
import { Compass, Info } from '@signozhq/icons';
import { Tooltip } from 'antd';
import styles from './ChartHeader.module.scss';
const DOCS_BASE_URL = `${process.env.DOCS_BASE_URL}/docs`;
interface ChartHeaderProps {
title: string;
docPath?: string;
tooltip?: string;
metricsExplorerUrl?: string;
metricsExplorerTestId?: string;
}
function ChartHeader({
title,
docPath,
tooltip,
metricsExplorerUrl,
metricsExplorerTestId = 'open-metrics-explorer',
}: ChartHeaderProps): JSX.Element {
const renderInfoIcon = (): React.ReactNode => {
if (docPath) {
const tooltipTitle = tooltip || 'Not sure what this represents?';
return (
<Tooltip
arrow
title={
<>
{tooltipTitle}{' '}
<a
href={`${DOCS_BASE_URL}${docPath}`}
target="_blank"
rel="noopener"
onClick={(e): void => e.stopPropagation()}
>
Learn more.
</a>
</>
}
>
<span className={styles.infoIcon} data-testid="chart-header-info-icon">
<Info size="md" />
</span>
</Tooltip>
);
}
if (tooltip) {
return (
<Tooltip title={tooltip}>
<span className={styles.infoIcon} data-testid="chart-header-info-icon">
<Info size="md" />
</span>
</Tooltip>
);
}
return null;
};
return (
<div className={styles.chartHeader} data-testid="chart-header">
<span className={styles.chartHeaderLabel}>{title}</span>
{renderInfoIcon()}
{metricsExplorerUrl && (
<Tooltip title="Open in Metrics Explorer">
<Link
to={metricsExplorerUrl}
className={styles.metricsExplorerLink}
data-testid={metricsExplorerTestId}
>
<Compass size={14} />
</Link>
</Tooltip>
)}
</div>
);
}
export default ChartHeader;

View File

@@ -14,28 +14,6 @@
box-sizing: border-box;
}
.entityMetricsTitleContainer {
display: flex;
align-items: center;
gap: 8px;
}
.entityMetricsTitle {
font-size: var(--periscope-font-size-base);
color: var(--l2-foreground);
}
.metricsExplorerLink {
display: flex;
align-items: center;
color: var(--l2-foreground);
transition: opacity 0.2s;
&:hover {
color: var(--l3-foreground);
}
}
.metricsHeader {
display: flex;
justify-content: flex-end;

View File

@@ -1,8 +1,6 @@
import { useCallback, useMemo, useRef } from 'react';
import { UseQueryResult } from 'react-query';
import { Link } from 'react-router-dom';
import { Compass } from '@signozhq/icons';
import { Skeleton, Tooltip } from 'antd';
import { Skeleton } from 'antd';
import cx from 'classnames';
import { PANEL_TYPES } from 'constants/queryBuilder';
import TimeSeries from 'container/DashboardContainer/visualization/charts/TimeSeries/TimeSeries';
@@ -24,6 +22,7 @@ import { MetricRangePayloadProps } from 'types/api/metrics/getQueryRange';
import { getMetricsExplorerUrl } from 'utils/explorerUtils';
import { buildEntityMetricsChartConfig } from './configBuilder';
import ChartHeader from './ChartHeader';
import { useEntityMetrics } from './hooks';
import { isKeyNotFoundError } from '../utils';
@@ -47,6 +46,7 @@ interface EntityMetricsProps<T> {
entityWidgetInfo: {
title: string;
yAxisUnit: string;
docPath?: string;
}[];
getEntityQueryPayload: (
node: T,
@@ -207,31 +207,24 @@ function EntityMetrics<T>({
key={entityWidgetInfo[idx].title}
className={styles.entityMetricsCol}
>
<div className={styles.entityMetricsTitleContainer}>
<span className={styles.entityMetricsTitle}>
{entityWidgetInfo[idx].title}
</span>
{queryPayloads[idx] &&
queryPayloads[idx].graphType !== PANEL_TYPES.TABLE && (
<Tooltip title="Open in Metrics Explorer">
<Link
to={getMetricsExplorerUrl({
query: queryPayloads[idx].query,
...(selectedInterval && selectedInterval !== 'custom'
? { relativeTime: selectedInterval }
: {
startTimeMs: timeRange.startTime * 1000,
endTimeMs: timeRange.endTime * 1000,
}),
})}
className={styles.metricsExplorerLink}
data-testid={`open-metrics-explorer-${idx}`}
>
<Compass size={14} />
</Link>
</Tooltip>
)}
</div>
<ChartHeader
title={entityWidgetInfo[idx].title}
docPath={entityWidgetInfo[idx].docPath}
metricsExplorerUrl={
queryPayloads[idx] && queryPayloads[idx].graphType !== PANEL_TYPES.TABLE
? getMetricsExplorerUrl({
query: queryPayloads[idx].query,
...(selectedInterval && selectedInterval !== 'custom'
? { relativeTime: selectedInterval }
: {
startTimeMs: timeRange.startTime * 1000,
endTimeMs: timeRange.endTime * 1000,
}),
})
: undefined
}
metricsExplorerTestId={`open-metrics-explorer-${idx}`}
/>
<div className={styles.entityMetricsCard} ref={graphRef}>
{renderCardContent(query, idx)}
</div>

View File

@@ -70,18 +70,22 @@ export const jobWidgetInfo = [
{
title: 'CPU usage',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#cpu-usage',
},
{
title: 'Memory Usage',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#memory-usage',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#network-io',
},
{
title: 'Network errors count',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#network-errors-count',
},
];

View File

@@ -132,10 +132,10 @@ export const k8sJobsColumnsConfig: JobTableColumnConfig[] = [
},
},
{
id: 'completion_status',
id: 'completion',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/jobs#completion-status">
Completion Status
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/jobs#completion">
Completions
</ColumnHeader>
),
accessorFn: (row): number => row.successfulPods,
@@ -155,7 +155,7 @@ export const k8sJobsColumnsConfig: JobTableColumnConfig[] = [
{
value: row.desiredSuccessfulPods,
label: 'Desired',
color: Color.BG_ROBIN_500,
color: Color.BG_AMBER_500,
},
]}
/>

View File

@@ -14,7 +14,9 @@ import { InfraMonitoringEntity } from '../constants';
import { SelectedItemParams } from '../hooks';
import {
getNamespaceMetricsQueryPayload,
k8sNamespaceDetailsCountsConfig,
k8sNamespaceDetailsMetadataConfig,
k8sNamespaceGetCountsFilterExpression,
k8sNamespaceGetEntityName,
k8sNamespaceGetSelectedItemExpression,
k8sNamespaceInitialEventsExpression,
@@ -137,6 +139,8 @@ function K8sNamespacesList({
getInitialLogTracesExpression={k8sNamespaceInitialLogTracesExpression}
getInitialEventsExpression={k8sNamespaceInitialEventsExpression}
metadataConfig={k8sNamespaceDetailsMetadataConfig}
countsConfig={k8sNamespaceDetailsCountsConfig}
getCountsFilterExpression={k8sNamespaceGetCountsFilterExpression}
entityWidgetInfo={namespaceWidgetInfo}
getEntityQueryPayload={getNamespaceMetricsQueryPayload}
queryKeyPrefix="namespace"

View File

@@ -6,9 +6,16 @@ import { EQueryType } from 'types/common/dashboard';
import { DataSource, ReduceOperators } from 'types/common/queryBuilder';
import { v4 } from 'uuid';
import { K8sDetailsMetadataConfig } from '../Base/K8sBaseDetails';
import { INFRA_MONITORING_ATTR_KEYS } from '../constants';
import {
K8sDetailsCountConfig,
K8sDetailsMetadataConfig,
} from '../Base/K8sBaseDetails';
import {
INFRA_MONITORING_ATTR_KEYS,
InfraMonitoringEntity,
} from '../constants';
import { SelectedItemParams } from '../hooks';
import { formatValueForExpression } from 'components/QueryBuilderV2/utils';
import {
buildEventsExpression,
buildExpressionFromSelectedItemParams,
@@ -33,6 +40,30 @@ export const k8sNamespaceDetailsMetadataConfig: K8sDetailsMetadataConfig<Inframo
},
];
export const k8sNamespaceDetailsCountsConfig: K8sDetailsCountConfig<InframonitoringtypesNamespaceRecordDTO>[] =
[
{
label: 'Deployments',
getValue: (p): number => p.counts?.deployments ?? 0,
targetCategory: InfraMonitoringEntity.DEPLOYMENTS,
},
{
label: 'StatefulSets',
getValue: (p): number => p.counts?.statefulSets ?? 0,
targetCategory: InfraMonitoringEntity.STATEFULSETS,
},
{
label: 'DaemonSets',
getValue: (p): number => p.counts?.daemonSets ?? 0,
targetCategory: InfraMonitoringEntity.DAEMONSETS,
},
{
label: 'Jobs',
getValue: (p): number => p.counts?.jobs ?? 0,
targetCategory: InfraMonitoringEntity.JOBS,
},
];
export const k8sNamespaceInitialEventsExpression = (
item: InframonitoringtypesNamespaceRecordDTO,
): string =>
@@ -55,46 +86,78 @@ export const k8sNamespaceGetEntityName = (
item: InframonitoringtypesNamespaceRecordDTO,
): string => item.namespaceName || '';
export const k8sNamespaceGetCountsFilterExpression = (
item: InframonitoringtypesNamespaceRecordDTO,
): string => {
const clusterName = item.meta?.[INFRA_MONITORING_ATTR_KEYS.K8S_CLUSTER_NAME];
const clauses: string[] = [];
if (clusterName) {
clauses.push(
`${INFRA_MONITORING_ATTR_KEYS.K8S_CLUSTER_NAME} = ${formatValueForExpression(clusterName)}`,
);
}
if (item.namespaceName) {
clauses.push(
`${INFRA_MONITORING_ATTR_KEYS.K8S_NAMESPACE_NAME} = ${formatValueForExpression(item.namespaceName)}`,
);
}
return clauses.join(' AND ');
};
export const namespaceWidgetInfo = [
{
title: 'CPU Usage (cores)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#cpu-usage-cores',
},
{
title: 'Memory Usage (bytes)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/namespaces/#memory-usage-bytes',
},
{
title: 'Pods CPU (top 10)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#pods-cpu-top-10',
},
{
title: 'Pods Memory (top 10)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/namespaces/#pods-memory-top-10',
},
{
title: 'Network rate',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#network-rate',
},
{
title: 'Network errors',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#network-errors',
},
{
title: 'StatefulSets',
title: 'StatefulSets (pods)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#statefulsets',
},
{
title: 'ReplicaSets',
title: 'ReplicaSets (pods)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#replicasets',
},
{
title: 'DaemonSets',
title: 'DaemonSets (nodes)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#daemonsets',
},
{
title: 'Deployments',
title: 'Deployments (pods)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#deployments',
},
];

View File

@@ -57,42 +57,53 @@ export const nodeWidgetInfo = [
{
title: 'CPU Usage (cores)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#cpu-usage-cores',
},
{
title: 'Memory Usage (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#memory-usage-bytes',
},
{
title: 'CPU Usage (%)',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#cpu-usage-',
},
{
title: 'Memory Usage (%)',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#memory-usage-',
},
{
title: 'Pods by CPU (top 10)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#pods-by-cpu-top-10',
},
{
title: 'Pods by Memory (top 10)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#pods-by-memory-top-10',
},
{
title: 'Network error count',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#network-error-count',
},
{
title: 'Network IO rate',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#network-io-rate',
},
{
title: 'Filesystem usage (bytes)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/nodes/#filesystem-usage-bytes',
},
{
title: 'Filesystem usage (%)',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#filesystem-usage-',
},
];

View File

@@ -67,54 +67,74 @@ export const podWidgetInfo = [
{
title: 'CPU Usage (cores)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/pods/#cpu-usage-cores-1',
},
{
title: 'CPU Request, Limit Utilization',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#cpu-request-limit-utilization',
},
{
title: 'Memory Usage (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/pods/#memory-usage-bytes',
},
{
title: 'Memory Request, Limit Utilization',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-request-limit-utilization',
},
{
title: 'Memory by State',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/pods/#memory-by-state',
},
{
title: 'Memory Major Page Faults',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-major-page-faults',
},
{
title: 'CPU Usage by Container (cores)',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#cpu-usage-by-container-cores',
},
{
title: 'CPU Request, Limit Utilization by Container',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#cpu-request-limit-utilization-by-container',
},
{
title: 'Memory Usage by Container (bytes)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-usage-by-container-bytes',
},
{
title: 'Memory Request, Limit Utilization by Container',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-request-limit-utilization-by-container',
},
{
title: 'Network rate',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/pods/#network-rate',
},
{
title: 'Network errors',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/pods/#network-errors',
},
{
title: 'File system (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/pods/#file-system-bytes',
},
];

View File

@@ -152,7 +152,11 @@ export const k8sPodColumnsConfig: PodTableColumnConfig[] = [
},
{
id: 'podAge',
header: 'Age',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/pods#age">
Age
</ColumnHeader>
),
accessorFn: (row): number => row.podAge,
width: { min: 100 },
enableSort: false,
@@ -316,7 +320,11 @@ export const k8sPodColumnsConfig: PodTableColumnConfig[] = [
},
{
id: 'namespace',
header: 'Namespace',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/pods#additional-columns">
Namespace
</ColumnHeader>
),
accessorFn: (row): string =>
row.meta?.[INFRA_MONITORING_ATTR_KEYS.K8S_NAMESPACE_NAME] || '',
width: { default: 100 },
@@ -328,7 +336,11 @@ export const k8sPodColumnsConfig: PodTableColumnConfig[] = [
},
{
id: 'node',
header: 'Node',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/pods#additional-columns">
Node
</ColumnHeader>
),
accessorFn: (row): string =>
row.meta?.[INFRA_MONITORING_ATTR_KEYS.K8S_NODE_NAME] || '',
width: { default: 100 },
@@ -340,7 +352,11 @@ export const k8sPodColumnsConfig: PodTableColumnConfig[] = [
},
{
id: 'cluster',
header: 'Cluster',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/pods#additional-columns">
Cluster
</ColumnHeader>
),
accessorFn: (row): string =>
row.meta?.[INFRA_MONITORING_ATTR_KEYS.K8S_CLUSTER_NAME] || '',
width: { default: 100 },

View File

@@ -72,26 +72,37 @@ export const statefulSetWidgetInfo = [
{
title: 'CPU usage, request, limits',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#cpu-usage-request-limits',
},
{
title: 'CPU request, limit util (%)',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#cpu-request-limit-utilization-',
},
{
title: 'Memory usage, request, limits',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#memory-usage-request-limits',
},
{
title: 'Memory request, limit util (%)',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#memory-request-limit-utilization-',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/statefulsets/#network-io',
},
{
title: 'Network errors count',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#network-errors-count',
},
];

View File

@@ -141,10 +141,10 @@ export const k8sStatefulSetsColumnsConfig: TableColumnDef<InframonitoringtypesSt
},
},
{
id: 'pod_status',
id: 'pod_replicas',
header: (): React.ReactNode => (
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/statefulsets#pod-status">
Pod Status
<ColumnHeader docPath="/infrastructure-monitoring/kubernetes/statefulsets#pod-replicas">
Pod Replicas
</ColumnHeader>
),
accessorFn: (row): number => row.currentPods,

View File

@@ -69,22 +69,27 @@ export const volumeWidgetInfo = [
{
title: 'Volume available',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-available',
},
{
title: 'Volume capacity',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-capacity',
},
{
title: 'Volume inodes used',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-inodes-used',
},
{
title: 'Volume inodes',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-inodes',
},
{
title: 'Volume inodes free',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-inodes-free',
},
];

View File

@@ -2872,17 +2872,72 @@ export const nodeWidgetInfo = [
];
export const hostWidgetInfo = [
{ title: 'CPU Usage', yAxisUnit: 'percentunit' },
{ title: 'Memory Usage', yAxisUnit: 'bytes' },
{ title: 'System Load Average', yAxisUnit: '' },
{ title: 'Network usage (bytes)', yAxisUnit: 'bytes' },
{ title: 'Network usage (packet/s)', yAxisUnit: 'pps' },
{ title: 'Network errors', yAxisUnit: 'short' },
{ title: 'Network drops', yAxisUnit: 'short' },
{ title: 'Network connections', yAxisUnit: 'short' },
{ title: 'System disk io (bytes transferred)', yAxisUnit: 'bytes' },
{ title: 'System disk operations/s', yAxisUnit: 'short' },
{ title: 'Queue size', yAxisUnit: 'short' },
{ title: 'System disk operation time/s', yAxisUnit: 's' },
{ title: 'Disk Usage (%) by mountpoint', yAxisUnit: 'percentunit' },
{
title: 'CPU Usage',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/host-monitoring/#cpu-usage',
},
{
title: 'Memory Usage',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/host-monitoring/#memory-usage',
},
{
title: 'System Load Average',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/host-monitoring/#system-load-average',
},
{
title: 'Network usage (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/host-monitoring/#network-usage-bytes',
},
{
title: 'Network usage (packet/s)',
yAxisUnit: 'pps',
docPath: '/infrastructure-monitoring/host-monitoring/#network-usage-packetss',
},
{
title: 'Network errors',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#network-errors',
},
{
title: 'Network drops',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#network-drops',
},
{
title: 'Network connections',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#network-connections',
},
{
title: 'System disk io (bytes transferred)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/host-monitoring/#system-disk-io-bytes',
},
{
title: 'System disk operations/s',
yAxisUnit: 'short',
docPath:
'/infrastructure-monitoring/host-monitoring/#system-disk-operationss',
},
{
title: 'Queue size',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#queue-size',
},
{
title: 'System disk operation time/s',
yAxisUnit: 's',
docPath:
'/infrastructure-monitoring/host-monitoring/#system-disk-operation-times',
},
{
title: 'Disk Usage (%) by mountpoint',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/host-monitoring/#disk-usage--by-mountpoint',
},
];

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -22,6 +23,7 @@ func newConditionBuilder(fm qbtypes.FieldMapper) qbtypes.ConditionBuilder {
func (c *conditionBuilder) ConditionFor(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,

View File

@@ -8,6 +8,7 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -30,6 +31,12 @@ func newFieldMapper() qbtypes.FieldMapper {
return &fieldMapper{}
}
// CandidateKeys returns nil: rule-state history has no attribute-map fallback, so a
// context-missing key stays unresolved and the caller errors.
func (m *fieldMapper) CandidateKeys(_ context.Context, _ valuer.UUID, _ *telemetrytypes.TelemetryFieldKey, _ any, _ map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return nil
}
func (m *fieldMapper) getColumn(_ context.Context, key *telemetrytypes.TelemetryFieldKey) (*schema.Column, error) { //nolint:unparam
name := strings.TrimSpace(key.Name)
if col, ok := ruleStateHistoryColumns[name]; ok {
@@ -57,7 +64,7 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, _, _ uint64, key *telemetry
return []*schema.Column{col}, nil
}
func (m *fieldMapper) ColumnExpressionFor(ctx context.Context, tsStart, tsEnd uint64, field *telemetrytypes.TelemetryFieldKey, _ map[string][]*telemetrytypes.TelemetryFieldKey) (string, error) {
func (m *fieldMapper) ColumnExpressionFor(ctx context.Context, _ valuer.UUID, tsStart, tsEnd uint64, field *telemetrytypes.TelemetryFieldKey, _ map[string][]*telemetrytypes.TelemetryFieldKey) (string, error) {
colName, err := m.FieldFor(ctx, tsStart, tsEnd, field)
if err != nil {
return "", err

View File

@@ -17,6 +17,7 @@ import (
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
const traceOutsideRangeWarn = "Query %s references a trace_id that exists between %s and %s (UTC) but lies outside the selected time range; adjust the time range to see results"
@@ -24,6 +25,7 @@ const traceOutsideRangeWarn = "Query %s references a trace_id that exists betwee
type builderQuery[T any] struct {
logger *slog.Logger
telemetryStore telemetrystore.TelemetryStore
orgID valuer.UUID
stmtBuilder qbtypes.StatementBuilder[T]
spec qbtypes.QueryBuilderQuery[T]
variables map[string]qbtypes.VariableItem
@@ -45,6 +47,7 @@ type builderConfig struct {
func newBuilderQuery[T any](
logger *slog.Logger,
telemetryStore telemetrystore.TelemetryStore,
orgID valuer.UUID,
stmtBuilder qbtypes.StatementBuilder[T],
spec qbtypes.QueryBuilderQuery[T],
tr qbtypes.TimeRange,
@@ -55,6 +58,7 @@ func newBuilderQuery[T any](
return &builderQuery[T]{
logger: logger,
telemetryStore: telemetryStore,
orgID: orgID,
stmtBuilder: stmtBuilder,
spec: spec,
variables: variables,
@@ -214,7 +218,7 @@ func (q *builderQuery[T]) isWindowList() bool {
// Statement renders the SQL without executing it, for the preview path.
func (q *builderQuery[T]) Statement(ctx context.Context) (*qbtypes.Statement, error) {
return q.stmtBuilder.Build(ctx, q.fromMS, q.toMS, q.kind, q.spec, q.variables)
return q.stmtBuilder.Build(ctx, q.orgID, q.fromMS, q.toMS, q.kind, q.spec, q.variables)
}
func (q *builderQuery[T]) Execute(ctx context.Context) (*qbtypes.Result, error) {
@@ -238,7 +242,7 @@ func (q *builderQuery[T]) Execute(ctx context.Context) (*qbtypes.Result, error)
}
}
stmt, err := q.stmtBuilder.Build(ctx, fromMS, toMS, q.kind, q.spec, q.variables)
stmt, err := q.stmtBuilder.Build(ctx, q.orgID, fromMS, toMS, q.kind, q.spec, q.variables)
if err != nil {
return nil, err
}
@@ -491,7 +495,7 @@ func (q *builderQuery[T]) executeWindowList(ctx context.Context) (*qbtypes.Resul
q.spec.Offset = 0
q.spec.Limit = need
stmt, err := q.stmtBuilder.Build(ctx, r.fromNS/1e6, r.toNS/1e6, q.kind, q.spec, q.variables)
stmt, err := q.stmtBuilder.Build(ctx, q.orgID, r.fromNS/1e6, r.toNS/1e6, q.kind, q.spec, q.variables)
if err != nil {
return nil, err
}

View File

@@ -21,6 +21,11 @@ import (
var (
aggRe = regexp.MustCompile(`^__result_(\d+)$`)
// keyAliasRe matches the traces statement builder's positional column-alias prefix
// `__SELECT_KEY_<n>_` / `__GROUP_BY_KEY_<n>_`, which disambiguates select/group-by
// aliases from real table columns in the generated SQL. It is stripped here so the
// original field name surfaces as the label / column / raw-data key.
keyAliasRe = regexp.MustCompile(`^__(?:SELECT|GROUP_BY)_KEY_\d+_`)
// legacyReservedColumnTargetAliases identifies result value from a user
// written clickhouse query. The column alias indcate which value is
// to be considered as final result (or target).
@@ -29,6 +34,12 @@ var (
CodeFailUnmarshalJSONColumn = errors.MustNewCode("fail_unmarshal_json_column")
)
// stripKeyAlias removes the __SELECT_KEY_<n>_ / __GROUP_BY_KEY_<n>_ prefix from a result
// column name, recovering the field name; unprefixed names are returned unchanged.
func stripKeyAlias(name string) string {
return keyAliasRe.ReplaceAllString(name, "")
}
// consume reads every row and shapes it into the payload expected for the
// given request type.
//
@@ -126,7 +137,7 @@ func readAsTimeSeries(rows driver.Rows, queryWindow *qbtypes.TimeRange, step qbt
)
for idx, ptr := range slots {
name := colNames[idx]
name := stripKeyAlias(colNames[idx])
switch v := ptr.(type) {
case *time.Time:
@@ -296,6 +307,7 @@ func readAsScalar(rows driver.Rows, queryName string) (*qbtypes.ScalarData, erro
var aggIndex int64
for i, name := range colNames {
name = stripKeyAlias(name)
colType := qbtypes.ColumnTypeGroup
// Builder queries aliases aggregation columns as __result_N (always numeric) and wraps group-by keys with toString (always string);
// Raw ClickHouse queries may use any aliases.
@@ -406,7 +418,7 @@ func readAsRaw(rows driver.Rows, queryName string) (*qbtypes.RawData, error) {
}
for i, cellPtr := range scan {
name := colNames[i]
name := stripKeyAlias(colNames[i])
// de-reference the typed pointer to any
val := reflect.ValueOf(cellPtr).Elem().Interface()

View File

@@ -78,7 +78,7 @@ func (q *querier) QueryRangePreview(
skip[name] = true
}
}
providers, buildErrs := q.buildPreviewProviders(req, dependencyQueries, missingMetricQuerySet, skip)
providers, buildErrs := q.buildPreviewProviders(orgID, req, dependencyQueries, missingMetricQuerySet, skip)
// Render each executing query's statement and collect the ClickHouse-bound
// analysis work to run concurrently.
@@ -192,6 +192,7 @@ func missingMetricNames(env qbtypes.QueryEnvelope) []string {
}
func (q *querier) buildPreviewProviders(
orgID valuer.UUID,
req *qbtypes.QueryRangeRequest,
dependencyQueries map[string]bool,
missingMetricQuerySet map[string]bool,
@@ -230,7 +231,7 @@ func (q *querier) buildPreviewProviders(
sub.CompositeQuery = qbtypes.CompositeQuery{Queries: []qbtypes.QueryEnvelope{query}}
}
built, _, bErr := q.buildQueries(&sub, deps, missingMetricQuerySet, event)
built, _, bErr := q.buildQueries(orgID, &sub, deps, missingMetricQuerySet, event)
if bErr != nil {
errs[name] = bErr
continue

View File

@@ -132,7 +132,7 @@ func (q *querier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtype
missingMetricQuerySet[name] = true
}
queries, steps, err := q.buildQueries(req, dependencyQueries, missingMetricQuerySet, event)
queries, steps, err := q.buildQueries(orgID, req, dependencyQueries, missingMetricQuerySet, event)
if err != nil {
return nil, err
}
@@ -176,6 +176,7 @@ func (q *querier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtype
}
func (q *querier) buildQueries(
orgID valuer.UUID,
req *qbtypes.QueryRangeRequest,
dependencyQueries map[string]bool,
missingMetricQuerySet map[string]bool,
@@ -221,6 +222,7 @@ func (q *querier) buildQueries(
}
toq := &traceOperatorQuery{
telemetryStore: q.telemetryStore,
orgID: orgID,
stmtBuilder: q.traceOperatorStmtBuilder,
spec: traceOpQuery,
compositeQuery: &req.CompositeQuery,
@@ -235,7 +237,7 @@ func (q *querier) buildQueries(
case qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]:
spec.ShiftBy = extractShiftFromBuilderQuery(spec)
timeRange := adjustTimeRangeForShift(spec, qbtypes.TimeRange{From: req.Start, To: req.End}, req.RequestType)
bq := newBuilderQuery(q.logger, q.telemetryStore, q.traceStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
bq := newBuilderQuery(q.logger, q.telemetryStore, orgID, q.traceStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
queries[spec.Name] = bq
steps[spec.Name] = spec.StepInterval
case qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]:
@@ -245,7 +247,7 @@ func (q *querier) buildQueries(
if spec.Source == telemetrytypes.SourceAudit {
stmtBuilder = q.auditStmtBuilder
}
bq := newBuilderQuery(q.logger, q.telemetryStore, stmtBuilder, spec, timeRange, req.RequestType, tmplVars, q.builderConfig)
bq := newBuilderQuery(q.logger, q.telemetryStore, orgID, stmtBuilder, spec, timeRange, req.RequestType, tmplVars, q.builderConfig)
queries[spec.Name] = bq
steps[spec.Name] = spec.StepInterval
case qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]:
@@ -262,9 +264,9 @@ func (q *querier) buildQueries(
if spec.Source == telemetrytypes.SourceMeter {
event.Source = telemetrytypes.SourceMeter.StringValue()
bq = newBuilderQuery(q.logger, q.telemetryStore, q.meterStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
bq = newBuilderQuery(q.logger, q.telemetryStore, orgID, q.meterStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
} else {
bq = newBuilderQuery(q.logger, q.telemetryStore, q.metricStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
bq = newBuilderQuery(q.logger, q.telemetryStore, orgID, q.metricStmtBuilder, spec, timeRange, req.RequestType, tmplVars, builderConfig{})
}
queries[spec.Name] = bq
@@ -535,7 +537,7 @@ func (q *querier) QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qb
if spec.Source == telemetrytypes.SourceAudit {
liveTailStmtBuilder = q.auditStmtBuilder
}
bq := newBuilderQuery(q.logger, q.telemetryStore, liveTailStmtBuilder, spec, timeRange, req.RequestType, map[string]qbtypes.VariableItem{
bq := newBuilderQuery(q.logger, q.telemetryStore, orgID, liveTailStmtBuilder, spec, timeRange, req.RequestType, map[string]qbtypes.VariableItem{
"id": {
Value: updatedLogID,
},
@@ -858,7 +860,7 @@ func (q *querier) createRangedQuery(originalQuery qbtypes.Query, timeRange qbtyp
specCopy := qt.spec.Copy()
specCopy.ShiftBy = extractShiftFromBuilderQuery(specCopy)
adjustedTimeRange := adjustTimeRangeForShift(specCopy, timeRange, qt.kind)
return newBuilderQuery(q.logger, q.telemetryStore, q.traceStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, q.traceStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
case *builderQuery[qbtypes.LogAggregation]:
specCopy := qt.spec.Copy()
@@ -868,20 +870,21 @@ func (q *querier) createRangedQuery(originalQuery qbtypes.Query, timeRange qbtyp
if qt.spec.Source == telemetrytypes.SourceAudit {
shiftStmtBuilder = q.auditStmtBuilder
}
return newBuilderQuery(q.logger, q.telemetryStore, shiftStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, q.builderConfig)
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, shiftStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, q.builderConfig)
case *builderQuery[qbtypes.MetricAggregation]:
specCopy := qt.spec.Copy()
specCopy.ShiftBy = extractShiftFromBuilderQuery(specCopy)
adjustedTimeRange := adjustTimeRangeForShift(specCopy, timeRange, qt.kind)
if qt.spec.Source == telemetrytypes.SourceMeter {
return newBuilderQuery(q.logger, q.telemetryStore, q.meterStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, q.meterStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
}
return newBuilderQuery(q.logger, q.telemetryStore, q.metricStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
return newBuilderQuery(q.logger, q.telemetryStore, qt.orgID, q.metricStmtBuilder, specCopy, adjustedTimeRange, qt.kind, qt.variables, builderConfig{})
case *traceOperatorQuery:
specCopy := qt.spec.Copy()
return &traceOperatorQuery{
telemetryStore: q.telemetryStore,
orgID: qt.orgID,
stmtBuilder: q.traceOperatorStmtBuilder,
spec: specCopy,
fromMS: uint64(timeRange.From),

View File

@@ -30,7 +30,7 @@ func (m *queryMatcherAny) Match(string, string) error { return nil }
// and returns a fixed query string so the mock ClickHouse can match it.
type mockMetricStmtBuilder struct{}
func (m *mockMetricStmtBuilder) Build(_ context.Context, _, _ uint64, _ qbtypes.RequestType, _ qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation], _ map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
func (m *mockMetricStmtBuilder) Build(_ context.Context, _ valuer.UUID, _, _ uint64, _ qbtypes.RequestType, _ qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation], _ map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
return &qbtypes.Statement{
Query: "SELECT ts, value FROM signoz_metrics",
Args: nil,

View File

@@ -10,10 +10,12 @@ import (
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type traceOperatorQuery struct {
telemetryStore telemetrystore.TelemetryStore
orgID valuer.UUID
stmtBuilder qbtypes.TraceOperatorStatementBuilder
spec qbtypes.QueryBuilderTraceOperator
compositeQuery *qbtypes.CompositeQuery
@@ -35,12 +37,13 @@ func (q *traceOperatorQuery) Window() (uint64, uint64) {
// Statement renders the SQL without executing it, for the preview path.
func (q *traceOperatorQuery) Statement(ctx context.Context) (*qbtypes.Statement, error) {
return q.stmtBuilder.Build(ctx, q.fromMS, q.toMS, q.kind, q.spec, q.compositeQuery)
return q.stmtBuilder.Build(ctx, q.orgID, q.fromMS, q.toMS, q.kind, q.spec, q.compositeQuery)
}
func (q *traceOperatorQuery) Execute(ctx context.Context) (*qbtypes.Result, error) {
stmt, err := q.stmtBuilder.Build(
ctx,
q.orgID,
q.fromMS,
q.toMS,
q.kind,

View File

@@ -53,6 +53,7 @@ func NewAggExprRewriter(
// and the args if the parametric aggregation function is used.
func (r *aggExprRewriter) Rewrite(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
expr string,
@@ -83,6 +84,7 @@ func (r *aggExprRewriter) Rewrite(
visitor := newExprVisitor(
ctx,
orgID,
startNs,
endNs,
r.logger,
@@ -107,6 +109,7 @@ func (r *aggExprRewriter) Rewrite(
// RewriteMulti rewrites a slice of expressions.
func (r *aggExprRewriter) RewriteMulti(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
exprs []string,
@@ -117,7 +120,7 @@ func (r *aggExprRewriter) RewriteMulti(
var errs []error
var chArgsList [][]any
for i, e := range exprs {
w, chArgs, err := r.Rewrite(ctx, startNs, endNs, e, rateInterval, keys)
w, chArgs, err := r.Rewrite(ctx, orgID, startNs, endNs, e, rateInterval, keys)
if err != nil {
errs = append(errs, err)
out[i] = e
@@ -135,6 +138,7 @@ func (r *aggExprRewriter) RewriteMulti(
// exprVisitor walks FunctionExpr nodes and applies the mappers.
type exprVisitor struct {
ctx context.Context
orgID valuer.UUID
startNs uint64
endNs uint64
chparser.DefaultASTVisitor
@@ -152,6 +156,7 @@ type exprVisitor struct {
func newExprVisitor(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
logger *slog.Logger,
@@ -164,6 +169,7 @@ func newExprVisitor(
) *exprVisitor {
return &exprVisitor{
ctx: ctx,
orgID: orgID,
startNs: startNs,
endNs: endNs,
logger: logger,
@@ -206,7 +212,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
dataType = telemetrytypes.FieldDataTypeFloat64
}
bodyJSONEnabled := v.flagger.BooleanOrEmpty(v.ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
bodyJSONEnabled := v.flagger.BooleanOrEmpty(v.ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(v.orgID))
// Handle *If functions with predicate + values
if aggFunc.FuncCombinator {
@@ -216,6 +222,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
origPred,
FilterExprVisitorOpts{
Context: v.ctx,
OrgID: v.orgID,
Logger: v.logger,
FieldKeys: v.fieldKeys,
FieldMapper: v.fieldMapper,
@@ -247,7 +254,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
for i := 0; i < len(args)-1; i++ {
origVal := args[i].String()
fieldKey := telemetrytypes.GetFieldKeyFromKeyText(origVal)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey, bodyJSONEnabled)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.orgID, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return errors.WrapInvalidInputf(err, errors.CodeInvalidInput, "failed to get table field name for %q", origVal)
}
@@ -265,7 +272,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
for i, arg := range args {
orig := arg.String()
fieldKey := telemetrytypes.GetFieldKeyFromKeyText(orig)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey, bodyJSONEnabled)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.orgID, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return err
}

View File

@@ -13,12 +13,14 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/exp/maps"
)
func CollisionHandledFinalExpr(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
field *telemetrytypes.TelemetryFieldKey,
@@ -47,7 +49,7 @@ func CollisionHandledFinalExpr(
addCondition := func(key *telemetrytypes.TelemetryFieldKey) error {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(ctx, startNs, endNs, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorExists, nil, sb)
conds, _, err := cb.ConditionFor(ctx, orgID, startNs, endNs, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorExists, nil, sb)
if err != nil {
return err
}
@@ -76,23 +78,23 @@ func CollisionHandledFinalExpr(
}
if len(keysForField) == 0 {
// - the context is not provided
// - there are not keys for the field
// - it is not a static field
// - the next best thing to do is see if there is a typo
// and suggest a correction
// No metadata match: let the field mapper synthesize type-variant keys.
// keys were already checked above, so pass nil here.
keysForField = fm.CandidateKeys(ctx, orgID, field, nil, nil)
}
if len(keysForField) == 0 {
// The mapper can't synthesize (e.g. metrics): fall back to the typo suggestion.
wrappedErr := errors.WithSuggestiveAdditionalf(fieldForErr, errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys)), "field `%s` not found", field.Name)
return "", nil, wrappedErr
} else {
for _, key := range keysForField {
err := addCondition(key)
if err != nil {
return "", nil, err
}
fieldExpression, _ = fm.FieldFor(ctx, startNs, endNs, key)
fieldExpression, _ = DataTypeCollisionHandledFieldName(key, dummyValue, fieldExpression, qbtypes.FilterOperatorUnknown)
stmts = append(stmts, fieldExpression)
}
for _, key := range keysForField {
err := addCondition(key)
if err != nil {
return "", nil, err
}
fieldExpression, _ = fm.FieldFor(ctx, startNs, endNs, key)
fieldExpression, _ = DataTypeCollisionHandledFieldName(key, dummyValue, fieldExpression, qbtypes.FilterOperatorUnknown)
stmts = append(stmts, fieldExpression)
}
} else {
err := addCondition(field)

View File

@@ -1,6 +1,7 @@
package querybuilder
import (
"encoding/json"
"fmt"
"github.com/SigNoz/signoz/pkg/errors"
@@ -68,6 +69,100 @@ func NewKeyNotFoundError(name string) error {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "key `%s` not found", name).WithUrl(KeyNotFoundDocURL)
}
// NewKeyNotFoundWarning is the warning surfaced when a referenced key is absent from
// metadata and the query falls back to synthesized keys.
func NewKeyNotFoundWarning(name string) string {
return fmt.Sprintf("key `%s` not found in metadata; querying the underlying data directly. If this is unexpected, check the key name for typos.", name)
}
// SynthesizeKeys builds the field keys to query when metadata has no match: a qualified
// key is honored as-is; a bare key defaults to attribute context with the data type
// inferred from the operand, or fanned out across string/number/bool without one.
func SynthesizeKeys(field *telemetrytypes.TelemetryFieldKey, value any) []*telemetrytypes.TelemetryFieldKey {
base := *field
if base.FieldContext == telemetrytypes.FieldContextUnspecified {
base.FieldContext = telemetrytypes.FieldContextAttribute
}
// Resource values are strings; pin the type so operand coercion applies.
if base.FieldContext == telemetrytypes.FieldContextResource &&
base.FieldDataType == telemetrytypes.FieldDataTypeUnspecified {
base.FieldDataType = telemetrytypes.FieldDataTypeString
}
// A set data type needs only one synthesized key.
if base.FieldDataType != telemetrytypes.FieldDataTypeUnspecified {
clone := base
return []*telemetrytypes.TelemetryFieldKey{&clone}
}
dataTypes := inferDataTypesFromOperand(value)
keys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(dataTypes))
for _, dt := range dataTypes {
clone := base
clone.FieldDataType = dt
keys = append(keys, &clone)
}
return keys
}
// allVariantDataTypes is the fanout used when no operand pins the type (exists / group by).
func allVariantDataTypes() []telemetrytypes.FieldDataType {
return []telemetrytypes.FieldDataType{
telemetrytypes.FieldDataTypeString,
telemetrytypes.FieldDataTypeNumber,
telemetrytypes.FieldDataTypeBool,
}
}
// inferDataTypesFromOperand maps an operand value to the data type(s) to query. With no
// operand or an unrecognized type it fans out across all variants.
func inferDataTypesFromOperand(value any) []telemetrytypes.FieldDataType {
switch v := value.(type) {
case nil:
return allVariantDataTypes()
case string:
return []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeString}
case bool:
return []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeBool}
case float32, float64, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, json.Number:
return []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeNumber}
case []any:
return inferDataTypesFromList(v)
default:
return allVariantDataTypes()
}
}
// inferDataTypesFromList derives the distinct data types present in an `in` list, kept
// in string/number/bool order; an empty or all-unknown list fans out.
func inferDataTypesFromList(values []any) []telemetrytypes.FieldDataType {
var hasString, hasNumber, hasBool bool
for _, v := range values {
switch v.(type) {
case string:
hasString = true
case bool:
hasBool = true
case float32, float64, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, json.Number:
hasNumber = true
}
}
var out []telemetrytypes.FieldDataType
if hasString {
out = append(out, telemetrytypes.FieldDataTypeString)
}
if hasNumber {
out = append(out, telemetrytypes.FieldDataTypeNumber)
}
if hasBool {
out = append(out, telemetrytypes.FieldDataTypeBool)
}
if len(out) == 0 {
return allVariantDataTypes()
}
return out
}
// NewFunctionUnsupportedError returns the error for a has/hasAny/hasAll/hasToken operator
// on a builder that doesn't support it (logs body only), or nil for other operators.
func NewFunctionUnsupportedError(operator qbtypes.FilterOperator) error {

View File

@@ -0,0 +1,89 @@
package querybuilder
import (
"testing"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSynthesizeKeys(t *testing.T) {
tests := []struct {
name string
field telemetrytypes.TelemetryFieldKey
value any
wantContexts []telemetrytypes.FieldContext
wantDataTypes []telemetrytypes.FieldDataType
}{
{
name: "bare key with string operand infers string",
field: telemetrytypes.TelemetryFieldKey{Name: "error.type"},
value: "timeout",
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextAttribute},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeString},
},
{
name: "bare key with number operand infers number",
field: telemetrytypes.TelemetryFieldKey{Name: "http.status"},
value: float64(500),
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextAttribute},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeNumber},
},
{
name: "bare key with bool operand infers bool",
field: telemetrytypes.TelemetryFieldKey{Name: "sampled"},
value: true,
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextAttribute},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeBool},
},
{
name: "bare key with no operand fans out across variants",
field: telemetrytypes.TelemetryFieldKey{Name: "exception.type"},
value: nil,
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextAttribute, telemetrytypes.FieldContextAttribute, telemetrytypes.FieldContextAttribute},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeString, telemetrytypes.FieldDataTypeNumber, telemetrytypes.FieldDataTypeBool},
},
{
name: "qualified data type honored as-is",
field: telemetrytypes.TelemetryFieldKey{Name: "custom.key", FieldDataType: telemetrytypes.FieldDataTypeString},
value: nil,
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextAttribute},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeString},
},
{
name: "qualified resource context honored as single string key",
field: telemetrytypes.TelemetryFieldKey{Name: "k8s.cluster.name", FieldContext: telemetrytypes.FieldContextResource},
value: nil,
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextResource},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeString},
},
{
name: "in list with homogeneous strings infers single string variant",
field: telemetrytypes.TelemetryFieldKey{Name: "error.type"},
value: []any{"a", "b"},
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextAttribute},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeString},
},
{
name: "in list with mixed types fans out to present variants",
field: telemetrytypes.TelemetryFieldKey{Name: "error.type"},
value: []any{"a", float64(1)},
wantContexts: []telemetrytypes.FieldContext{telemetrytypes.FieldContextAttribute, telemetrytypes.FieldContextAttribute},
wantDataTypes: []telemetrytypes.FieldDataType{telemetrytypes.FieldDataTypeString, telemetrytypes.FieldDataTypeNumber},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
field := tt.field
keys := SynthesizeKeys(&field, tt.value)
require.Len(t, keys, len(tt.wantDataTypes))
for i, k := range keys {
assert.Equal(t, tt.field.Name, k.Name, "name preserved")
assert.Equal(t, tt.wantContexts[i], k.FieldContext)
assert.Equal(t, tt.wantDataTypes[i], k.FieldDataType)
}
})
}
}

View File

@@ -12,6 +12,7 @@ import (
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/antlr4-go/antlr/v4"
sqlbuilder "github.com/huandu/go-sqlbuilder"
@@ -25,6 +26,7 @@ const stringMatchingOperatorDocURL = "https://signoz.io/docs/userguide/operators
// to convert the parsed filter expressions into ClickHouse WHERE clause.
type filterExpressionVisitor struct {
context context.Context
orgID valuer.UUID
fieldMapper qbtypes.FieldMapper
conditionBuilder qbtypes.ConditionBuilder
warnings []string
@@ -45,6 +47,7 @@ type filterExpressionVisitor struct {
type FilterExprVisitorOpts struct {
Context context.Context
OrgID valuer.UUID
Logger *slog.Logger
FieldMapper qbtypes.FieldMapper
ConditionBuilder qbtypes.ConditionBuilder
@@ -62,6 +65,7 @@ type FilterExprVisitorOpts struct {
func newFilterExpressionVisitor(opts FilterExprVisitorOpts) *filterExpressionVisitor {
return &filterExpressionVisitor{
context: opts.Context,
orgID: opts.OrgID,
fieldMapper: opts.FieldMapper,
conditionBuilder: opts.ConditionBuilder,
fieldKeys: opts.FieldKeys,
@@ -365,12 +369,11 @@ func (v *filterExpressionVisitor) VisitPrimary(ctx *grammar.PrimaryContext) any
// VisitComparison handles all comparison operators.
func (v *filterExpressionVisitor) VisitComparison(ctx *grammar.ComparisonContext) any {
key := v.Visit(ctx.Key()).(*telemetrytypes.TelemetryFieldKey)
matching := matchingFieldKeys(key, v.fieldKeys)
matching := MatchingFieldKeys(key, v.fieldKeys)
// Skip resource filtering on the main table when a sub-query covers it. Resolve
// ambiguity first (resource+attribute defaults to resource), then drop resource
// matches; skip the term if nothing remains. Empty matches flow to the builder.
if v.skipResourceFilter && len(matching) > 0 {
// Skip resource filtering on the main table when a sub-query covers it; a resolving
// condition builder applies this itself in ConditionForKeys.
if _, resolvesKeys := v.conditionBuilder.(qbtypes.ResolvingConditionBuilder); v.skipResourceFilter && len(matching) > 0 && !resolvesKeys {
resolved, warning := ResolveKeys(key, matching)
// emit the ambiguity warning even when the term is skipped below
v.addWarnings([]string{warning}, len(matching) > 1)
@@ -731,7 +734,7 @@ func (v *filterExpressionVisitor) VisitFunctionCall(ctx *grammar.FunctionCallCon
value := params[1:]
conds, ok := v.buildConditions(key, matchingFieldKeys(key, v.fieldKeys), operator, value)
conds, ok := v.buildConditions(key, MatchingFieldKeys(key, v.fieldKeys), operator, value)
if !ok {
return ErrorConditionLiteral
}
@@ -811,10 +814,20 @@ func (v *filterExpressionVisitor) VisitKey(ctx *grammar.KeyContext) any {
}
// buildConditions invokes the condition builder for a filter term, folding its
// warnings/errors into visitor state. It returns the conditions and false if an error
// was recorded.
// warnings/errors into visitor state; returns false if an error was recorded.
func (v *filterExpressionVisitor) buildConditions(key *telemetrytypes.TelemetryFieldKey, matching []*telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) ([]string, bool) {
conds, warns, err := v.conditionBuilder.ConditionFor(v.context, v.startNs, v.endNs, key, matching, op, value, v.builder)
var (
conds []string
warns []string
err error
)
// A resolving condition builder owns key resolution, so hand it the raw key + full map;
// other signals use the pre-matched ConditionFor path.
if rcb, ok := v.conditionBuilder.(qbtypes.ResolvingConditionBuilder); ok {
conds, warns, err = rcb.ConditionForKeys(v.context, v.orgID, v.startNs, v.endNs, key, v.fieldKeys, qbtypes.ConditionBuilderOptions{SkipResourceFilter: v.skipResourceFilter}, op, value, v.builder)
} else {
conds, warns, err = v.conditionBuilder.ConditionFor(v.context, v.orgID, v.startNs, v.endNs, key, matching, op, value, v.builder)
}
if err != nil {
_, _, _, _, errURL, _ := errors.Unwrapb(err)
assignIfEmpty(&v.mainErrorURL, errURL)
@@ -870,10 +883,9 @@ func assignIfEmpty(s *string, value string) {
}
}
// matchingFieldKeys returns the field keys from the provided map that match the given
// key, honoring any context/data type the user specified. It also resolves the case
// where GetFieldKeyFromKeyText split a context off a name that legitimately contained it.
func matchingFieldKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
// MatchingFieldKeys returns the field keys from the map that match the given key,
// honoring any context/data type the user specified.
func MatchingFieldKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
fieldKeysForName := []*telemetrytypes.TelemetryFieldKey{}
// match by name; keep items whose context and data type match (unspecified matches any)

View File

@@ -10,6 +10,7 @@ import (
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/antlr4-go/antlr/v4"
sqlbuilder "github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
@@ -588,7 +589,7 @@ func TestVisitKey(t *testing.T) {
// VisitKey only parses; the condition builder matches, resolves ambiguity
// and decides not-found handling. Replay that here against the generic
// builder behavior (error unless the key is ignored).
matching := matchingFieldKeys(key, tt.fieldKeys)
matching := MatchingFieldKeys(key, tt.fieldKeys)
keys, warning := ResolveKeys(key, matching)
var gotErrors []string
@@ -746,6 +747,7 @@ type resourceConditionBuilder struct{}
func (b *resourceConditionBuilder) ConditionFor(
_ context.Context,
_ valuer.UUID,
_ uint64,
_ uint64,
key *telemetrytypes.TelemetryFieldKey,
@@ -781,6 +783,7 @@ type conditionBuilder struct{}
func (b *conditionBuilder) ConditionFor(
_ context.Context,
_ valuer.UUID,
_ uint64,
_ uint64,
key *telemetrytypes.TelemetryFieldKey,

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -172,6 +173,7 @@ func (c *conditionBuilder) conditionFor(
func (c *conditionBuilder) ConditionFor(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,

View File

@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/exp/maps"
@@ -97,6 +98,7 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, _, _ uint64, key *telemetry
func (m *fieldMapper) ColumnExpressionFor(
ctx context.Context,
_ valuer.UUID,
tsStart, tsEnd uint64,
field *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -119,3 +121,9 @@ func (m *fieldMapper) ColumnExpressionFor(
return fmt.Sprintf("%s AS `%s`", sqlbuilder.Escape(fieldExpression), field.Name), nil
}
// CandidateKeys returns nil: audit has no synthesize-on-unknown-key fallback, so an
// unknown key stays unresolved and the caller errors.
func (m *fieldMapper) CandidateKeys(_ context.Context, _ valuer.UUID, _ *telemetrytypes.TelemetryFieldKey, _ any, _ map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return nil
}

View File

@@ -13,6 +13,7 @@ import (
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -66,6 +67,7 @@ func NewAuditQueryStatementBuilder(
func (b *auditQueryStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
@@ -88,11 +90,11 @@ func (b *auditQueryStatementBuilder) Build(
var stmt *qbtypes.Statement
switch requestType {
case qbtypes.RequestTypeRaw, qbtypes.RequestTypeRawStream:
stmt, err = b.buildListQuery(ctx, q, query, start, end, keys, variables)
stmt, err = b.buildListQuery(ctx, orgID, q, query, start, end, keys, variables)
case qbtypes.RequestTypeTimeSeries:
stmt, err = b.buildTimeSeriesQuery(ctx, q, query, start, end, keys, variables)
stmt, err = b.buildTimeSeriesQuery(ctx, orgID, q, query, start, end, keys, variables)
case qbtypes.RequestTypeScalar:
stmt, err = b.buildScalarQuery(ctx, q, query, start, end, keys, false, variables)
stmt, err = b.buildScalarQuery(ctx, orgID, q, query, start, end, keys, false, variables)
default:
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported request type: %s", requestType)
}
@@ -201,6 +203,7 @@ func (b *auditQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFiel
func (b *auditQueryStatementBuilder) buildListQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
@@ -212,7 +215,7 @@ func (b *auditQueryStatementBuilder) buildListQuery(
cteArgs [][]any
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
if frag, args, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
@@ -242,7 +245,7 @@ func (b *auditQueryStatementBuilder) buildListQuery(
continue
}
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &query.SelectFields[index], keys)
colExpr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &query.SelectFields[index], keys)
if err != nil {
return nil, err
}
@@ -252,13 +255,13 @@ func (b *auditQueryStatementBuilder) buildListQuery(
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {
return nil, err
}
for _, orderBy := range query.Order {
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &orderBy.Key.TelemetryFieldKey, keys)
colExpr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &orderBy.Key.TelemetryFieldKey, keys)
if err != nil {
return nil, err
}
@@ -292,6 +295,7 @@ func (b *auditQueryStatementBuilder) buildListQuery(
func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
@@ -303,7 +307,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
cteArgs [][]any
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
if frag, args, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
@@ -319,7 +323,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
fieldNames := make([]string, 0, len(query.GroupBy))
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, false)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, orgID, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, false)
if err != nil {
return nil, err
}
@@ -332,7 +336,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
allAggChArgs := make([]any, 0)
for i, agg := range query.Aggregations {
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(ctx, start, end, agg.Expression, uint64(query.StepInterval.Seconds()), keys)
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(ctx, orgID, start, end, agg.Expression, uint64(query.StepInterval.Seconds()), keys)
if err != nil {
return nil, err
}
@@ -342,7 +346,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {
return nil, err
}
@@ -352,7 +356,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
if query.Limit > 0 && len(query.GroupBy) > 0 {
cteSB := sqlbuilder.NewSelectBuilder()
cteStmt, err := b.buildScalarQuery(ctx, cteSB, query, start, end, keys, true, variables)
cteStmt, err := b.buildScalarQuery(ctx, orgID, cteSB, query, start, end, keys, true, variables)
if err != nil {
return nil, err
}
@@ -430,6 +434,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
func (b *auditQueryStatementBuilder) buildScalarQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
@@ -442,7 +447,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
cteArgs [][]any
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
if frag, args, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables); err != nil {
return nil, err
} else if frag != "" && !skipResourceCTE {
cteFragments = append(cteFragments, frag)
@@ -454,7 +459,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
var allGroupByArgs []any
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, false)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, orgID, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, false)
if err != nil {
return nil, err
}
@@ -469,7 +474,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
if len(query.Aggregations) > 0 {
for idx := range query.Aggregations {
aggExpr := query.Aggregations[idx]
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(ctx, start, end, aggExpr.Expression, rateInterval, keys)
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(ctx, orgID, start, end, aggExpr.Expression, rateInterval, keys)
if err != nil {
return nil, err
}
@@ -480,7 +485,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
if err != nil {
return nil, err
}
@@ -532,6 +537,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
func (b *auditQueryStatementBuilder) addFilterCondition(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
@@ -544,6 +550,7 @@ func (b *auditQueryStatementBuilder) addFilterCondition(
if query.Filter != nil && query.Filter.Expression != "" {
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
@@ -591,12 +598,13 @@ func aggOrderBy(k qbtypes.OrderBy, q qbtypes.QueryBuilderQuery[qbtypes.LogAggreg
func (b *auditQueryStatementBuilder) maybeAttachResourceFilter(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
variables map[string]qbtypes.VariableItem,
) (cteSQL string, cteArgs []any, err error) {
stmt, err := b.resourceFilterStmtBuilder.Build(ctx, start, end, qbtypes.RequestTypeRaw, query, variables)
stmt, err := b.resourceFilterStmtBuilder.Build(ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables)
if err != nil {
return "", nil, err
}

View File

@@ -11,6 +11,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
@@ -213,7 +214,7 @@ func TestStatementBuilder(t *testing.T) {
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
q, err := statementBuilder.Build(ctx, 1747947419000, 1747983448000, testCase.requestType, testCase.query, nil)
q, err := statementBuilder.Build(ctx, valuer.UUID{}, 1747947419000, 1747983448000, testCase.requestType, testCase.query, nil)
if testCase.expectedErr != nil {
require.Error(t, err)
require.Contains(t, err.Error(), testCase.expectedErr.Error())

View File

@@ -372,6 +372,7 @@ func (c *conditionBuilder) conditionFor(
func (c *conditionBuilder) ConditionFor(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,

View File

@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -131,7 +132,7 @@ func TestExistsConditionForWithEvolutions(t *testing.T) {
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
cond, _, err := conditionBuilder.ConditionFor(ctx, tc.startTs, tc.endTs, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
cond, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, tc.startTs, tc.endTs, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
sb.Where(cond...)
if tc.expectedError != nil {
@@ -522,7 +523,7 @@ func TestConditionFor(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
tc.key.Evolutions = tc.evolutions
cond, _, err := conditionBuilder.ConditionFor(ctx, 0, 0, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
cond, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
sb.Where(cond...)
if tc.expectedError != nil {

View File

@@ -242,6 +242,7 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, _, _ uint64, key *telemetry
func (m *fieldMapper) ColumnExpressionFor(
ctx context.Context,
_ valuer.UUID,
tsStart, tsEnd uint64,
field *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -285,6 +286,12 @@ func (m *fieldMapper) ColumnExpressionFor(
return fmt.Sprintf("%s AS `%s`", sqlbuilder.Escape(fieldExpression), field.Name), nil
}
// CandidateKeys returns nil: logs has no synthesize-on-unknown-key fallback, so an
// unknown key stays unresolved and the caller errors.
func (m *fieldMapper) CandidateKeys(_ context.Context, _ valuer.UUID, _ *telemetrytypes.TelemetryFieldKey, _ any, _ map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return nil
}
// buildFieldForJSON builds the field expression for body JSON fields using arrayConcat pattern.
func (m *fieldMapper) buildFieldForJSON(key *telemetrytypes.TelemetryFieldKey) (string, error) {
plan := key.JSONPlan

View File

@@ -14,6 +14,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
@@ -94,7 +95,7 @@ func TestJSONStmtBuilder_TimeSeries(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErrContains != "" {
require.Error(t, err)
@@ -155,7 +156,7 @@ func TestStmtBuilderTimeSeriesBodyGroupByPromoted(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErrContains != "" {
require.Error(t, err)
require.Contains(t, err.Error(), c.expectedErrContains)
@@ -308,7 +309,7 @@ func TestJSONStmtBuilder_PrimitivePaths(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, qbtypes.RequestTypeRaw,
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, qbtypes.RequestTypeRaw,
qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Filter: &qbtypes.Filter{Expression: c.filter},
@@ -477,7 +478,7 @@ func TestStatementBuilderListQueryBodyPromoted(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -779,7 +780,7 @@ func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, qbtypes.RequestTypeRaw,
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, qbtypes.RequestTypeRaw,
qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Filter: &qbtypes.Filter{Expression: c.filter},
@@ -904,7 +905,7 @@ func TestJSONStmtBuilder_IndexedPaths(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, qbtypes.RequestTypeRaw, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, qbtypes.RequestTypeRaw, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
require.Contains(t, err.Error(), c.expectedErr.Error())
@@ -991,7 +992,7 @@ func TestJSONStmtBuilder_SelectField(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErrContains != "" {
require.Error(t, err)
require.Contains(t, err.Error(), c.expectedErrContains)
@@ -1068,7 +1069,7 @@ func TestJSONStmtBuilder_OrderBy(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErrContains != "" {
require.Error(t, err)
require.Contains(t, err.Error(), c.expectedErrContains)
@@ -1131,7 +1132,7 @@ func TestResourceAggrAndGroupBy_WithJSONEnabled(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErrContains != "" {
require.Error(t, err)
require.Contains(t, err.Error(), c.expectedErrContains)

View File

@@ -80,6 +80,7 @@ func NewLogQueryStatementBuilder(
// Build builds a SQL query for logs based on the given parameters.
func (b *logQueryStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
@@ -106,11 +107,11 @@ func (b *logQueryStatementBuilder) Build(
var stmt *qbtypes.Statement
switch requestType {
case qbtypes.RequestTypeRaw, qbtypes.RequestTypeRawStream:
stmt, err = b.buildListQuery(ctx, q, query, start, end, keys, variables)
stmt, err = b.buildListQuery(ctx, orgID, q, query, start, end, keys, variables)
case qbtypes.RequestTypeTimeSeries:
stmt, err = b.buildTimeSeriesQuery(ctx, q, query, start, end, keys, variables)
stmt, err = b.buildTimeSeriesQuery(ctx, orgID, q, query, start, end, keys, variables)
case qbtypes.RequestTypeScalar:
stmt, err = b.buildScalarQuery(ctx, q, query, start, end, keys, false, variables)
stmt, err = b.buildScalarQuery(ctx, orgID, q, query, start, end, keys, false, variables)
default:
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported request type: %s", requestType)
}
@@ -264,6 +265,7 @@ func (b *logQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldK
// buildListQuery builds a query for list panel type.
func (b *logQueryStatementBuilder) buildListQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
@@ -278,7 +280,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
bodyJSONEnabled = b.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables)
if err != nil {
return nil, err
}
@@ -314,7 +316,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
}
// get column expression for the field - use array index directly to avoid pointer to loop variable
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &query.SelectFields[index], keys)
colExpr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &query.SelectFields[index], keys)
if err != nil {
return nil, err
}
@@ -324,7 +326,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables, skipResourceFilter)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
if err != nil {
return nil, err
@@ -333,7 +335,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
// Add order by
for _, orderBy := range query.Order {
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &orderBy.Key.TelemetryFieldKey, keys)
colExpr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &orderBy.Key.TelemetryFieldKey, keys)
if err != nil {
return nil, err
}
@@ -368,6 +370,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
@@ -382,7 +385,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
bodyJSONEnabled = b.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables)
if err != nil {
return nil, err
}
@@ -401,7 +404,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
// Keep original column expressions so we can build the tuple
fieldNames := make([]string, 0, len(query.GroupBy))
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, bodyJSONEnabled)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, orgID, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return nil, err
}
@@ -416,7 +419,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
allAggChArgs := make([]any, 0)
for i, agg := range query.Aggregations {
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(
ctx, start, end, agg.Expression,
ctx, orgID, start, end, agg.Expression,
uint64(query.StepInterval.Seconds()),
keys,
)
@@ -430,7 +433,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
// Add FROM clause
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables, skipResourceFilter)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
if err != nil {
return nil, err
@@ -442,7 +445,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
if query.Limit > 0 && len(query.GroupBy) > 0 {
// build the scalar “top/bottom-N” query in its own builder.
cteSB := sqlbuilder.NewSelectBuilder()
cteStmt, err := b.buildScalarQuery(ctx, cteSB, query, start, end, keys, true, variables)
cteStmt, err := b.buildScalarQuery(ctx, orgID, cteSB, query, start, end, keys, true, variables)
if err != nil {
return nil, err
}
@@ -527,6 +530,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
// buildScalarQuery builds a query for scalar panel type.
func (b *logQueryStatementBuilder) buildScalarQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
@@ -542,7 +546,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
bodyJSONEnabled = b.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables)
if err != nil {
return nil, err
}
@@ -556,7 +560,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
var allGroupByArgs []any
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, bodyJSONEnabled)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, orgID, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return nil, err
}
@@ -574,7 +578,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
for idx := range query.Aggregations {
aggExpr := query.Aggregations[idx]
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(
ctx, start, end, aggExpr.Expression,
ctx, orgID, start, end, aggExpr.Expression,
rateInterval,
keys,
)
@@ -589,7 +593,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables, skipResourceFilter)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
if err != nil {
return nil, err
@@ -648,6 +652,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
// buildFilterCondition builds SQL condition from filter expression.
func (b *logQueryStatementBuilder) addFilterCondition(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
@@ -663,6 +668,7 @@ func (b *logQueryStatementBuilder) addFilterCondition(
// add filter expression
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
@@ -713,6 +719,7 @@ func aggOrderBy(k qbtypes.OrderBy, q qbtypes.QueryBuilderQuery[qbtypes.LogAggreg
func (b *logQueryStatementBuilder) maybeAttachResourceFilter(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation],
start, end uint64,
@@ -720,7 +727,7 @@ func (b *logQueryStatementBuilder) maybeAttachResourceFilter(
) (cteSQL string, cteArgs []any, skipResourceFilter bool, err error) {
if b.skipResourceFingerprintEnabled {
decision, err := b.resourceFilterResolver.Resolve(ctx, query, start, end, variables)
decision, err := b.resourceFilterResolver.Resolve(ctx, orgID, query, start, end, variables)
if err != nil {
return "", nil, true, err
}
@@ -733,7 +740,7 @@ func (b *logQueryStatementBuilder) maybeAttachResourceFilter(
}
stmt, err := b.resourceFilterResolver.StatementBuilder().Build(
ctx, start, end, qbtypes.RequestTypeRaw, query, variables,
ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables,
)
if err != nil {
return "", nil, true, err

View File

@@ -16,6 +16,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
@@ -237,7 +238,7 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(ctx, c.startTs, c.endTs, c.requestType, c.query, nil)
q, err := statementBuilder.Build(ctx, valuer.UUID{}, c.startTs, c.endTs, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -381,7 +382,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(ctx, 1747947419000, 1747983448000, c.requestType, c.query, c.variables)
q, err := statementBuilder.Build(ctx, valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, c.variables)
if c.expectedErr != nil {
require.Error(t, err)
@@ -530,7 +531,7 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(ctx, 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(ctx, valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -609,7 +610,7 @@ func TestStatementBuilderTimeSeriesBodyGroupBy(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(ctx, 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(ctx, valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErrContains != "" {
require.Error(t, err)
@@ -707,7 +708,7 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(ctx, 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(ctx, valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -1079,7 +1080,7 @@ func TestStmtBuilderBodyField(t *testing.T) {
100000,
)
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
require.Contains(t, err.Error(), c.expectedErr.Error())
@@ -1181,7 +1182,7 @@ func TestStmtBuilderBodyFullTextSearch(t *testing.T) {
100000,
)
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
require.Contains(t, err.Error(), c.expectedErr.Error())
@@ -1220,7 +1221,7 @@ func TestSkipResourceFingerprintLogs(t *testing.T) {
t.Run("disabled uses the legacy CTE", func(t *testing.T) {
sb := newSkipResourceFingerprintLogsBuilder(t, nil, false, threshold)
stmt, err := sb.Build(context.Background(), startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
stmt, err := sb.Build(context.Background(), valuer.UUID{}, startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
require.NoError(t, err)
require.Contains(t, stmt.Query, "__resource_filter AS (SELECT fingerprint")
require.Contains(t, stmt.Query, "resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)")
@@ -1237,7 +1238,7 @@ func TestSkipResourceFingerprintLogs(t *testing.T) {
sb := newSkipResourceFingerprintLogsBuilder(t, mockStore, true, threshold)
stmt, err := sb.Build(context.Background(), startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
stmt, err := sb.Build(context.Background(), valuer.UUID{}, startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
require.NoError(t, err)
require.Contains(t, stmt.Query, "__resource_filter AS (SELECT fingerprint")
@@ -1257,7 +1258,7 @@ func TestSkipResourceFingerprintLogs(t *testing.T) {
sb := newSkipResourceFingerprintLogsBuilder(t, mockStore, true, threshold)
stmt, err := sb.Build(context.Background(), startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
stmt, err := sb.Build(context.Background(), valuer.UUID{}, startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
require.NoError(t, err)
require.NotContains(t, stmt.Query, "__resource_filter AS")

View File

@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -21,6 +22,7 @@ func NewConditionBuilder(fm qbtypes.FieldMapper) *conditionBuilder {
func (c *conditionBuilder) ConditionFor(
ctx context.Context,
orgID valuer.UUID,
tsStart, tsEnd uint64,
key *telemetrytypes.TelemetryFieldKey,
fieldKeysForName []*telemetrytypes.TelemetryFieldKey,

View File

@@ -6,6 +6,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -53,7 +54,7 @@ func TestConditionFor(t *testing.T) {
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
cond, _, err := conditionBuilder.ConditionFor(ctx, 0, 0, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
cond, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
sb.Where(cond...)
if tc.expectedError != nil {

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/exp/maps"
)
@@ -29,6 +30,12 @@ var (
type fieldMapper struct {
}
// CandidateKeys returns nil: this mapper has no attribute-map fallback, so a context-missing
// key stays unresolved and the caller errors.
func (m *fieldMapper) CandidateKeys(_ context.Context, _ valuer.UUID, _ *telemetrytypes.TelemetryFieldKey, _ any, _ map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return nil
}
func NewFieldMapper() qbtypes.FieldMapper {
return &fieldMapper{}
}
@@ -69,6 +76,7 @@ func (m *fieldMapper) FieldFor(ctx context.Context, startNs, endNs uint64, key *
func (m *fieldMapper) ColumnExpressionFor(
ctx context.Context,
_ valuer.UUID,
startNs, endNs uint64,
field *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,

View File

@@ -1446,20 +1446,20 @@ func (t *telemetryMetaStore) getRelatedValues(ctx context.Context, fieldValueSel
// search on attributes
key.FieldContext = telemetrytypes.FieldContextAttribute
attrConds, _, err := t.conditionBuilder.ConditionFor(ctx, 0, 0, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorContains, fieldValueSelector.Value, sb)
attrConds, _, err := t.conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorContains, fieldValueSelector.Value, sb)
if err == nil {
conds = append(conds, attrConds...)
}
// search on resource
key.FieldContext = telemetrytypes.FieldContextResource
resourceConds, _, err := t.conditionBuilder.ConditionFor(ctx, 0, 0, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorContains, fieldValueSelector.Value, sb)
resourceConds, _, err := t.conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorContains, fieldValueSelector.Value, sb)
if err == nil {
conds = append(conds, resourceConds...)
}
key.FieldContext = origContext
} else {
keyConds, _, err := t.conditionBuilder.ConditionFor(ctx, 0, 0, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorContains, fieldValueSelector.Value, sb)
keyConds, _, err := t.conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, key, []*telemetrytypes.TelemetryFieldKey{key}, qbtypes.FilterOperatorContains, fieldValueSelector.Value, sb)
if err == nil {
conds = append(conds, keyConds...)
}

View File

@@ -12,6 +12,7 @@ import (
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -45,6 +46,7 @@ func NewMeterQueryStatementBuilder(
func (b *meterQueryStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
_ qbtypes.RequestType,
@@ -59,11 +61,12 @@ func (b *meterQueryStatementBuilder) Build(
start, end = querybuilder.AdjustedMetricTimeRange(start, end, uint64(query.StepInterval.Seconds()), query)
return b.buildPipelineStatement(ctx, start, end, query, keys, variables)
return b.buildPipelineStatement(ctx, orgID, start, end, query, keys, variables)
}
func (b *meterQueryStatementBuilder) buildPipelineStatement(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -76,7 +79,7 @@ func (b *meterQueryStatementBuilder) buildPipelineStatement(
if qbtypes.CanShortCircuitDelta(query.Aggregations[0]) {
// spatial_aggregation_cte directly for certain delta queries
if frag, args, err := b.buildTemporalAggDeltaFastPath(ctx, start, end, query, keys, variables); err != nil {
if frag, args, err := b.buildTemporalAggDeltaFastPath(ctx, orgID, start, end, query, keys, variables); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
@@ -84,7 +87,7 @@ func (b *meterQueryStatementBuilder) buildPipelineStatement(
}
} else {
// temporal_aggregation_cte
if frag, args, err := b.buildTemporalAggregationCTE(ctx, start, end, query, keys, variables); err != nil {
if frag, args, err := b.buildTemporalAggregationCTE(ctx, orgID, start, end, query, keys, variables); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
@@ -106,6 +109,7 @@ func (b *meterQueryStatementBuilder) buildPipelineStatement(
func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -122,7 +126,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
stepSec,
))
for _, g := range query.GroupBy {
col, err := b.fm.ColumnExpressionFor(ctx, start, end, &g.TelemetryFieldKey, keys)
col, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &g.TelemetryFieldKey, keys)
if err != nil {
return "", nil, err
}
@@ -148,6 +152,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
if query.Filter != nil && query.Filter.Expression != "" {
filterWhere, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
@@ -177,19 +182,21 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
func (b *meterQueryStatementBuilder) buildTemporalAggregationCTE(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
) (string, []any, error) {
if query.Aggregations[0].Temporality == metrictypes.Delta {
return b.buildTemporalAggDelta(ctx, start, end, query, keys, variables)
return b.buildTemporalAggDelta(ctx, orgID, start, end, query, keys, variables)
}
return b.buildTemporalAggCumulativeOrUnspecified(ctx, start, end, query, keys, variables)
return b.buildTemporalAggCumulativeOrUnspecified(ctx, orgID, start, end, query, keys, variables)
}
func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -208,7 +215,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
))
for _, g := range query.GroupBy {
col, err := b.fm.ColumnExpressionFor(ctx, start, end, &g.TelemetryFieldKey, keys)
col, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &g.TelemetryFieldKey, keys)
if err != nil {
return "", nil, err
}
@@ -237,6 +244,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
if query.Filter != nil && query.Filter.Expression != "" {
filterWhere, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
@@ -268,6 +276,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -284,7 +293,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
stepSec,
))
for _, g := range query.GroupBy {
col, err := b.fm.ColumnExpressionFor(ctx, start, end, &g.TelemetryFieldKey, keys)
col, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &g.TelemetryFieldKey, keys)
if err != nil {
return "", nil, err
}
@@ -307,6 +316,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
if query.Filter != nil && query.Filter.Expression != "" {
filterWhere, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,

View File

@@ -12,6 +12,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
@@ -181,7 +182,7 @@ func TestStatementBuilder(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -143,6 +144,7 @@ func (c *conditionBuilder) conditionFor(
func (c *conditionBuilder) ConditionFor(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,

View File

@@ -6,6 +6,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -234,7 +235,7 @@ func TestConditionFor(t *testing.T) {
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
cond, _, err := conditionBuilder.ConditionFor(ctx, 0, 0, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
cond, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
sb.Where(cond...)
if tc.expectedError != nil {
@@ -289,7 +290,7 @@ func TestConditionForMultipleKeys(t *testing.T) {
t.Run(tc.name, func(t *testing.T) {
var err error
for _, key := range tc.keys {
cond, _, err := conditionBuilder.ConditionFor(ctx, 0, 0, &key, []*telemetrytypes.TelemetryFieldKey{&key}, tc.operator, tc.value, sb)
cond, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, []*telemetrytypes.TelemetryFieldKey{&key}, tc.operator, tc.value, sb)
sb.Where(cond...)
if err != nil {
t.Fatalf("Error getting condition for key %s: %v", key.Name, err)

View File

@@ -8,6 +8,7 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -37,6 +38,12 @@ var (
type fieldMapper struct{}
// CandidateKeys returns nil: metrics has no attribute-map fallback, so a context-missing
// key stays unresolved and the caller errors.
func (m *fieldMapper) CandidateKeys(_ context.Context, _ valuer.UUID, _ *telemetrytypes.TelemetryFieldKey, _ any, _ map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return nil
}
func NewFieldMapper() qbtypes.FieldMapper {
return &fieldMapper{}
}
@@ -92,6 +99,7 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, tsStart, tsEnd uint64, key
func (m *fieldMapper) ColumnExpressionFor(
ctx context.Context,
_ valuer.UUID,
startNs, endNs uint64,
field *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,

View File

@@ -11,6 +11,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
@@ -127,7 +128,7 @@ func TestReducedStatementBuilder(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
got, err := sb.Build(context.Background(), start, end, qbtypes.RequestTypeTimeSeries, c.query, nil)
got, err := sb.Build(context.Background(), valuer.UUID{}, start, end, qbtypes.RequestTypeTimeSeries, c.query, nil)
require.NoError(t, err)
require.Equal(t, c.expected.Query, got.Query)
require.Equal(t, c.expected.Args, got.Args)
@@ -137,7 +138,7 @@ func TestReducedStatementBuilder(t *testing.T) {
t.Run("buffer_recent_window", func(t *testing.T) {
now := time.Now().UnixMilli()
q := reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationLatest, metrictypes.SpaceAggregationSum)
got, err := sb.Build(context.Background(), uint64(now-2*time.Hour.Milliseconds()), uint64(now), qbtypes.RequestTypeTimeSeries, q, nil)
got, err := sb.Build(context.Background(), valuer.UUID{}, uint64(now-2*time.Hour.Milliseconds()), uint64(now), qbtypes.RequestTypeTimeSeries, q, nil)
require.NoError(t, err)
require.Contains(t, got.Query, "signoz_metrics.distributed_samples_v4_buffer")
require.Contains(t, got.Query, "signoz_metrics.time_series_v4_buffer")
@@ -148,7 +149,7 @@ func TestReducedStatementBuilder(t *testing.T) {
t.Run("not_reduced", func(t *testing.T) {
q := reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationLatest, metrictypes.SpaceAggregationSum)
q.Aggregations[0].Reduced = false
got, err := sb.Build(context.Background(), start, end, qbtypes.RequestTypeTimeSeries, q, nil)
got, err := sb.Build(context.Background(), valuer.UUID{}, start, end, qbtypes.RequestTypeTimeSeries, q, nil)
require.NoError(t, err)
require.NotContains(t, got.Query, "UNION ALL")
require.NotContains(t, got.Query, "reduced")

View File

@@ -12,6 +12,7 @@ import (
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/exp/slices"
)
@@ -90,6 +91,7 @@ func GetKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation])
func (b *MetricQueryStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
_ qbtypes.RequestType,
@@ -121,11 +123,12 @@ func (b *MetricQueryStatementBuilder) Build(
start, end = querybuilder.AdjustedMetricTimeRange(start, end, uint64(query.StepInterval.Seconds()), query)
return b.buildPipelineStatement(ctx, start, end, query, keys, variables)
return b.buildPipelineStatement(ctx, orgID, start, end, query, keys, variables)
}
func (b *MetricQueryStatementBuilder) buildPipelineStatement(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -195,7 +198,7 @@ func (b *MetricQueryStatementBuilder) buildPipelineStatement(
var timeSeriesCTEArgs []any
var err error
if timeSeriesCTE, timeSeriesCTEArgs, err = b.buildTimeSeriesCTE(ctx, tsStart, tsEnd, query, keys, variables, tsTable); err != nil {
if timeSeriesCTE, timeSeriesCTEArgs, err = b.buildTimeSeriesCTE(ctx, orgID, tsStart, tsEnd, query, keys, variables, tsTable); err != nil {
return nil, err
}
@@ -228,7 +231,7 @@ func (b *MetricQueryStatementBuilder) buildPipelineStatement(
if agg.Reduced && !useBuffer {
var tsCTE string
var tsArgs []any
if tsCTE, tsArgs, err = b.buildReducedTimeSeriesCTE(ctx, start, end, query, keys, variables); err != nil {
if tsCTE, tsArgs, err = b.buildReducedTimeSeriesCTE(ctx, orgID, start, end, query, keys, variables); err != nil {
return nil, err
}
if temporalFrag, temporalArgs, ok := b.buildReducedTemporalAggregationCTE(start, end, query, tsCTE, tsArgs); ok {
@@ -270,6 +273,7 @@ func unionStatements(main, reduced *qbtypes.Statement, query qbtypes.QueryBuilde
func (b *MetricQueryStatementBuilder) buildReducedTimeSeriesCTE(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -282,6 +286,7 @@ func (b *MetricQueryStatementBuilder) buildReducedTimeSeriesCTE(
if query.Filter != nil && query.Filter.Expression != "" {
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
@@ -299,7 +304,7 @@ func (b *MetricQueryStatementBuilder) buildReducedTimeSeriesCTE(
sb.From(fmt.Sprintf("%s.%s", DBName, TimeseriesV4ReducedLocalTableName))
sb.Select("fingerprint")
for _, g := range query.GroupBy {
col, err := b.fm.ColumnExpressionFor(ctx, start, end, &g.TelemetryFieldKey, keys)
col, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &g.TelemetryFieldKey, keys)
if err != nil {
return "", nil, err
}
@@ -454,6 +459,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
ctx context.Context,
orgID valuer.UUID,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
@@ -468,6 +474,7 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
if query.Filter != nil && query.Filter.Expression != "" {
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
@@ -486,7 +493,7 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
sb.Select("fingerprint")
for _, g := range query.GroupBy {
col, err := b.fm.ColumnExpressionFor(ctx, start, end, &g.TelemetryFieldKey, keys)
col, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &g.TelemetryFieldKey, keys)
if err != nil {
return "", nil, err
}

View File

@@ -11,6 +11,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
@@ -245,7 +246,7 @@ func TestStatementBuilder(t *testing.T) {
},
expected: qbtypes.Statement{
Query: "WITH __temporal_aggregation_cte AS (SELECT ts, `k8s.statefulset.name`, multiIf(row_number() OVER rate_window = 1, nan, (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) < 0, per_series_value / (ts - lagInFrame(ts, 1) OVER rate_window), (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) / (ts - lagInFrame(ts, 1) OVER rate_window)) AS per_series_value FROM (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(30)) AS ts, `k8s.statefulset.name`, max(value) AS per_series_value FROM signoz_metrics.distributed_samples_v4 AS points INNER JOIN (SELECT fingerprint, JSONExtractString(labels, 'k8s.statefulset.name') AS `k8s.statefulset.name` FROM signoz_metrics.time_series_v4_6hrs WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? AND JSONExtractString(labels, 'k8s.statefulset.name') = ? GROUP BY fingerprint, `k8s.statefulset.name`) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts, `k8s.statefulset.name` ORDER BY fingerprint, ts) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)), __spatial_aggregation_cte AS (SELECT ts, `k8s.statefulset.name`, sum(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts, `k8s.statefulset.name`) SELECT * FROM __spatial_aggregation_cte ORDER BY `k8s.statefulset.name`, ts",
Args: []any{"signoz_calls_total", uint64(1747936800000), uint64(1747983420000), "cumulative", false, "my-statefulset", "signoz_calls_total", uint64(1747947360000), uint64(1747983420000), 0},
Args: []any{"signoz_calls_total", uint64(1747936800000), uint64(1747983420000), "cumulative", false, "my-statefulset", "signoz_calls_total", uint64(1747947360000), uint64(1747983420000), 0},
},
expectedErr: nil,
},
@@ -283,7 +284,7 @@ func TestStatementBuilder(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)

View File

@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -15,7 +16,10 @@ type defaultConditionBuilder struct {
fm qbtypes.FieldMapper
}
var _ qbtypes.ConditionBuilder = (*defaultConditionBuilder)(nil)
var (
_ qbtypes.ConditionBuilder = (*defaultConditionBuilder)(nil)
_ qbtypes.ResolvingConditionBuilder = (*defaultConditionBuilder)(nil)
)
func NewConditionBuilder(fm qbtypes.FieldMapper) *defaultConditionBuilder {
return &defaultConditionBuilder{fm: fm}
@@ -43,8 +47,11 @@ func keyIndexFilter(key *telemetrytypes.TelemetryFieldKey) any {
return fmt.Sprintf(`%%%s%%`, key.Name)
}
// ConditionFor builds resource-fingerprint conditions from the caller's pre-matched
// fieldKeysForName. Resolution and the build live in conditionsForKeys.
func (b *defaultConditionBuilder) ConditionFor(
ctx context.Context,
_ valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
@@ -53,6 +60,37 @@ func (b *defaultConditionBuilder) ConditionFor(
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
return b.conditionsForKeys(ctx, startNs, endNs, key, fieldKeysForName, op, value, sb)
}
// ConditionForKeys owns key resolution from the full metadata map so the where-clause
// visitor need not pre-resolve; SkipResourceFilter is not applicable here (the fingerprint
// table only stores resource attributes). See qbtypes.ResolvingConditionBuilder.
func (b *defaultConditionBuilder) ConditionForKeys(
ctx context.Context,
_ valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
_ qbtypes.ConditionBuilderOptions,
op qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
return b.conditionsForKeys(ctx, startNs, endNs, key, querybuilder.MatchingFieldKeys(key, keys), op, value, sb)
}
func (b *defaultConditionBuilder) conditionsForKeys(
ctx context.Context,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
matches []*telemetrytypes.TelemetryFieldKey,
op qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
// has/hasAny/hasAll/hasToken are logs-body-only functions; they never apply to the
// resource fingerprint table, so skip them (the main query still evaluates them).
@@ -60,7 +98,7 @@ func (b *defaultConditionBuilder) ConditionFor(
return nil, nil, nil
}
keys, warning := querybuilder.ResolveKeys(key, fieldKeysForName)
keys, warning := querybuilder.ResolveKeys(key, matches)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)

View File

@@ -6,6 +6,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -205,7 +206,7 @@ func TestConditionBuilder(t *testing.T) {
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
cond, _, err := conditionBuilder.ConditionFor(context.Background(), 0, 0, tc.key, []*telemetrytypes.TelemetryFieldKey{tc.key}, tc.op, tc.value, sb)
cond, _, err := conditionBuilder.ConditionFor(context.Background(), valuer.UUID{}, 0, 0, tc.key, []*telemetrytypes.TelemetryFieldKey{tc.key}, tc.op, tc.value, sb)
sb.Where(cond...)
if tc.expectedErr != nil {

View File

@@ -7,6 +7,7 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
var (
@@ -21,6 +22,12 @@ type defaultFieldMapper struct{}
var _ qbtypes.FieldMapper = (*defaultFieldMapper)(nil)
// CandidateKeys returns nil: the resource filter has no attribute-map fallback, so a
// context-missing key stays unresolved and the caller errors.
func (m *defaultFieldMapper) CandidateKeys(_ context.Context, _ valuer.UUID, _ *telemetrytypes.TelemetryFieldKey, _ any, _ map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return nil
}
func NewFieldMapper() *defaultFieldMapper {
return &defaultFieldMapper{}
}
@@ -64,6 +71,7 @@ func (m *defaultFieldMapper) FieldFor(
func (m *defaultFieldMapper) ColumnExpressionFor(
ctx context.Context,
_ valuer.UUID,
tsStart, tsEnd uint64,
key *telemetrytypes.TelemetryFieldKey,
_ map[string][]*telemetrytypes.TelemetryFieldKey,

View File

@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/telemetrystore"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type ResourceFingerprintResolver[T any] struct {
@@ -50,11 +51,12 @@ func (r *ResourceFingerprintResolver[T]) StatementBuilder() qbtypes.StatementBui
func (r *ResourceFingerprintResolver[T]) Resolve(
ctx context.Context,
orgID valuer.UUID,
query qbtypes.QueryBuilderQuery[T],
start, end uint64,
variables map[string]qbtypes.VariableItem,
) (qbtypes.ResourceFilterResolveKind, error) {
countStmt, err := r.stmtBuilder.BuildCount(ctx, start, end, query, variables)
countStmt, err := r.stmtBuilder.BuildCount(ctx, orgID, start, end, query, variables)
if err != nil {
return qbtypes.ResourceFilterResolveKindNoOp, err
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -87,6 +88,7 @@ func (b *resourceFilterStatementBuilder[T]) getKeySelectors(query qbtypes.QueryB
// Build builds a SQL query based on the given parameters.
func (b *resourceFilterStatementBuilder[T]) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
@@ -103,7 +105,7 @@ func (b *resourceFilterStatementBuilder[T]) Build(
return nil, err
}
isNoOp, err := b.addConditions(ctx, q, start, end, query, keys, variables)
isNoOp, err := b.addConditions(ctx, orgID, q, start, end, query, keys, variables)
if err != nil {
return nil, err
}
@@ -126,12 +128,13 @@ func (b *resourceFilterStatementBuilder[T]) Build(
// the resource filter. Returns (nil, nil) when the filter is a no-op.
func (b *resourceFilterStatementBuilder[T]) BuildCount(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
query qbtypes.QueryBuilderQuery[T],
variables map[string]qbtypes.VariableItem,
) (*qbtypes.Statement, error) {
inner, err := b.Build(ctx, start, end, qbtypes.RequestTypeRaw, query, variables)
inner, err := b.Build(ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables)
if err != nil || inner == nil {
return nil, err
}
@@ -146,6 +149,7 @@ func (b *resourceFilterStatementBuilder[T]) BuildCount(
// meaning the CTE would select all fingerprints and should be skipped entirely.
func (b *resourceFilterStatementBuilder[T]) addConditions(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
start, end uint64,
query qbtypes.QueryBuilderQuery[T],
@@ -159,6 +163,7 @@ func (b *resourceFilterStatementBuilder[T]) addConditions(
// warnings would be encountered as part of the main condition already
filterWhereClause, err := querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fieldMapper,
ConditionBuilder: b.conditionBuilder,

View File

@@ -9,6 +9,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
@@ -366,7 +367,7 @@ func TestResourceFilterStatementBuilder_Traces(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
stmt, err := builder.Build(context.Background(), c.start, c.end, qbtypes.RequestTypeTimeSeries, c.query, nil)
stmt, err := builder.Build(context.Background(), valuer.UUID{}, c.start, c.end, qbtypes.RequestTypeTimeSeries, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -559,7 +560,7 @@ func TestResourceFilterStatementBuilder_Logs(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
stmt, err := builder.Build(context.Background(), c.start, c.end, qbtypes.RequestTypeTimeSeries, c.query, nil)
stmt, err := builder.Build(context.Background(), valuer.UUID{}, c.start, c.end, qbtypes.RequestTypeTimeSeries, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -626,7 +627,7 @@ func TestResourceFilterStatementBuilder_Variables(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
stmt, err := builder.Build(context.Background(), c.start, c.end, qbtypes.RequestTypeTimeSeries, c.query, c.variables)
stmt, err := builder.Build(context.Background(), valuer.UUID{}, c.start, c.end, qbtypes.RequestTypeTimeSeries, c.query, c.variables)
if c.expectedErr != nil {
require.Error(t, err)

View File

@@ -13,6 +13,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/exp/maps"
)
@@ -21,7 +22,10 @@ type conditionBuilder struct {
fm qbtypes.FieldMapper
}
var _ qbtypes.ConditionBuilder = (*conditionBuilder)(nil)
var (
_ qbtypes.ConditionBuilder = (*conditionBuilder)(nil)
_ qbtypes.ResolvingConditionBuilder = (*conditionBuilder)(nil)
)
func NewConditionBuilder(fm qbtypes.FieldMapper) *conditionBuilder {
return &conditionBuilder{fm: fm}
@@ -254,8 +258,11 @@ func (c *conditionBuilder) conditionFor(
return "", nil
}
// ConditionFor builds the conditions for a filter term from the caller's pre-matched
// fieldKeysForName; skip-resource-filter is the caller's responsibility on this path.
func (c *conditionBuilder) ConditionFor(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
@@ -264,19 +271,76 @@ func (c *conditionBuilder) ConditionFor(
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
return c.conditionsForKeys(ctx, orgID, startNs, endNs, key, fieldKeysForName, false, operator, value, sb)
}
// ConditionForKeys owns key resolution from the full metadata map so the where-clause
// visitor hands over the raw key + map. See qbtypes.ResolvingConditionBuilder.
func (c *conditionBuilder) ConditionForKeys(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
options qbtypes.ConditionBuilderOptions,
operator qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
return c.conditionsForKeys(ctx, orgID, startNs, endNs, key, querybuilder.MatchingFieldKeys(key, keys), options.SkipResourceFilter, operator, value, sb)
}
// conditionsForKeys resolves matches to the key(s) to filter on (ResolveKeys, else
// synthesized keys with a warning) and builds one condition per resolved key.
func (c *conditionBuilder) conditionsForKeys(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
matches []*telemetrytypes.TelemetryFieldKey,
skipResourceFilter bool,
operator qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
// has/hasAny/hasAll/hasToken are logs-body-only; reject for traces.
if err := querybuilder.NewFunctionUnsupportedError(operator); err != nil {
return nil, nil, err
}
keys, warning := querybuilder.ResolveKeys(key, fieldKeysForName)
keys, warning := querybuilder.ResolveKeys(key, matches)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)
}
synthesized := false
if len(keys) == 0 {
return nil, warnings, querybuilder.NewKeyNotFoundError(key.Name)
// Synthesize key(s) to query anyway; a mapper without synthesis returns nil and
// the unknown key stays a hard "not found" error.
keys = c.fm.CandidateKeys(ctx, orgID, key, value, nil)
if len(keys) == 0 {
return nil, warnings, querybuilder.NewKeyNotFoundError(key.Name)
}
synthesized = true
warnings = append(warnings, querybuilder.NewKeyNotFoundWarning(key.Name))
}
// When a resource sub-query already covers the term, drop resource keys from the main
// query. Synthesized keys are exempt: the sub-query skips keys absent from metadata.
if skipResourceFilter && !synthesized {
filtered := make([]*telemetrytypes.TelemetryFieldKey, 0, len(keys))
for _, k := range keys {
if k.FieldContext != telemetrytypes.FieldContextResource {
filtered = append(filtered, k)
}
}
if len(filtered) == 0 {
return nil, warnings, nil
}
keys = filtered
}
conds := make([]string, 0, len(keys))

View File

@@ -7,6 +7,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -293,7 +294,7 @@ func TestConditionFor(t *testing.T) {
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
cond, _, err := conditionBuilder.ConditionFor(ctx, 1761437108000000000, 1761458708000000000, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
cond, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 1761437108000000000, 1761458708000000000, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, tc.value, sb)
sb.Where(cond...)
if tc.expectedError != nil {
@@ -380,7 +381,7 @@ func TestConditionForResourceWithEvolution(t *testing.T) {
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
cond, _, err := conditionBuilder.ConditionFor(ctx, tc.tsStart, tc.tsEnd, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, nil, sb)
cond, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, tc.tsStart, tc.tsEnd, &tc.key, []*telemetrytypes.TelemetryFieldKey{&tc.key}, tc.operator, nil, sb)
require.NoError(t, err)
sb.Where(cond...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -388,3 +389,130 @@ func TestConditionForResourceWithEvolution(t *testing.T) {
})
}
}
// TestConditionForSynthesizedKeys covers the KeyNotFound fallback: when a
// referenced attribute key has no metadata match, the builder synthesizes key(s) from
// user input and queries anyway, emitting a warning instead of failing.
func TestConditionForSynthesizedKeys(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
// no metadata matches -> the builder must synthesize from user input
var noMatches []*telemetrytypes.TelemetryFieldKey
t.Run("bare key with string operand -> attribute string", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "error.type"}
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorEqual, "timeout", sb)
assert.NoError(t, err)
assert.NotEmpty(t, warnings, "a not-found warning should be emitted")
sb.Where(conds...)
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "attributes_string['error.type']")
assert.Contains(t, args, "timeout")
})
t.Run("bare key with number operand -> attribute number", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "http.status"}
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorGreaterThan, float64(5), sb)
assert.NoError(t, err)
assert.NotEmpty(t, warnings)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "attributes_number['http.status']")
})
t.Run("bare key with bool operand -> attribute bool", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "sampled"}
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorEqual, true, sb)
assert.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "attributes_bool['sampled']")
})
t.Run("exists with no operand fans out across type variants", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "exception.type"}
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorExists, nil, sb)
assert.NoError(t, err)
assert.NotEmpty(t, warnings)
assert.Len(t, conds, 3, "exists should fan out to string/number/bool")
sb.Where(sb.Or(conds...))
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "mapContains(attributes_string, 'exception.type')")
assert.Contains(t, sql, "mapContains(attributes_number, 'exception.type')")
assert.Contains(t, sql, "mapContains(attributes_bool, 'exception.type')")
})
t.Run("qualified data type honored without fanout", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "custom.key", FieldDataType: telemetrytypes.FieldDataTypeString}
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorEqual, "v", sb)
assert.NoError(t, err)
assert.Len(t, conds, 1)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "attributes_string['custom.key']")
})
t.Run("qualified resource context honored", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "k8s.cluster.name", FieldContext: telemetrytypes.FieldContextResource}
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorEqual, "prod", sb)
assert.NoError(t, err)
assert.Len(t, conds, 1)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "resources_string")
})
t.Run("synthesized resource key survives skip-resource-filter", func(t *testing.T) {
// the resource sub-query never covered a key absent from metadata
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "deployment.environment", FieldContext: telemetrytypes.FieldContextResource}
conds, warnings, err := cb.ConditionForKeys(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{SkipResourceFilter: true}, qbtypes.FilterOperatorEqual, "prod", sb)
assert.NoError(t, err)
assert.NotEmpty(t, warnings)
assert.Len(t, conds, 1, "the synthesized resource condition must not be dropped")
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "resources_string['deployment.environment']")
})
t.Run("metadata-backed resource key still dropped under skip-resource-filter", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource}
keysMap := map[string][]*telemetrytypes.TelemetryFieldKey{
"service.name": {{Name: "service.name", FieldContext: telemetrytypes.FieldContextResource, FieldDataType: telemetrytypes.FieldDataTypeString}},
}
conds, _, err := cb.ConditionForKeys(ctx, valuer.UUID{}, 0, 0, &key, keysMap, qbtypes.ConditionBuilderOptions{SkipResourceFilter: true}, qbtypes.FilterOperatorEqual, "redis", sb)
assert.NoError(t, err)
assert.Empty(t, conds, "the resource CTE covers metadata-backed keys")
})
t.Run("synthesized resource key coerces numeric operand", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "replica.count", FieldContext: telemetrytypes.FieldContextResource}
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorEqual, float64(3), sb)
assert.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "toFloat64OrNull(")
assert.Contains(t, sql, "resources_string['replica.count']")
})
t.Run("negative operator builds without exists guard (matches-everything semantics)", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "error.type"}
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.FilterOperatorNotEqual, "fatal", sb)
assert.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "attributes_string['error.type'] <> ?")
assert.NotContains(t, sql, "mapContains")
})
}

View File

@@ -7,9 +7,10 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/huandu/go-sqlbuilder"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/maps"
)
@@ -158,8 +159,7 @@ var (
}
)
type defaultFieldMapper struct {
}
type defaultFieldMapper struct{}
var _ qbtypes.FieldMapper = (*defaultFieldMapper)(nil)
@@ -250,80 +250,11 @@ func (m *defaultFieldMapper) FieldFor(
return key.Name, nil
}
columns, err := m.getColumn(ctx, startNs, endNs, key)
exprs, existExpr, columns, err := m.resolveColumnExprs(ctx, startNs, endNs, key)
if err != nil {
return "", err
}
var newColumns []*schema.Column
var evolutionsEntries []*telemetrytypes.EvolutionEntry
if len(key.Evolutions) > 0 {
// we will use the corresponding column and its evolution entry for the query
newColumns, evolutionsEntries, err = qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
if err != nil {
return "", err
}
} else {
newColumns = columns
}
exprs := []string{}
existExpr := []string{}
for i, column := range newColumns {
// Use evolution column name if available, otherwise use the column name
columnName := column.Name
if evolutionsEntries != nil && evolutionsEntries[i] != nil {
columnName = evolutionsEntries[i].ColumnName
}
switch column.Type.GetType() {
case schema.ColumnTypeEnumJSON:
// json is only supported for resource context as of now
if key.FieldContext != telemetrytypes.FieldContextResource {
return "", errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource context fields are supported for json columns, got %s", key.FieldContext.String)
}
// have to add ::string as clickHouse throws an error :- data types Variant/Dynamic are not allowed in GROUP BY
// once clickHouse dependency is updated, we need to check if we can remove it.
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
existExpr = append(existExpr, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
case schema.ColumnTypeEnumString,
schema.ColumnTypeEnumUInt64,
schema.ColumnTypeEnumUInt32,
schema.ColumnTypeEnumInt8,
schema.ColumnTypeEnumInt16,
schema.ColumnTypeEnumBool,
schema.ColumnTypeEnumDateTime64,
schema.ColumnTypeEnumFixedString:
exprs = append(exprs, column.Name)
case schema.ColumnTypeEnumLowCardinality:
switch elementType := column.Type.(schema.LowCardinalityColumnType).ElementType; elementType.GetType() {
case schema.ColumnTypeEnumString:
exprs = append(exprs, column.Name)
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "value type %s is not supported for low cardinality column type %s", elementType, column.Type)
}
case schema.ColumnTypeEnumMap:
keyType := column.Type.(schema.MapColumnType).KeyType
if _, ok := keyType.(schema.LowCardinalityColumnType); !ok {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "key type %s is not supported for map column type %s", keyType, column.Type)
}
switch valueType := column.Type.(schema.MapColumnType).ValueType; valueType.GetType() {
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumFloat64, schema.ColumnTypeEnumBool:
// a key could have been materialized, if so return the materialized column name
if key.Materialized {
exprs = append(exprs, telemetrytypes.FieldKeyToMaterializedColumnName(key))
existExpr = append(existExpr, fmt.Sprintf("%s==true", telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key)))
} else {
exprs = append(exprs, fmt.Sprintf("%s['%s']", columnName, key.Name))
existExpr = append(existExpr, fmt.Sprintf("mapContains(%s, '%s')", columnName, key.Name))
}
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "value type %s is not supported for map column type %s", valueType, column.Type)
}
}
}
if len(exprs) == 1 {
return exprs[0], nil
} else if len(exprs) > 1 {
@@ -342,48 +273,177 @@ func (m *defaultFieldMapper) FieldFor(
return columns[0].Name, nil
}
// ColumnExpressionFor returns the column expression for the given field
// if it exists otherwise it returns qbtypes.ErrColumnNotFound.
// resolveColumnExprs resolves key to its per-column value expressions and existence guards
// (after evolution selection); existExprs only carries guards for guardable column types.
func (m *defaultFieldMapper) resolveColumnExprs(
ctx context.Context,
startNs, endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
) (exprs []string, existExprs []string, columns []*schema.Column, err error) {
columns, err = m.getColumn(ctx, startNs, endNs, key)
if err != nil {
return nil, nil, nil, err
}
var newColumns []*schema.Column
var evolutionsEntries []*telemetrytypes.EvolutionEntry
if len(key.Evolutions) > 0 {
// we will use the corresponding column and its evolution entry for the query
newColumns, evolutionsEntries, err = qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
if err != nil {
return nil, nil, nil, err
}
} else {
newColumns = columns
}
for i, column := range newColumns {
// Use evolution column name if available, otherwise use the column name
columnName := column.Name
if evolutionsEntries != nil && evolutionsEntries[i] != nil {
columnName = evolutionsEntries[i].ColumnName
}
switch column.Type.GetType() {
case schema.ColumnTypeEnumJSON:
// json is only supported for resource context as of now
if key.FieldContext != telemetrytypes.FieldContextResource {
return nil, nil, nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource context fields are supported for json columns, got %s", key.FieldContext.String)
}
// have to add ::string as clickHouse throws an error :- data types Variant/Dynamic are not allowed in GROUP BY
// once clickHouse dependency is updated, we need to check if we can remove it.
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
case schema.ColumnTypeEnumString,
schema.ColumnTypeEnumUInt64,
schema.ColumnTypeEnumUInt32,
schema.ColumnTypeEnumInt8,
schema.ColumnTypeEnumInt16,
schema.ColumnTypeEnumBool,
schema.ColumnTypeEnumDateTime64,
schema.ColumnTypeEnumFixedString:
exprs = append(exprs, column.Name)
case schema.ColumnTypeEnumLowCardinality:
switch elementType := column.Type.(schema.LowCardinalityColumnType).ElementType; elementType.GetType() {
case schema.ColumnTypeEnumString:
exprs = append(exprs, column.Name)
default:
return nil, nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "value type %s is not supported for low cardinality column type %s", elementType, column.Type)
}
case schema.ColumnTypeEnumMap:
keyType := column.Type.(schema.MapColumnType).KeyType
if _, ok := keyType.(schema.LowCardinalityColumnType); !ok {
return nil, nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "key type %s is not supported for map column type %s", keyType, column.Type)
}
switch valueType := column.Type.(schema.MapColumnType).ValueType; valueType.GetType() {
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumFloat64, schema.ColumnTypeEnumBool:
// a key could have been materialized, if so return the materialized column name
if key.Materialized {
exprs = append(exprs, telemetrytypes.FieldKeyToMaterializedColumnName(key))
existExprs = append(existExprs, fmt.Sprintf("%s==true", telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key)))
} else {
exprs = append(exprs, fmt.Sprintf("%s['%s']", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("mapContains(%s, '%s')", columnName, key.Name))
}
default:
return nil, nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "value type %s is not supported for map column type %s", valueType, column.Type)
}
}
}
return exprs, existExprs, columns, nil
}
// ColumnExpressionFor returns the bare (unaliased) SQL expression for the field, resolving
// unknown keys via CandidateKeys and wrapping guardable columns with exists-guard multiIfs
// so an absent key yields NULL.
func (m *defaultFieldMapper) ColumnExpressionFor(
ctx context.Context,
orgID valuer.UUID,
startNs, endNs uint64,
field *telemetrytypes.TelemetryFieldKey,
keys map[string][]*telemetrytypes.TelemetryFieldKey,
) (string, error) {
fieldExpression, err := m.FieldFor(ctx, startNs, endNs, field)
if errors.Is(err, qbtypes.ErrColumnNotFound) {
// the key didn't have the right context to be added to the query
// we try to use the context we know of
keysForField := keys[field.Name]
if len(keysForField) == 0 {
// is it a static field?
if _, ok := indexV3Columns[field.Name]; ok {
// if it is, attach the column name directly
field.FieldContext = telemetrytypes.FieldContextSpan
fieldExpression, _ = m.FieldFor(ctx, startNs, endNs, field)
} else {
// - the context is not provided
// - there are not keys for the field
// - it is not a static field
// - the next best thing to do is see if there is a typo
// and suggest a correction
wrappedErr := errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys))...)
return "", wrappedErr
}
} else if len(keysForField) == 1 {
// we have a single key for the field, use it
fieldExpression, _ = m.FieldFor(ctx, startNs, endNs, keysForField[0])
// Resolve the candidate column(s).
var candidates []*telemetrytypes.TelemetryFieldKey
switch _, err := m.FieldFor(ctx, startNs, endNs, field); {
case err == nil:
candidates = []*telemetrytypes.TelemetryFieldKey{field}
case errors.Is(err, qbtypes.ErrColumnNotFound):
// is it a static field? attach the column directly.
if _, ok := indexV3Columns[field.Name]; ok {
field.FieldContext = telemetrytypes.FieldContextSpan
candidates = []*telemetrytypes.TelemetryFieldKey{field}
} else {
// select any non-empty value from the keys
args := []string{}
for _, key := range keysForField {
fieldExpression, _ = m.FieldFor(ctx, startNs, endNs, key)
args = append(args, fmt.Sprintf("toString(%s) != '', toString(%s)", fieldExpression, fieldExpression))
// metadata matches (incl. the collision set) or synthesized type-variant keys.
candidates = m.CandidateKeys(ctx, orgID, field, nil, keys)
if len(candidates) == 0 {
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys))...)
}
fieldExpression = fmt.Sprintf("multiIf(%s, NULL)", strings.Join(args, ", "))
}
default:
return "", err
}
return fmt.Sprintf("%s AS `%s`", sqlbuilder.Escape(fieldExpression), field.Name), nil
// Single candidate: exists-guard wrap so an absent key yields NULL; multi-column and
// physical columns stay bare.
if len(candidates) == 1 {
value, err := m.FieldFor(ctx, startNs, endNs, candidates[0])
if err != nil {
return "", err
}
exprs, existExprs, _, _ := m.resolveColumnExprs(ctx, startNs, endNs, candidates[0])
if len(exprs) == 1 && len(existExprs) == 1 {
return fmt.Sprintf("multiIf(%s, %s, NULL)", existExprs[0], value), nil
}
return value, nil
}
// Multiple candidates (collision / synth): multiIf picks the first that exists,
// stringified so branches share a type.
args := make([]string, 0, len(candidates))
for _, key := range candidates {
value, err := m.FieldFor(ctx, startNs, endNs, key)
if err != nil {
return "", err
}
args = append(args, fmt.Sprintf("%s, toString(%s)", m.existsGuardFor(ctx, startNs, endNs, key, value), value))
}
return fmt.Sprintf("multiIf(%s, NULL)", strings.Join(args, ", ")), nil
}
// membershipGuard returns the key's existence guard (mapContains / IS NOT NULL / _exists),
// OR-combined across its columns, or "" when the column type has none (physical column).
func (m *defaultFieldMapper) membershipGuard(ctx context.Context, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) string {
_, existExprs, _, err := m.resolveColumnExprs(ctx, startNs, endNs, key)
if err != nil || len(existExprs) == 0 {
return ""
}
if len(existExprs) == 1 {
return existExprs[0]
}
return "(" + strings.Join(existExprs, " OR ") + ")"
}
// existsGuardFor is membershipGuard with a non-empty fallback, for use as a multiIf branch
// condition where a bare guard-less column still needs a discriminator.
func (m *defaultFieldMapper) existsGuardFor(ctx context.Context, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey, value string) string {
if g := m.membershipGuard(ctx, startNs, endNs, key); g != "" {
return g
}
return fmt.Sprintf("toString(%s) != ''", value)
}
// CandidateKeys prefers metadata matches for the name (or `{context}.{name}`); when none
// match it synthesizes type-variant keys across the span attribute maps.
func (m *defaultFieldMapper) CandidateKeys(_ context.Context, _ valuer.UUID, field *telemetrytypes.TelemetryFieldKey, value any, keys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
if matches := keys[field.Name]; len(matches) > 0 {
return matches
}
if matches := keys[fmt.Sprintf("%s.%s", field.FieldContext.StringValue(), field.Name)]; len(matches) > 0 {
return matches
}
return querybuilder.SynthesizeKeys(field, value)
}

View File

@@ -14,6 +14,7 @@ import (
"github.com/SigNoz/signoz/pkg/telemetrystore"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -73,6 +74,7 @@ func NewTraceQueryStatementBuilder(
// Build builds a SQL query for traces based on the given parameters.
func (b *traceQueryStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
@@ -109,13 +111,13 @@ func (b *traceQueryStatementBuilder) Build(
switch requestType {
case qbtypes.RequestTypeRaw:
return b.buildListQuery(ctx, q, query, start, end, keys, variables, isSelectFieldsEmpty)
return b.buildListQuery(ctx, orgID, q, query, start, end, keys, variables, isSelectFieldsEmpty)
case qbtypes.RequestTypeTimeSeries:
return b.buildTimeSeriesQuery(ctx, q, query, start, end, keys, variables)
return b.buildTimeSeriesQuery(ctx, orgID, q, query, start, end, keys, variables)
case qbtypes.RequestTypeScalar:
return b.buildScalarQuery(ctx, q, query, start, end, keys, variables, false, false)
return b.buildScalarQuery(ctx, orgID, q, query, start, end, keys, variables, false, false)
case qbtypes.RequestTypeTrace:
return b.buildTraceQuery(ctx, q, query, start, end, keys, variables)
return b.buildTraceQuery(ctx, orgID, q, query, start, end, keys, variables)
}
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported request type: %s", requestType)
@@ -270,6 +272,7 @@ func adjustTraceKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*te
// buildListQuery builds a query for list panel type.
func (b *traceQueryStatementBuilder) buildListQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
start, end uint64,
@@ -283,7 +286,7 @@ func (b *traceQueryStatementBuilder) buildListQuery(
cteArgs [][]any
)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables)
if err != nil {
return nil, err
}
@@ -292,12 +295,12 @@ func (b *traceQueryStatementBuilder) buildListQuery(
cteArgs = append(cteArgs, args)
}
for _, field := range query.SelectFields {
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &field, keys)
for i, field := range query.SelectFields {
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &field, keys)
if err != nil {
return nil, err
}
sb.SelectMore(colExpr)
sb.SelectMore(fmt.Sprintf("%s AS `%s`", sqlbuilder.Escape(expr), selectColumnAlias(i, field.Name)))
}
if isSelectFieldsEmpty {
@@ -310,18 +313,18 @@ func (b *traceQueryStatementBuilder) buildListQuery(
sb.From(fmt.Sprintf("%s.%s", DBName, SpanIndexV3TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables, skipResourceFilter)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
if err != nil {
return nil, err
}
// Add order by
for _, orderBy := range query.Order {
colExpr, err := b.fm.ColumnExpressionFor(ctx, start, end, &orderBy.Key.TelemetryFieldKey, keys)
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &orderBy.Key.TelemetryFieldKey, keys)
if err != nil {
return nil, err
}
sb.OrderBy(fmt.Sprintf("%s %s", colExpr, orderBy.Direction.StringValue()))
sb.OrderBy(fmt.Sprintf("%s %s", sqlbuilder.Escape(expr), orderBy.Direction.StringValue()))
}
// Add limit and offset
@@ -352,6 +355,7 @@ func (b *traceQueryStatementBuilder) buildListQuery(
func (b *traceQueryStatementBuilder) buildTraceQuery(
ctx context.Context,
orgID valuer.UUID,
_ *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
start, end uint64,
@@ -371,7 +375,7 @@ func (b *traceQueryStatementBuilder) buildTraceQuery(
cteArgs [][]any
)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, distSB, query, start, end, variables)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, orgID, distSB, query, start, end, variables)
if err != nil {
return nil, err
}
@@ -381,7 +385,7 @@ func (b *traceQueryStatementBuilder) buildTraceQuery(
}
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, distSB, start, end, query, keys, variables, skipResourceFilter)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, distSB, start, end, query, keys, variables, skipResourceFilter)
if err != nil {
return nil, err
}
@@ -470,6 +474,7 @@ func (b *traceQueryStatementBuilder) buildTraceQuery(
func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
start, end uint64,
@@ -482,7 +487,7 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
cteArgs [][]any
)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables)
if err != nil {
return nil, err
}
@@ -496,26 +501,23 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
int64(query.StepInterval.Seconds()),
))
var allGroupByArgs []any
// Keep original column expressions so we can build the tuple
fieldNames := make([]string, 0, len(query.GroupBy))
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, nil, false)
for i, gb := range query.GroupBy {
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &gb.TelemetryFieldKey, keys)
if err != nil {
return nil, err
}
colExpr := fmt.Sprintf("toString(%s) AS `%s`", expr, gb.Name)
allGroupByArgs = append(allGroupByArgs, args...)
sb.SelectMore(colExpr)
fieldNames = append(fieldNames, fmt.Sprintf("`%s`", gb.Name))
fieldAlias := groupByColumnAlias(i, gb.Name)
sb.SelectMore(fmt.Sprintf("toString(%s) AS `%s`", sqlbuilder.Escape(expr), fieldAlias))
fieldNames = append(fieldNames, fmt.Sprintf("`%s`", fieldAlias))
}
// Aggregations
allAggChArgs := make([]any, 0)
for i, agg := range query.Aggregations {
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(
ctx, start, end, agg.Expression,
ctx, orgID, start, end, agg.Expression,
uint64(query.StepInterval.Seconds()),
keys,
)
@@ -527,7 +529,7 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
}
sb.From(fmt.Sprintf("%s.%s", DBName, SpanIndexV3TableName))
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables, skipResourceFilter)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
if err != nil {
return nil, err
}
@@ -538,7 +540,7 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
if query.Limit > 0 && len(query.GroupBy) > 0 {
// build the scalar “top/bottom-N” query in its own builder.
cteSB := sqlbuilder.NewSelectBuilder()
cteStmt, err := b.buildScalarQuery(ctx, cteSB, query, start, end, keys, variables, true, true)
cteStmt, err := b.buildScalarQuery(ctx, orgID, cteSB, query, start, end, keys, variables, true, true)
if err != nil {
return nil, err
}
@@ -552,7 +554,7 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
// Group by all dimensions
sb.GroupBy("ts")
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
sb.GroupBy(fieldNames...)
if query.Having != nil && query.Having.Expression != "" {
rewriter := querybuilder.NewHavingExpressionRewriter()
rewrittenExpr, err := rewriter.RewriteForTraces(query.Having.Expression, query.Aggregations)
@@ -566,13 +568,17 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
for _, orderBy := range query.Order {
_, ok := aggOrderBy(orderBy, query)
if !ok {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderBy.Key.Name, orderBy.Direction.StringValue()))
orderCol := orderBy.Key.Name
if alias, ok := groupByOrderAlias(orderBy.Key.Name, query.GroupBy); ok {
orderCol = alias
}
sb.OrderBy(fmt.Sprintf("`%s` %s", orderCol, orderBy.Direction.StringValue()))
}
}
sb.OrderBy("ts desc")
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
combinedArgs := allAggChArgs
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse, combinedArgs...)
// Stitch it all together: WITH … SELECT …
@@ -581,7 +587,7 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
} else {
sb.GroupBy("ts")
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
sb.GroupBy(fieldNames...)
if query.Having != nil && query.Having.Expression != "" {
rewriter := querybuilder.NewHavingExpressionRewriter()
rewrittenExpr, err := rewriter.RewriteForTraces(query.Having.Expression, query.Aggregations)
@@ -595,13 +601,17 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
for _, orderBy := range query.Order {
_, ok := aggOrderBy(orderBy, query)
if !ok {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderBy.Key.Name, orderBy.Direction.StringValue()))
orderCol := orderBy.Key.Name
if alias, ok := groupByOrderAlias(orderBy.Key.Name, query.GroupBy); ok {
orderCol = alias
}
sb.OrderBy(fmt.Sprintf("`%s` %s", orderCol, orderBy.Direction.StringValue()))
}
}
sb.OrderBy("ts desc")
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
combinedArgs := allAggChArgs
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse, combinedArgs...)
// Stitch it all together: WITH … SELECT …
@@ -622,6 +632,7 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
// buildScalarQuery builds a query for scalar panel type.
func (b *traceQueryStatementBuilder) buildScalarQuery(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
start, end uint64,
@@ -636,7 +647,7 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
cteArgs [][]any
)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables)
frag, args, skipResourceFilter, err := b.maybeAttachResourceFilter(ctx, orgID, sb, query, start, end, variables)
if err != nil {
return nil, err
}
@@ -647,15 +658,15 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
allAggChArgs := []any{}
var allGroupByArgs []any
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, nil, false)
fieldNames := make([]string, 0, len(query.GroupBy))
for i, gb := range query.GroupBy {
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &gb.TelemetryFieldKey, keys)
if err != nil {
return nil, err
}
colExpr := fmt.Sprintf("toString(%s) AS `%s`", expr, gb.Name)
allGroupByArgs = append(allGroupByArgs, args...)
sb.SelectMore(colExpr)
fieldAlias := groupByColumnAlias(i, gb.Name)
sb.SelectMore(fmt.Sprintf("toString(%s) AS `%s`", sqlbuilder.Escape(expr), fieldAlias))
fieldNames = append(fieldNames, fmt.Sprintf("`%s`", fieldAlias))
}
// for scalar queries, the rate would be end-start
@@ -666,7 +677,7 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
for idx := range query.Aggregations {
aggExpr := query.Aggregations[idx]
rewritten, chArgs, err := b.aggExprRewriter.Rewrite(
ctx, start, end, aggExpr.Expression,
ctx, orgID, start, end, aggExpr.Expression,
rateInterval,
keys,
)
@@ -682,13 +693,13 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
sb.From(fmt.Sprintf("%s.%s", DBName, SpanIndexV3TableName))
// Add filter conditions
preparedWhereClause, err := b.addFilterCondition(ctx, sb, start, end, query, keys, variables, skipResourceFilter)
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
if err != nil {
return nil, err
}
// Group by dimensions
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
sb.GroupBy(fieldNames...)
// Add having clause if needed
if query.Having != nil && query.Having.Expression != "" && !skipHaving {
@@ -706,7 +717,11 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
if ok {
sb.OrderBy(fmt.Sprintf("__result_%d %s", idx, orderBy.Direction.StringValue()))
} else {
sb.OrderBy(fmt.Sprintf("`%s` %s", orderBy.Key.Name, orderBy.Direction.StringValue()))
orderCol := orderBy.Key.Name
if alias, ok := groupByOrderAlias(orderBy.Key.Name, query.GroupBy); ok {
orderCol = alias
}
sb.OrderBy(fmt.Sprintf("`%s` %s", orderCol, orderBy.Direction.StringValue()))
}
}
@@ -720,7 +735,7 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
sb.Limit(query.Limit)
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
combinedArgs := allAggChArgs
mainSQL, mainArgs := sb.BuildWithFlavor(sqlbuilder.ClickHouse, combinedArgs...)
@@ -740,6 +755,7 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
// buildFilterCondition builds SQL condition from filter expression.
func (b *traceQueryStatementBuilder) addFilterCondition(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
@@ -755,6 +771,7 @@ func (b *traceQueryStatementBuilder) addFilterCondition(
// add filter expression
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
@@ -794,8 +811,32 @@ func aggOrderBy(k qbtypes.OrderBy, q qbtypes.QueryBuilderQuery[qbtypes.TraceAggr
return 0, false
}
// groupByColumnAlias returns the positional SQL alias for the i-th group-by dimension;
// the querier (consume.go stripKeyAlias) strips the prefix to recover the field name.
func groupByColumnAlias(i int, name string) string {
return fmt.Sprintf("__GROUP_BY_KEY_%d_%s", i, name)
}
// selectColumnAlias returns the positional SQL alias for the i-th raw select field. Like
// groupByColumnAlias, the prefix is stripped by the querier.
func selectColumnAlias(i int, name string) string {
return fmt.Sprintf("__SELECT_KEY_%d_%s", i, name)
}
// groupByOrderAlias returns the group-by column alias to order by when orderKey names a
// group-by dimension (matching how the SELECT aliases it), else ("", false).
func groupByOrderAlias(orderKey string, groupBy []qbtypes.GroupByKey) (string, bool) {
for i := range groupBy {
if groupBy[i].Name == orderKey {
return groupByColumnAlias(i, groupBy[i].Name), true
}
}
return "", false
}
func (b *traceQueryStatementBuilder) maybeAttachResourceFilter(
ctx context.Context,
orgID valuer.UUID,
sb *sqlbuilder.SelectBuilder,
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
start, end uint64,
@@ -803,7 +844,7 @@ func (b *traceQueryStatementBuilder) maybeAttachResourceFilter(
) (cteSQL string, cteArgs []any, skipResourceFilter bool, err error) {
if b.skipResourceFingerprintEnabled {
decision, err := b.resourceFilterResolver.Resolve(ctx, query, start, end, variables)
decision, err := b.resourceFilterResolver.Resolve(ctx, orgID, query, start, end, variables)
if err != nil {
return "", nil, true, err
}
@@ -816,7 +857,7 @@ func (b *traceQueryStatementBuilder) maybeAttachResourceFilter(
}
stmt, err := b.resourceFilterResolver.StatementBuilder().Build(
ctx, start, end, qbtypes.RequestTypeRaw, query, variables,
ctx, orgID, start, end, qbtypes.RequestTypeRaw, query, variables,
)
if err != nil {
return "", nil, true, err

View File

@@ -16,6 +16,8 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -67,7 +69,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name`",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -96,7 +98,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL) OR (attributes_string['http.request.method'] = ? AND mapContains(attributes_string, 'http.request.method') = ?)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL) OR (attributes_string['http.request.method'] = ? AND mapContains(attributes_string, 'http.request.method') = ?)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name`",
Query: "WITH __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL) OR (attributes_string['http.request.method'] = ? AND mapContains(attributes_string, 'http.request.method') = ?)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL) OR (attributes_string['http.request.method'] = ? AND mapContains(attributes_string, 'http.request.method') = ?)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"redis-manual", "GET", true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "redis-manual", "GET", true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -125,7 +127,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = ?) OR (multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = ?) OR (multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name`",
Query: "WITH __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = ?) OR (multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = ?) OR (multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ? AND multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"redis-manual", true, "redis-manual", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "redis-manual", true, "redis-manual", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -156,8 +158,8 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(attribute_string_http$$route <> ?, attribute_string_http$$route, NULL)) AS `httpRoute`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `httpRoute` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(attribute_string_http$$route <> ?, attribute_string_http$$route, NULL)) AS `httpRoute`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`httpRoute`) GLOBAL IN (SELECT `httpRoute` FROM __limit_cte) GROUP BY ts, `httpRoute`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(attribute_string_http$$route) AS `__GROUP_BY_KEY_0_httpRoute`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_httpRoute` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(attribute_string_http$$route) AS `__GROUP_BY_KEY_0_httpRoute`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_httpRoute`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_httpRoute` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_httpRoute`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
},
@@ -194,8 +196,8 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(attribute_string_http$$route <> ?, attribute_string_http$$route, NULL)) AS `httpRoute`, toString(multiIf(http_method <> ?, http_method, NULL)) AS `httpMethod`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((resource_string_service$$name = ? AND resource_string_service$$name <> ?) AND http_method <> ? AND kind_string = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `httpRoute`, `httpMethod` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(attribute_string_http$$route <> ?, attribute_string_http$$route, NULL)) AS `httpRoute`, toString(multiIf(http_method <> ?, http_method, NULL)) AS `httpMethod`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((resource_string_service$$name = ? AND resource_string_service$$name <> ?) AND http_method <> ? AND kind_string = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`httpRoute`, `httpMethod`) GLOBAL IN (SELECT `httpRoute`, `httpMethod` FROM __limit_cte) GROUP BY ts, `httpRoute`, `httpMethod`",
Args: []any{"", "", "redis-manual", "", "", "Server", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "", "", "redis-manual", "", "", "Server", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
Query: "WITH __limit_cte AS (SELECT toString(attribute_string_http$$route) AS `__GROUP_BY_KEY_0_httpRoute`, toString(http_method) AS `__GROUP_BY_KEY_1_httpMethod`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((resource_string_service$$name = ? AND resource_string_service$$name <> ?) AND http_method <> ? AND kind_string = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_httpRoute`, `__GROUP_BY_KEY_1_httpMethod` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(attribute_string_http$$route) AS `__GROUP_BY_KEY_0_httpRoute`, toString(http_method) AS `__GROUP_BY_KEY_1_httpMethod`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((resource_string_service$$name = ? AND resource_string_service$$name <> ?) AND http_method <> ? AND kind_string = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_httpRoute`, `__GROUP_BY_KEY_1_httpMethod`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_httpRoute`, `__GROUP_BY_KEY_1_httpMethod` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_httpRoute`, `__GROUP_BY_KEY_1_httpMethod`",
Args: []any{"redis-manual", "", "", "Server", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "redis-manual", "", "", "Server", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
},
@@ -235,7 +237,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, sum(multiIf(mapContains(attributes_number, 'metric.max_count') = ?, toFloat64(attributes_number['metric.max_count']), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 desc LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, sum(multiIf(mapContains(attributes_number, 'metric.max_count') = ?, toFloat64(attributes_number['metric.max_count']), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name` ORDER BY ts desc",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(mapContains(attributes_number, 'metric.max_count') = ?, toFloat64(attributes_number['metric.max_count']), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 desc LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(mapContains(attributes_number, 'metric.max_count') = ?, toFloat64(attributes_number['metric.max_count']), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name` ORDER BY ts desc",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -264,7 +266,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name`",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -303,7 +305,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY `service.name` desc LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name` ORDER BY `service.name` desc, ts desc",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY `__GROUP_BY_KEY_0_service.name` desc LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = ?, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name` ORDER BY `__GROUP_BY_KEY_0_service.name` desc, ts desc",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, true, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -334,8 +336,8 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(response_status_code <> ?, response_status_code, NULL)) AS `responseStatusCode`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `responseStatusCode` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(response_status_code <> ?, response_status_code, NULL)) AS `responseStatusCode`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`responseStatusCode`) GLOBAL IN (SELECT `responseStatusCode` FROM __limit_cte) GROUP BY ts, `responseStatusCode`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(response_status_code) AS `__GROUP_BY_KEY_0_responseStatusCode`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_responseStatusCode` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(response_status_code) AS `__GROUP_BY_KEY_0_responseStatusCode`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_responseStatusCode`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_responseStatusCode` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_responseStatusCode`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
},
@@ -365,18 +367,18 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(response_status_code <> ?, response_status_code, NULL)) AS `responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> ?, duration_nano, NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `responseStatusCode` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(response_status_code <> ?, response_status_code, NULL)) AS `responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> ?, duration_nano, NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`responseStatusCode`) GLOBAL IN (SELECT `responseStatusCode` FROM __limit_cte) GROUP BY ts, `responseStatusCode`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "", 0, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "", 0, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(response_status_code) AS `__GROUP_BY_KEY_0_responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> ?, duration_nano, NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_responseStatusCode` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(response_status_code) AS `__GROUP_BY_KEY_0_responseStatusCode`, quantile(0.90)(multiIf(duration_nano <> ?, duration_nano, NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_responseStatusCode`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_responseStatusCode` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_responseStatusCode`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), 0, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, 0, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
},
}
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
@@ -400,7 +402,7 @@ func TestStatementBuilder(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, vars)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, vars)
if c.expectedErr != nil {
require.Error(t, err)
@@ -462,7 +464,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `timestamp`, trace_id AS `trace_id`, span_id AS `span_id`, name AS `name`, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) AS `service.name`, duration_nano AS `duration_nano`, `attribute_number_cart$$items_count` AS `cart.items_count` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) AS `__SELECT_KEY_4_service.name`, duration_nano AS `__SELECT_KEY_5_duration_nano`, multiIf(`attribute_number_cart$$items_count_exists`==true, `attribute_number_cart$$items_count`, NULL) AS `__SELECT_KEY_6_cart.items_count` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -491,7 +493,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `timestamp`, trace_id AS `trace_id`, span_id AS `span_id`, trace_state AS `trace_state`, parent_span_id AS `parent_span_id`, flags AS `flags`, name AS `name`, kind AS `kind`, kind_string AS `kind_string`, duration_nano AS `duration_nano`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY attributes_string['user.id'] AS `user.id` desc LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, trace_state AS `__SELECT_KEY_3_trace_state`, parent_span_id AS `__SELECT_KEY_4_parent_span_id`, flags AS `__SELECT_KEY_5_flags`, name AS `__SELECT_KEY_6_name`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, duration_nano AS `__SELECT_KEY_9_duration_nano`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY multiIf(mapContains(attributes_string, 'user.id'), attributes_string['user.id'], NULL) desc LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -535,7 +537,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `timestamp`, trace_id AS `trace_id`, span_id AS `span_id`, name AS `name`, resource_string_service$$name AS `serviceName`, duration_nano AS `durationNano`, http_method AS `httpMethod`, response_status_code AS `responseStatusCode` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, response_status_code AS `__SELECT_KEY_7_responseStatusCode` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -579,7 +581,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `timestamp`, trace_id AS `trace_id`, span_id AS `span_id`, name AS `name`, resource_string_service$$name AS `serviceName`, duration_nano AS `durationNano`, http_method AS `httpMethod`, multiIf(toString(`attribute_string_mixed$$materialization$$key`) != '', toString(`attribute_string_mixed$$materialization$$key`), toString(multiIf(resource.`mixed.materialization.key` IS NOT NULL, resource.`mixed.materialization.key`::String, mapContains(resources_string, 'mixed.materialization.key'), resources_string['mixed.materialization.key'], NULL)) != '', toString(multiIf(resource.`mixed.materialization.key` IS NOT NULL, resource.`mixed.materialization.key`::String, mapContains(resources_string, 'mixed.materialization.key'), resources_string['mixed.materialization.key'], NULL)), NULL) AS `mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, multiIf(`attribute_string_mixed$$materialization$$key_exists`==true, toString(`attribute_string_mixed$$materialization$$key`), (resource.`mixed.materialization.key` IS NOT NULL OR mapContains(resources_string, 'mixed.materialization.key')), toString(multiIf(resource.`mixed.materialization.key` IS NOT NULL, resource.`mixed.materialization.key`::String, mapContains(resources_string, 'mixed.materialization.key'), resources_string['mixed.materialization.key'], NULL)), NULL) AS `__SELECT_KEY_7_mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -624,13 +626,13 @@ func TestStatementBuilderListQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `timestamp`, trace_id AS `trace_id`, span_id AS `span_id`, name AS `name`, resource_string_service$$name AS `serviceName`, duration_nano AS `durationNano`, http_method AS `httpMethod`, `attribute_string_mixed$$materialization$$key` AS `mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, multiIf(`attribute_string_mixed$$materialization$$key_exists`==true, `attribute_string_mixed$$materialization$$key`, NULL) AS `__SELECT_KEY_7_mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
},
{
name: "List query with legacy fields with field that doesn't exist",
name: "List query with a select field not in metadata synthesizes type-variant columns",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
@@ -665,16 +667,19 @@ func TestStatementBuilderListQuery(t *testing.T) {
},
Limit: 10,
},
expected: qbtypes.Statement{},
expectedErr: errors.NewInvalidInputf(errors.CodeInvalidInput, "field not found"),
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, multiIf(mapContains(attributes_string, 'non-existent.key'), toString(attributes_string['non-existent.key']), mapContains(attributes_number, 'non-existent.key'), toString(attributes_number['non-existent.key']), mapContains(attributes_bool, 'non-existent.key'), toString(attributes_bool['non-existent.key']), NULL) AS `__SELECT_KEY_7_non-existent.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
},
}
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
@@ -692,7 +697,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -746,7 +751,7 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `timestamp`, trace_id AS `trace_id`, span_id AS `span_id`, trace_state AS `trace_state`, parent_span_id AS `parent_span_id`, flags AS `flags`, name AS `name`, kind AS `kind`, kind_string AS `kind_string`, duration_nano AS `duration_nano`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, trace_state AS `__SELECT_KEY_3_trace_state`, parent_span_id AS `__SELECT_KEY_4_parent_span_id`, flags AS `__SELECT_KEY_5_flags`, name AS `__SELECT_KEY_6_name`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, duration_nano AS `__SELECT_KEY_9_duration_nano`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -788,7 +793,7 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
}},
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `timestamp`, trace_id AS `trace_id`, span_id AS `span_id`, trace_state AS `trace_state`, parent_span_id AS `parent_span_id`, flags AS `flags`, name AS `name`, kind AS `kind`, kind_string AS `kind_string`, duration_nano AS `duration_nano`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY timestamp AS `timestamp` asc LIMIT ?",
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, trace_state AS `__SELECT_KEY_3_trace_state`, parent_span_id AS `__SELECT_KEY_4_parent_span_id`, flags AS `__SELECT_KEY_5_flags`, name AS `__SELECT_KEY_6_name`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, duration_nano AS `__SELECT_KEY_9_duration_nano`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY timestamp asc LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -797,6 +802,7 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
@@ -804,7 +810,6 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
if mockMetadataStore.KeysMap == nil {
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
}
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
@@ -819,7 +824,7 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
100000,
)
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -848,7 +853,7 @@ func TestStatementBuilderGroupByResourceEvolution(t *testing.T) {
startMs: 1747947419000, // 2025-05-22 21:56:59 UTC, ~3m before release
endMs: 1747983448000, // 2025-05-23 07:57:28 UTC, ~10h after release
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name`",
Query: "WITH __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
},
@@ -857,7 +862,7 @@ func TestStatementBuilderGroupByResourceEvolution(t *testing.T) {
startMs: 1747960000000, // 2025-05-23 00:26:40 UTC, ~2.5h after release
endMs: 1747983448000, // 2025-05-23 07:57:28 UTC
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(resource.`service.name`::String IS NOT NULL, resource.`service.name`::String, NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name`::String IS NOT NULL, resource.`service.name`::String, NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name`",
Query: "WITH __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"1747960000000000000", "1747983448000000000", uint64(1747958200), uint64(1747983448), 10, "1747960000000000000", "1747983448000000000", uint64(1747958200), uint64(1747983448)},
},
},
@@ -866,17 +871,17 @@ func TestStatementBuilderGroupByResourceEvolution(t *testing.T) {
startMs: 1747900000000, // 2025-05-22 08:26:40 UTC, ~13.5h before release
endMs: 1747947000000, // 2025-05-22 21:50:00 UTC, ~10m before release
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(mapContains(resources_string, 'service.name') = ?, resources_string['service.name'], NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(mapContains(resources_string, 'service.name') = ?, resources_string['service.name'], NULL)) AS `service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`service.name`) GLOBAL IN (SELECT `service.name` FROM __limit_cte) GROUP BY ts, `service.name`",
Args: []any{true, "1747900000000000000", "1747947000000000000", uint64(1747898200), uint64(1747947000), 10, true, "1747900000000000000", "1747947000000000000", uint64(1747898200), uint64(1747947000)},
Query: "WITH __limit_cte AS (SELECT toString(multiIf(mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"1747900000000000000", "1747947000000000000", uint64(1747898200), uint64(1747947000), 10, "1747900000000000000", "1747947000000000000", uint64(1747898200), uint64(1747947000)},
},
},
}
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
@@ -910,7 +915,7 @@ func TestStatementBuilderGroupByResourceEvolution(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), c.startMs, c.endMs, qbtypes.RequestTypeTimeSeries, query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, c.startMs, c.endMs, qbtypes.RequestTypeTimeSeries, query, nil)
require.NoError(t, err)
require.Equal(t, c.expected.Query, q.Query)
require.Equal(t, c.expected.Args, q.Args)
@@ -1039,11 +1044,11 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
},
}
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
@@ -1061,7 +1066,7 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, c.requestType, c.query, nil)
if c.expectedErr != nil {
require.Error(t, err)
@@ -1617,7 +1622,7 @@ func TestSkipResourceFingerprint(t *testing.T) {
t.Run("disabled uses the legacy CTE", func(t *testing.T) {
sb := newSkipResourceFingerprintBuilder(t, nil, false, threshold)
stmt, err := sb.Build(context.Background(), startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
stmt, err := sb.Build(context.Background(), valuer.UUID{}, startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
require.NoError(t, err)
require.Contains(t, stmt.Query, "__resource_filter AS (SELECT fingerprint")
require.Contains(t, stmt.Query, "resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)")
@@ -1636,7 +1641,7 @@ func TestSkipResourceFingerprint(t *testing.T) {
sb := newSkipResourceFingerprintBuilder(t, mockStore, true, threshold)
stmt, err := sb.Build(context.Background(), startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
stmt, err := sb.Build(context.Background(), valuer.UUID{}, startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
require.NoError(t, err)
require.Contains(t, stmt.Query, "__resource_filter AS (SELECT fingerprint")
@@ -1656,7 +1661,7 @@ func TestSkipResourceFingerprint(t *testing.T) {
sb := newSkipResourceFingerprintBuilder(t, mockStore, true, threshold)
stmt, err := sb.Build(context.Background(), startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
stmt, err := sb.Build(context.Background(), valuer.UUID{}, startMs, endMs, qbtypes.RequestTypeRaw, query, nil)
require.NoError(t, err)
require.NotContains(t, stmt.Query, "__resource_filter AS")
@@ -1677,13 +1682,13 @@ func newSkipResourceFingerprintBuilder(
) *traceQueryStatementBuilder {
t.Helper()
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(
instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl,
)
@@ -1700,3 +1705,84 @@ func newSkipResourceFingerprintBuilder(
threshold,
)
}
// TestStatementBuilderGroupByUnseenKey covers the group-by/aggregation half of the
// KeyNotFound fallback: grouping by an attribute absent from metadata must
// fan out across type variants and build, not error.
func TestStatementBuilderGroupByUnseenKey(t *testing.T) {
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
fm,
cb,
aggExprRewriter,
nil,
fl,
false,
100000,
)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "count()"}},
Limit: 10,
GroupBy: []qbtypes.GroupByKey{
{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "error.type"}},
},
}
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, qbtypes.RequestTypeTimeSeries, query, nil)
require.NoError(t, err) // guard: q is nil on error
assert.Contains(t, q.Query, "attributes_string['error.type']")
assert.Contains(t, q.Query, "attributes_number['error.type']")
assert.Contains(t, q.Query, "attributes_bool['error.type']")
}
// TestStatementBuilderAggregationUnseenKey covers the aggregation half of the KeyNotFound
// fallback: aggregating over an attribute absent from metadata (via the shared
// agg-expr rewriter) must fan out across type variants and build, not error.
func TestStatementBuilderAggregationUnseenKey(t *testing.T) {
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
fm,
cb,
aggExprRewriter,
nil,
fl,
false,
100000,
)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "count_distinct(error.type)"}},
Limit: 10,
}
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, qbtypes.RequestTypeTimeSeries, query, nil)
require.NoError(t, err) // guard: q is nil on error
assert.Contains(t, q.Query, "attributes_string['error.type']")
assert.Contains(t, q.Query, "attributes_number['error.type']")
assert.Contains(t, q.Query, "attributes_bool['error.type']")
}

View File

@@ -12,6 +12,7 @@ import (
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type cteNode struct {
@@ -24,6 +25,7 @@ type cteNode struct {
type traceOperatorCTEBuilder struct {
start uint64
end uint64
orgID valuer.UUID
operator *qbtypes.QueryBuilderTraceOperator
stmtBuilder *traceOperatorStatementBuilder
queries map[string]*qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]
@@ -158,6 +160,7 @@ func (b *traceOperatorCTEBuilder) buildTimeConstantsCTE() string {
func (b *traceOperatorCTEBuilder) buildResourceFilterCTE(ctx context.Context, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) (*qbtypes.Statement, error) {
return b.stmtBuilder.resourceFilterStmtBuilder.Build(
ctx,
b.orgID,
b.start,
b.end,
qbtypes.RequestTypeRaw,
@@ -261,6 +264,7 @@ func (b *traceOperatorCTEBuilder) buildQueryCTE(ctx context.Context, queryName s
query.Filter.Expression,
querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: b.orgID,
Logger: b.stmtBuilder.logger,
FieldMapper: b.stmtBuilder.fm,
ConditionBuilder: b.stmtBuilder.cb,
@@ -482,18 +486,18 @@ func (b *traceOperatorCTEBuilder) buildListQuery(ctx context.Context, selectFrom
"parent_span_id": true,
}
// Add selectFields using ColumnExpressionFor since we now have all base table columns
for _, field := range b.operator.SelectFields {
// Add selectFields since we now have all base table columns
for i, field := range b.operator.SelectFields {
if selectedFields[field.Name] {
continue
}
colExpr, err := b.stmtBuilder.fm.ColumnExpressionFor(ctx, b.start, b.end, &field, keys)
expr, err := b.stmtBuilder.fm.ColumnExpressionFor(ctx, b.orgID, b.start, b.end, &field, keys)
if err != nil {
b.stmtBuilder.logger.WarnContext(ctx, "failed to map select field",
slog.String("field", field.Name), errors.Attr(err))
continue
}
sb.SelectMore(colExpr)
sb.SelectMore(fmt.Sprintf("%s AS `%s`", sqlbuilder.Escape(expr), selectColumnAlias(i, field.Name)))
selectedFields[field.Name] = true
}
@@ -505,14 +509,14 @@ func (b *traceOperatorCTEBuilder) buildListQuery(ctx context.Context, selectFrom
sb.From(selectFromCTE)
// Add order by support using ColumnExpressionFor
// Add order by support
orderApplied := false
for _, orderBy := range b.operator.Order {
colExpr, err := b.stmtBuilder.fm.ColumnExpressionFor(ctx, b.start, b.end, &orderBy.Key.TelemetryFieldKey, keys)
expr, err := b.stmtBuilder.fm.ColumnExpressionFor(ctx, b.orgID, b.start, b.end, &orderBy.Key.TelemetryFieldKey, keys)
if err != nil {
return nil, err
}
sb.OrderBy(fmt.Sprintf("%s %s", colExpr, orderBy.Direction.StringValue()))
sb.OrderBy(fmt.Sprintf("%s %s", sqlbuilder.Escape(expr), orderBy.Direction.StringValue()))
orderApplied = true
}
@@ -626,21 +630,8 @@ func (b *traceOperatorCTEBuilder) buildTimeSeriesQuery(ctx context.Context, sele
int64(b.operator.StepInterval.Seconds()),
))
var allGroupByArgs []any
for _, gb := range b.operator.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(
ctx,
b.start,
b.end,
&gb.TelemetryFieldKey,
b.stmtBuilder.fm,
b.stmtBuilder.cb,
keys,
telemetrytypes.FieldDataTypeString,
nil,
false,
)
expr, err := b.stmtBuilder.fm.ColumnExpressionFor(ctx, b.orgID, b.start, b.end, &gb.TelemetryFieldKey, keys)
if err != nil {
return nil, errors.NewInvalidInputf(
errors.CodeInvalidInput,
@@ -649,15 +640,14 @@ func (b *traceOperatorCTEBuilder) buildTimeSeriesQuery(ctx context.Context, sele
err,
)
}
colExpr := fmt.Sprintf("toString(%s) AS `%s`", expr, gb.Name)
allGroupByArgs = append(allGroupByArgs, args...)
sb.SelectMore(colExpr)
sb.SelectMore(fmt.Sprintf("toString(%s) AS `%s`", sqlbuilder.Escape(expr), gb.Name))
}
var allAggChArgs []any
for i, agg := range b.operator.Aggregations {
rewritten, chArgs, err := b.stmtBuilder.aggExprRewriter.Rewrite(
ctx,
b.orgID,
b.start,
b.end,
agg.Expression,
@@ -701,7 +691,7 @@ func (b *traceOperatorCTEBuilder) buildTimeSeriesQuery(ctx context.Context, sele
}
sb.OrderBy("ts desc")
combinedArgs := append(allGroupByArgs, allAggChArgs...)
combinedArgs := allAggChArgs
// Add HAVING clause if specified
if err := b.addHavingClause(sb); err != nil {
@@ -736,21 +726,8 @@ func (b *traceOperatorCTEBuilder) buildTraceQuery(ctx context.Context, selectFro
sb := sqlbuilder.NewSelectBuilder()
var allGroupByArgs []any
for _, gb := range b.operator.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(
ctx,
b.start,
b.end,
&gb.TelemetryFieldKey,
b.stmtBuilder.fm,
b.stmtBuilder.cb,
keys,
telemetrytypes.FieldDataTypeString,
nil,
false,
)
expr, err := b.stmtBuilder.fm.ColumnExpressionFor(ctx, b.orgID, b.start, b.end, &gb.TelemetryFieldKey, keys)
if err != nil {
return nil, errors.NewInvalidInputf(
errors.CodeInvalidInput,
@@ -759,9 +736,7 @@ func (b *traceOperatorCTEBuilder) buildTraceQuery(ctx context.Context, selectFro
err,
)
}
colExpr := fmt.Sprintf("toString(%s) AS `%s`", expr, gb.Name)
allGroupByArgs = append(allGroupByArgs, args...)
sb.SelectMore(colExpr)
sb.SelectMore(fmt.Sprintf("toString(%s) AS `%s`", sqlbuilder.Escape(expr), gb.Name))
}
rateInterval := (b.end - b.start) / querybuilder.NsToSeconds
@@ -770,6 +745,7 @@ func (b *traceOperatorCTEBuilder) buildTraceQuery(ctx context.Context, selectFro
for i, agg := range b.operator.Aggregations {
rewritten, chArgs, err := b.stmtBuilder.aggExprRewriter.Rewrite(
ctx,
b.orgID,
b.start,
b.end,
agg.Expression,
@@ -864,7 +840,7 @@ func (b *traceOperatorCTEBuilder) buildTraceQuery(ctx context.Context, selectFro
sb.Limit(b.operator.Limit)
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
combinedArgs := allAggChArgs
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse, combinedArgs...)
return &qbtypes.Statement{
@@ -876,21 +852,8 @@ func (b *traceOperatorCTEBuilder) buildTraceQuery(ctx context.Context, selectFro
func (b *traceOperatorCTEBuilder) buildScalarQuery(ctx context.Context, selectFromCTE string, keys map[string][]*telemetrytypes.TelemetryFieldKey) (*qbtypes.Statement, error) {
sb := sqlbuilder.NewSelectBuilder()
var allGroupByArgs []any
for _, gb := range b.operator.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(
ctx,
b.start,
b.end,
&gb.TelemetryFieldKey,
b.stmtBuilder.fm,
b.stmtBuilder.cb,
keys,
telemetrytypes.FieldDataTypeString,
nil,
false,
)
expr, err := b.stmtBuilder.fm.ColumnExpressionFor(ctx, b.orgID, b.start, b.end, &gb.TelemetryFieldKey, keys)
if err != nil {
return nil, errors.NewInvalidInputf(
errors.CodeInvalidInput,
@@ -899,15 +862,14 @@ func (b *traceOperatorCTEBuilder) buildScalarQuery(ctx context.Context, selectFr
err,
)
}
colExpr := fmt.Sprintf("toString(%s) AS `%s`", expr, gb.Name)
allGroupByArgs = append(allGroupByArgs, args...)
sb.SelectMore(colExpr)
sb.SelectMore(fmt.Sprintf("toString(%s) AS `%s`", sqlbuilder.Escape(expr), gb.Name))
}
var allAggChArgs []any
for i, agg := range b.operator.Aggregations {
rewritten, chArgs, err := b.stmtBuilder.aggExprRewriter.Rewrite(
ctx,
b.orgID,
b.start,
b.end,
agg.Expression,
@@ -954,7 +916,7 @@ func (b *traceOperatorCTEBuilder) buildScalarQuery(ctx context.Context, selectFr
sb.OrderBy("__result_0 DESC")
}
combinedArgs := append(allGroupByArgs, allAggChArgs...)
combinedArgs := allAggChArgs
// Add HAVING clause if specified
if err := b.addHavingClause(sb); err != nil {

View File

@@ -11,17 +11,18 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
func newTestTraceOperatorStatementBuilder(t *testing.T) *traceOperatorStatementBuilder {
t.Helper()
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
fl := flaggertest.New(t)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
traceStmtBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -86,7 +87,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_DIR_DESC_B AS (SELECT p.* FROM A AS p INNER JOIN B AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) AS `service.name` FROM A_DIR_DESC_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_DIR_DESC_B AS (SELECT p.* FROM A AS p INNER JOIN B AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) AS `__SELECT_KEY_0_service.name` FROM A_DIR_DESC_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "backend", "%service.name%", "%service.name\":\"backend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -123,7 +124,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B AS (WITH RECURSIVE up AS (SELECT d.trace_id, d.span_id, d.parent_span_id, 0 AS depth FROM B AS d UNION ALL SELECT p.trace_id, p.span_id, p.parent_span_id, up.depth + 1 FROM all_spans AS p JOIN up ON p.trace_id = up.trace_id AND p.span_id = up.parent_span_id WHERE up.depth < 100) SELECT DISTINCT a.* FROM A AS a GLOBAL INNER JOIN (SELECT DISTINCT trace_id, span_id FROM up WHERE depth > 0 ) AS ancestors ON ancestors.trace_id = a.trace_id AND ancestors.span_id = a.span_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `trace_state`, flags AS `flags`, kind AS `kind`, kind_string AS `kind_string`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_INDIR_DESC_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B AS (WITH RECURSIVE up AS (SELECT d.trace_id, d.span_id, d.parent_span_id, 0 AS depth FROM B AS d UNION ALL SELECT p.trace_id, p.span_id, p.parent_span_id, up.depth + 1 FROM all_spans AS p JOIN up ON p.trace_id = up.trace_id AND p.span_id = up.parent_span_id WHERE up.depth < 100) SELECT DISTINCT a.* FROM A AS a GLOBAL INNER JOIN (SELECT DISTINCT trace_id, span_id FROM up WHERE depth > 0 ) AS ancestors ON ancestors.trace_id = a.trace_id AND ancestors.span_id = a.span_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `__SELECT_KEY_3_trace_state`, flags AS `__SELECT_KEY_5_flags`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_INDIR_DESC_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "gateway", "%service.name%", "%service.name\":\"gateway%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "database", "%service.name%", "%service.name\":\"database%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 5},
},
expectedErr: nil,
@@ -160,7 +161,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_AND_B AS (SELECT l.* FROM A AS l INNER JOIN B AS r ON l.trace_id = r.trace_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `trace_state`, flags AS `flags`, kind AS `kind`, kind_string AS `kind_string`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_AND_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_AND_B AS (SELECT l.* FROM A AS l INNER JOIN B AS r ON l.trace_id = r.trace_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `__SELECT_KEY_3_trace_state`, flags AS `__SELECT_KEY_5_flags`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_AND_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "backend", "%service.name%", "%service.name\":\"backend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 15},
},
expectedErr: nil,
@@ -197,7 +198,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_OR_B AS (SELECT * FROM A UNION DISTINCT SELECT * FROM B) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `trace_state`, flags AS `flags`, kind AS `kind`, kind_string AS `kind_string`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_OR_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_OR_B AS (SELECT * FROM A UNION DISTINCT SELECT * FROM B) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `__SELECT_KEY_3_trace_state`, flags AS `__SELECT_KEY_5_flags`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_OR_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "backend", "%service.name%", "%service.name\":\"backend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 20},
},
expectedErr: nil,
@@ -234,7 +235,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_not_B AS (SELECT l.* FROM A AS l WHERE l.trace_id GLOBAL NOT IN (SELECT DISTINCT trace_id FROM B)) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `trace_state`, flags AS `flags`, kind AS `kind`, kind_string AS `kind_string`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_not_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_not_B AS (SELECT l.* FROM A AS l WHERE l.trace_id GLOBAL NOT IN (SELECT DISTINCT trace_id FROM B)) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `__SELECT_KEY_3_trace_state`, flags AS `__SELECT_KEY_5_flags`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_not_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "backend", "%service.name%", "%service.name\":\"backend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -283,7 +284,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_DIR_DESC_B AS (SELECT p.* FROM A AS p INNER JOIN B AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id) SELECT toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, count() AS __result_0 FROM A_DIR_DESC_B GROUP BY ts, `service.name` ORDER BY ts desc SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_DIR_DESC_B AS (SELECT p.* FROM A AS p INNER JOIN B AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id) SELECT toStartOfInterval(timestamp, INTERVAL 60 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `service.name`, count() AS __result_0 FROM A_DIR_DESC_B GROUP BY ts, `service.name` ORDER BY ts desc SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "backend", "%service.name%", "%service.name\":\"backend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -342,7 +343,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND toFloat64(response_status_code) < ?), A_AND_B AS (SELECT l.* FROM A AS l INNER JOIN B AS r ON l.trace_id = r.trace_id) SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `service.name`, avg(multiIf(duration_nano <> ?, duration_nano, NULL)) AS __result_0 FROM A_AND_B GROUP BY `service.name` ORDER BY __result_0 desc SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND toFloat64(response_status_code) < ?), A_AND_B AS (SELECT l.* FROM A AS l INNER JOIN B AS r ON l.trace_id = r.trace_id) SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `service.name`, avg(multiIf(duration_nano <> ?, duration_nano, NULL)) AS __result_0 FROM A_AND_B GROUP BY `service.name` ORDER BY __result_0 desc SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), float64(400), 0},
},
expectedErr: nil,
@@ -399,7 +400,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_DIR_DESC_B AS (SELECT p.* FROM A AS p INNER JOIN B AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id), __resource_filter_C AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), C AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_C) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_D AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), D AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_D) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), C_DIR_DESC_D AS (SELECT p.* FROM C AS p INNER JOIN D AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id), A_DIR_DESC_B_AND_C_DIR_DESC_D AS (SELECT l.* FROM A_DIR_DESC_B AS l INNER JOIN C_DIR_DESC_D AS r ON l.trace_id = r.trace_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `trace_state`, flags AS `flags`, kind AS `kind`, kind_string AS `kind_string`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_DIR_DESC_B_AND_C_DIR_DESC_D ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_DIR_DESC_B AS (SELECT p.* FROM A AS p INNER JOIN B AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id), __resource_filter_C AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), C AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_C) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_D AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), D AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_D) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), C_DIR_DESC_D AS (SELECT p.* FROM C AS p INNER JOIN D AS c ON p.trace_id = c.trace_id AND p.span_id = c.parent_span_id), A_DIR_DESC_B_AND_C_DIR_DESC_D AS (SELECT l.* FROM A_DIR_DESC_B AS l INNER JOIN C_DIR_DESC_D AS r ON l.trace_id = r.trace_id) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `__SELECT_KEY_3_trace_state`, flags AS `__SELECT_KEY_5_flags`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM A_DIR_DESC_B_AND_C_DIR_DESC_D ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "frontend", "%service.name%", "%service.name\":\"frontend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "backend", "%service.name%", "%service.name\":\"backend%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "auth", "%service.name%", "%service.name\":\"auth%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "database", "%service.name%", "%service.name\":\"database%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 5},
},
expectedErr: nil,
@@ -433,7 +434,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B AS (WITH RECURSIVE up AS (SELECT d.trace_id, d.span_id, d.parent_span_id, 0 AS depth FROM B AS d UNION ALL SELECT p.trace_id, p.span_id, p.parent_span_id, up.depth + 1 FROM all_spans AS p JOIN up ON p.trace_id = up.trace_id AND p.span_id = up.parent_span_id WHERE up.depth < 100) SELECT DISTINCT a.* FROM A AS a GLOBAL INNER JOIN (SELECT DISTINCT trace_id, span_id FROM up WHERE depth > 0 ) AS ancestors ON ancestors.trace_id = a.trace_id AND ancestors.span_id = a.span_id), __return_from_B AS (SELECT * FROM B WHERE trace_id IN (SELECT DISTINCT trace_id FROM A_INDIR_DESC_B)) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `trace_state`, flags AS `flags`, kind AS `kind`, kind_string AS `kind_string`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM __return_from_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B AS (WITH RECURSIVE up AS (SELECT d.trace_id, d.span_id, d.parent_span_id, 0 AS depth FROM B AS d UNION ALL SELECT p.trace_id, p.span_id, p.parent_span_id, up.depth + 1 FROM all_spans AS p JOIN up ON p.trace_id = up.trace_id AND p.span_id = up.parent_span_id WHERE up.depth < 100) SELECT DISTINCT a.* FROM A AS a GLOBAL INNER JOIN (SELECT DISTINCT trace_id, span_id FROM up WHERE depth > 0 ) AS ancestors ON ancestors.trace_id = a.trace_id AND ancestors.span_id = a.span_id), __return_from_B AS (SELECT * FROM B WHERE trace_id IN (SELECT DISTINCT trace_id FROM A_INDIR_DESC_B)) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `__SELECT_KEY_3_trace_state`, flags AS `__SELECT_KEY_5_flags`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM __return_from_B ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "gateway", "%service.name%", "%service.name\":\"gateway%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "database", "%service.name%", "%service.name\":\"database%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -475,7 +476,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B AS (WITH RECURSIVE up AS (SELECT d.trace_id, d.span_id, d.parent_span_id, 0 AS depth FROM B AS d UNION ALL SELECT p.trace_id, p.span_id, p.parent_span_id, up.depth + 1 FROM all_spans AS p JOIN up ON p.trace_id = up.trace_id AND p.span_id = up.parent_span_id WHERE up.depth < 100) SELECT DISTINCT a.* FROM A AS a GLOBAL INNER JOIN (SELECT DISTINCT trace_id, span_id FROM up WHERE depth > 0 ) AS ancestors ON ancestors.trace_id = a.trace_id AND ancestors.span_id = a.span_id), __resource_filter_C AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), C AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_C) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B_AND_C AS (SELECT l.* FROM A_INDIR_DESC_B AS l INNER JOIN C AS r ON l.trace_id = r.trace_id), __return_from_C AS (SELECT * FROM C WHERE trace_id IN (SELECT DISTINCT trace_id FROM A_INDIR_DESC_B_AND_C)) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `trace_state`, flags AS `flags`, kind AS `kind`, kind_string AS `kind_string`, status_code AS `status_code`, status_message AS `status_message`, status_code_string AS `status_code_string`, events AS `events`, links AS `links`, response_status_code AS `response_status_code`, external_http_url AS `external_http_url`, http_url AS `http_url`, external_http_method AS `external_http_method`, http_method AS `http_method`, http_host AS `http_host`, db_name AS `db_name`, db_operation AS `db_operation`, has_error AS `has_error`, is_remote AS `is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM __return_from_C ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH toDateTime64(1747947419000000000, 9) AS t_from, toDateTime64(1747983448000000000, 9) AS t_to, 1747945619 AS bucket_from, 1747983448 AS bucket_to, all_spans AS (SELECT *, resource_string_service$$name AS `service.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_A AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), A AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_A) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __resource_filter_B AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), B AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_B) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B AS (WITH RECURSIVE up AS (SELECT d.trace_id, d.span_id, d.parent_span_id, 0 AS depth FROM B AS d UNION ALL SELECT p.trace_id, p.span_id, p.parent_span_id, up.depth + 1 FROM all_spans AS p JOIN up ON p.trace_id = up.trace_id AND p.span_id = up.parent_span_id WHERE up.depth < 100) SELECT DISTINCT a.* FROM A AS a GLOBAL INNER JOIN (SELECT DISTINCT trace_id, span_id FROM up WHERE depth > 0 ) AS ancestors ON ancestors.trace_id = a.trace_id AND ancestors.span_id = a.span_id), __resource_filter_C AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), C AS (SELECT * FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter_C) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), A_INDIR_DESC_B_AND_C AS (SELECT l.* FROM A_INDIR_DESC_B AS l INNER JOIN C AS r ON l.trace_id = r.trace_id), __return_from_C AS (SELECT * FROM C WHERE trace_id IN (SELECT DISTINCT trace_id FROM A_INDIR_DESC_B_AND_C)) SELECT timestamp, trace_id, span_id, name, duration_nano, parent_span_id, trace_state AS `__SELECT_KEY_3_trace_state`, flags AS `__SELECT_KEY_5_flags`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM __return_from_C ORDER BY timestamp DESC LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "gateway", "%service.name%", "%service.name\":\"gateway%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "database", "%service.name%", "%service.name\":\"database%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "auth", "%service.name%", "%service.name\":\"auth%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -492,6 +493,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
q, err := statementBuilder.Build(
context.Background(),
valuer.UUID{},
1747947419000,
1747983448000,
c.requestType,
@@ -582,6 +584,7 @@ func TestTraceOperatorStatementBuilderErrors(t *testing.T) {
if err == nil { // Only proceed if parsing succeeded
_, err = statementBuilder.Build(
context.Background(),
valuer.UUID{},
1747947419000,
1747983448000,
qbtypes.RequestTypeRaw,
@@ -657,6 +660,7 @@ func TestTraceOperatorStatementBuilderAdjustsKeys(t *testing.T) {
q, err := statementBuilder.Build(
context.Background(),
valuer.UUID{},
1747947419000,
1747983448000,
c.requestType,
@@ -711,6 +715,7 @@ func TestTraceOperatorStatementBuilderDeduplicatesKeys(t *testing.T) {
q, err := statementBuilder.Build(
context.Background(),
valuer.UUID{},
1747947419000,
1747983448000,
qbtypes.RequestTypeScalar,
@@ -732,5 +737,5 @@ func TestTraceOperatorStatementBuilderDeduplicatesKeys(t *testing.T) {
require.NoError(t, err)
require.Contains(t, q.Query,
"SELECT toString(multiIf(mapContains(attributes_string, 'http.method') = ?, attributes_string['http.method'], NULL)) AS `http.method`, count() AS __result_0 FROM A GROUP BY `http.method` ORDER BY __result_0 DESC")
"SELECT toString(multiIf(mapContains(attributes_string, 'http.method'), attributes_string['http.method'], NULL)) AS `http.method`, count() AS __result_0 FROM A GROUP BY `http.method` ORDER BY __result_0 DESC")
}

View File

@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type traceOperatorStatementBuilder struct {
@@ -61,6 +62,7 @@ func NewTraceOperatorStatementBuilder(
// Build builds a SQL query based on the given parameters.
func (b *traceOperatorStatementBuilder) Build(
ctx context.Context,
orgID valuer.UUID,
start uint64,
end uint64,
requestType qbtypes.RequestType,
@@ -91,6 +93,7 @@ func (b *traceOperatorStatementBuilder) Build(
builder := &traceOperatorCTEBuilder{
start: start,
end: end,
orgID: orgID,
operator: &query,
stmtBuilder: b,
queries: make(map[string]*qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]),

View File

@@ -12,6 +12,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -114,6 +115,7 @@ func TestTraceTimeRangeOptimization(t *testing.T) {
stmt, err := statementBuilder.Build(
ctx,
valuer.UUID{},
1747947419000, // start time in ms
1747983448000, // end time in ms
qbtypes.RequestTypeRaw,

View File

@@ -6,6 +6,7 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -25,8 +26,14 @@ type FieldMapper interface {
FieldFor(ctx context.Context, tsStart, tsEnd uint64, key *telemetrytypes.TelemetryFieldKey) (string, error)
// ColumnFor returns the column for the given key.
ColumnFor(ctx context.Context, tsStart, tsEnd uint64, key *telemetrytypes.TelemetryFieldKey) ([]*schema.Column, error)
// ColumnExpressionFor returns the column expression for the given key.
ColumnExpressionFor(ctx context.Context, tsStart, tsEnd uint64, key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) (string, error)
// ColumnExpressionFor returns the column expression for the given key. Most mappers
// return it aliased (`expr AS name`); the traces mapper returns a bare expression and
// callers add their own alias.
ColumnExpressionFor(ctx context.Context, orgID valuer.UUID, tsStart, tsEnd uint64, key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) (string, error)
// CandidateKeys returns the key(s) to query for a referenced field: metadata matches for
// the name (or `{context}.{name}`) first, else synthesized type-variant keys for sources
// that support it, else nil (caller errors). value is the filter operand, nil otherwise.
CandidateKeys(ctx context.Context, orgID valuer.UUID, field *telemetrytypes.TelemetryFieldKey, value any, keys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey
}
// ConditionBuilder builds the conditions for the filter.
@@ -36,13 +43,24 @@ type ConditionBuilder interface {
// the set of known field keys matching it (may be empty). The builder owns the
// decision of what to do — resolve ambiguity, fall back to a body JSON search,
// emit a "not found" error, or skip — and which errors/warnings are apt.
ConditionFor(ctx context.Context, startNs uint64, endNs uint64, key *telemetrytypes.TelemetryFieldKey, fieldKeysForName []*telemetrytypes.TelemetryFieldKey, operator FilterOperator, value any, sb *sqlbuilder.SelectBuilder) ([]string, []string, error)
ConditionFor(ctx context.Context, orgID valuer.UUID, startNs uint64, endNs uint64, key *telemetrytypes.TelemetryFieldKey, fieldKeysForName []*telemetrytypes.TelemetryFieldKey, operator FilterOperator, value any, sb *sqlbuilder.SelectBuilder) ([]string, []string, error)
}
// ResolvingConditionBuilder is an optional ConditionBuilder that owns key resolution itself:
// the where-clause visitor hands over the raw key + full metadata map instead of pre-matching,
// so lookup, collision handling, resource-filter skip, and synthesis live in one place.
type ResolvingConditionBuilder interface {
ConditionForKeys(ctx context.Context, orgID valuer.UUID, startNs uint64, endNs uint64, key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey, options ConditionBuilderOptions, operator FilterOperator, value any, sb *sqlbuilder.SelectBuilder) ([]string, []string, error)
}
type ConditionBuilderOptions struct {
// SkipResourceFilter drops the resource context from the candidate set.
SkipResourceFilter bool
}
type AggExprRewriter interface {
// Rewrite rewrites the aggregation expression to be used in the query.
Rewrite(ctx context.Context, startNs, endNs uint64, expr string, rateInterval uint64, keys map[string][]*telemetrytypes.TelemetryFieldKey) (string, []any, error)
RewriteMulti(ctx context.Context, startNs, endNs uint64, exprs []string, rateInterval uint64, keys map[string][]*telemetrytypes.TelemetryFieldKey) ([]string, [][]any, error)
Rewrite(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, expr string, rateInterval uint64, keys map[string][]*telemetrytypes.TelemetryFieldKey) (string, []any, error)
RewriteMulti(ctx context.Context, orgID valuer.UUID, startNs, endNs uint64, exprs []string, rateInterval uint64, keys map[string][]*telemetrytypes.TelemetryFieldKey) ([]string, [][]any, error)
}
type Statement struct {
@@ -55,12 +73,12 @@ type Statement struct {
// StatementBuilder builds the query.
type StatementBuilder[T any] interface {
// Build builds the query.
Build(ctx context.Context, start, end uint64, requestType RequestType, query QueryBuilderQuery[T], variables map[string]VariableItem) (*Statement, error)
Build(ctx context.Context, orgID valuer.UUID, start, end uint64, requestType RequestType, query QueryBuilderQuery[T], variables map[string]VariableItem) (*Statement, error)
}
type TraceOperatorStatementBuilder interface {
// Build builds the trace operator query.
Build(ctx context.Context, start, end uint64, requestType RequestType, query QueryBuilderTraceOperator, compositeQuery *CompositeQuery) (*Statement, error)
Build(ctx context.Context, orgID valuer.UUID, start, end uint64, requestType RequestType, query QueryBuilderTraceOperator, compositeQuery *CompositeQuery) (*Statement, error)
}
// StatementProvider renders a query's underlying statement without executing it.

View File

@@ -28,6 +28,7 @@ pytest_plugins = [
"fixtures.serviceaccount",
"fixtures.role",
"fixtures.seed_golden_dataset",
"fixtures.maildev",
]

View File

@@ -4,6 +4,7 @@ import time
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
from urllib.parse import urlparse
import pytest
import requests
@@ -13,6 +14,7 @@ from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.fs import get_testdata_file_path
from fixtures.logger import setup_logger
from fixtures.logs import Logs
from fixtures.maildev import verify_email_received
from fixtures.metrics import Metrics
from fixtures.traces import Traces
@@ -218,3 +220,131 @@ def update_rule_channel_name(rule_data: dict, channel_name: str):
# loop over all the sepcs and update the channels
for spec in thresholds["spec"]:
spec["channels"] = [channel_name]
def _is_json_subset(subset, superset) -> bool:
"""Check if subset is contained within superset recursively.
- For dicts: all keys in subset must exist in superset with matching values
- For lists: all items in subset must be present in superset
- For scalars: exact equality
"""
if isinstance(subset, dict):
if not isinstance(superset, dict):
return False
return all(key in superset and _is_json_subset(value, superset[key]) for key, value in subset.items())
if isinstance(subset, list):
if not isinstance(superset, list):
return False
return all(any(_is_json_subset(sub_item, sup_item) for sup_item in superset) for sub_item in subset)
return subset == superset
def verify_webhook_notification_expectation(
notification_channel: types.TestContainerDocker,
validation_data: dict,
) -> bool:
"""Check if wiremock received a request at the given path
whose JSON body is a superset of the expected json_body."""
path = validation_data["path"]
json_body = validation_data["json_body"]
url = notification_channel.host_configs["8080"].get("__admin/requests/find")
try:
res = requests.post(url, json={"method": "POST", "url": path}, timeout=10)
except requests.exceptions.RequestException:
return False
if res.status_code != HTTPStatus.OK:
return False
for req in res.json()["requests"]:
body = json.loads(base64.b64decode(req["bodyAsBase64"]).decode("utf-8"))
# logger.info("Webhook request body: %s", json.dumps(body, indent=2))
if _is_json_subset(json_body, body):
return True
return False
def _check_notification_validation(
validation: types.NotificationValidation,
notification_channel: types.TestContainerDocker,
maildev: types.TestContainerDocker,
) -> bool:
"""Dispatch a single validation check to the appropriate verifier."""
if validation.destination_type == "webhook":
return verify_webhook_notification_expectation(notification_channel, validation.validation_data)
if validation.destination_type == "email":
return verify_email_received(maildev, validation.validation_data)
raise ValueError(f"Invalid destination type: {validation.destination_type}")
def verify_notification_expectation(
notification_channel: types.TestContainerDocker,
maildev: types.TestContainerDocker,
expected_notification: types.AMNotificationExpectation,
) -> bool:
"""Poll for expected notifications across webhook and email channels."""
time_to_wait = datetime.now() + timedelta(seconds=expected_notification.wait_time_seconds)
while datetime.now() < time_to_wait:
all_found = all(_check_notification_validation(v, notification_channel, maildev) for v in expected_notification.notification_validations)
if expected_notification.should_notify and all_found:
logger.info("All expected notifications found")
return True
time.sleep(1)
# Timeout reached
if not expected_notification.should_notify:
# Verify no notifications were received
for validation in expected_notification.notification_validations:
found = _check_notification_validation(validation, notification_channel, maildev)
assert not found, f"Expected no notification but found one for {validation.destination_type} with data {validation.validation_data}"
logger.info("No notifications found, as expected")
return True
# Expected notifications but didn't get them all — report missing
missing = [v for v in expected_notification.notification_validations if not _check_notification_validation(v, notification_channel, maildev)]
assert len(missing) == 0, f"Expected all notifications to be found but missing: {missing}"
return True
def update_raw_channel_config(
channel_config: dict,
channel_name: str,
notification_channel: types.TestContainerDocker,
maildev: types.TestContainerDocker | None = None,
) -> dict:
"""
Updates the channel config to point to the given wiremock
notification_channel container to receive notifications.
"""
config = channel_config.copy()
config["name"] = channel_name
url_field_map = {
"slack_configs": "api_url",
"msteamsv2_configs": "webhook_url",
"webhook_configs": "url",
"pagerduty_configs": "url",
"opsgenie_configs": "api_url",
}
for config_key, url_field in url_field_map.items():
if config_key in config:
for entry in config[config_key]:
if url_field in entry:
original_url = entry[url_field]
path = urlparse(original_url).path
entry[url_field] = notification_channel.container_configs["8080"].get(path)
# Inject live maildev SMTP address into email configs so the test works
if maildev is not None and "email_configs" in config:
smtp_host = maildev.container_configs["1025"].address
smtp_port = str(maildev.container_configs["1025"].port)
for entry in config["email_configs"]:
entry["smarthost"] = f"{smtp_host}:{smtp_port}"
entry["require_tls"] = False
return config

View File

@@ -124,14 +124,19 @@ def gateway(
@pytest.fixture(name="make_http_mocks", scope="function")
def make_http_mocks() -> Callable[[types.TestContainerDocker, list[Mapping]], None]:
def make_http_mocks(
request: pytest.FixtureRequest,
) -> Callable[[types.TestContainerDocker, list[Mapping]], None]:
def _make_http_mocks(container: types.TestContainerDocker, mappings: list[Mapping]) -> None:
Config.base_url = container.host_configs["8080"].get("/__admin")
for mapping in mappings:
Mappings.create_mapping(mapping=mapping)
yield _make_http_mocks
def cleanup():
Mappings.delete_all_mappings()
Requests.reset_request_journal()
Mappings.delete_all_mappings()
Requests.reset_request_journal()
request.addfinalizer(cleanup)
return _make_http_mocks

122
tests/fixtures/maildev.py vendored Normal file
View File

@@ -0,0 +1,122 @@
import json
from http import HTTPStatus
import docker
import docker.errors
import pytest
import requests
from testcontainers.core.container import DockerContainer, Network
from fixtures import reuse, types
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
@pytest.fixture(name="maildev", scope="package")
def maildev(network: Network, request: pytest.FixtureRequest, pytestconfig: pytest.Config) -> types.TestContainerDocker:
"""
Package-scoped fixture for MailDev container.
Provides SMTP (port 1025) and HTTP API (port 1080) for email testing.
"""
def create() -> types.TestContainerDocker:
container = DockerContainer(image="maildev/maildev:2.2.1")
container.with_exposed_ports(1025, 1080)
container.with_network(network=network)
container.start()
return types.TestContainerDocker(
id=container.get_wrapped_container().id,
host_configs={
"1025": types.TestContainerUrlConfig(
scheme="smtp",
address=container.get_container_host_ip(),
port=container.get_exposed_port(1025),
),
"1080": types.TestContainerUrlConfig(
scheme="http",
address=container.get_container_host_ip(),
port=container.get_exposed_port(1080),
),
},
container_configs={
"1025": types.TestContainerUrlConfig(
scheme="smtp",
address=container.get_wrapped_container().name,
port=1025,
),
"1080": types.TestContainerUrlConfig(
scheme="http",
address=container.get_wrapped_container().name,
port=1080,
),
},
)
def delete(container: types.TestContainerDocker):
client = docker.from_env()
try:
client.containers.get(container_id=container.id).stop()
client.containers.get(container_id=container.id).remove(v=True)
except docker.errors.NotFound:
logger.info(
"Skipping removal of MailDev, MailDev(%s) not found. Maybe it was manually removed?",
{"id": container.id},
)
def restore(cache: dict) -> types.TestContainerDocker:
return types.TestContainerDocker.from_cache(cache)
return reuse.wrap(
request,
pytestconfig,
"maildev",
lambda: types.TestContainerDocker(id="", host_configs={}, container_configs={}),
create,
delete,
restore,
)
def get_all_mails(_maildev: types.TestContainerDocker) -> list[dict]:
"""
Fetches all emails from the MailDev HTTP API.
Returns list of dicts with keys: subject, html, text.
"""
url = _maildev.host_configs["1080"].get("/email")
response = requests.get(url, timeout=5)
assert response.status_code == HTTPStatus.OK, f"Failed to fetch emails from MailDev, status code: {response.status_code}, response: {response.text}"
emails = response.json()
# logger.info("Emails: %s", json.dumps(emails, indent=2))
return [
{
"subject": email.get("subject", ""),
"html": email.get("html", ""),
"text": email.get("text", ""),
}
for email in emails
]
def verify_email_received(_maildev: types.TestContainerDocker, filters: dict) -> bool:
"""
Checks if any email in MailDev matches all the given filters.
Filters are matched with exact equality against the email fields (subject, html, text).
Returns True if at least one matching email is found.
"""
emails = get_all_mails(_maildev)
for email in emails:
logger.info("Email: %s", json.dumps(email, indent=2))
if all(key in email and filter_value == email[key] for key, filter_value in filters.items()):
return True
return False
def delete_all_mails(_maildev: types.TestContainerDocker) -> None:
"""
Deletes all emails from the MailDev inbox.
"""
url = _maildev.host_configs["1080"].get("/email/all")
response = requests.delete(url, timeout=5)
assert response.status_code == HTTPStatus.OK, f"Failed to delete emails from MailDev, status code: {response.status_code}, response: {response.text}"

View File

@@ -1,3 +1,4 @@
# pylint: disable=line-too-long
from collections.abc import Callable
from http import HTTPStatus
@@ -15,6 +16,87 @@ from fixtures.logger import setup_logger
logger = setup_logger(__name__)
"""
Default notification channel configs shared across alertmanager tests.
"""
slack_default_config = {
# channel name configured on runtime
"slack_configs": [
{
"api_url": "services/TEAM_ID/BOT_ID/TOKEN_ID", # base_url configured on runtime
"title": '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }} for {{ .CommonLabels.job }}\n {{- if gt (len .CommonLabels) (len .GroupLabels) -}}\n {{" "}}(\n {{- with .CommonLabels.Remove .GroupLabels.Names }}\n {{- range $index, $label := .SortedPairs -}}\n {{ if $index }}, {{ end }}\n {{- $label.Name }}="{{ $label.Value -}}"\n {{- end }}\n {{- end -}}\n )\n {{- end }}',
"text": '{{ range .Alerts -}}\r\n *Alert:* {{ .Labels.alertname }}{{ if .Labels.severity }} - {{ .Labels.severity }}{{ end }}\r\n\r\n *Summary:* {{ .Annotations.summary }}\r\n *Description:* {{ .Annotations.description }}\r\n *RelatedLogs:* {{ if gt (len .Annotations.related_logs) 0 -}} View in <{{ .Annotations.related_logs }}|logs explorer> {{- end}}\r\n *RelatedTraces:* {{ if gt (len .Annotations.related_traces) 0 -}} View in <{{ .Annotations.related_traces }}|traces explorer> {{- end}}\r\n\r\n *Details:*\r\n {{ range .Labels.SortedPairs -}}\r\n {{- if ne .Name "ruleId" -}}\r\n \u2022 *{{ .Name }}:* {{ .Value }}\r\n {{ end -}}\r\n {{ end -}}\r\n{{ end }}',
}
],
}
# MSTeams default config
msteams_default_config = {
"msteamsv2_configs": [
{
"webhook_url": "msteams/webhook_url", # base_url configured on runtime
"title": '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }} for {{ .CommonLabels.job }}\n {{- if gt (len .CommonLabels) (len .GroupLabels) -}}\n {{" "}}(\n {{- with .CommonLabels.Remove .GroupLabels.Names }}\n {{- range $index, $label := .SortedPairs -}}\n {{ if $index }}, {{ end }}\n {{- $label.Name }}="{{ $label.Value -}}"\n {{- end }}\n {{- end -}}\n )\n {{- end }}',
"text": '{{ range .Alerts -}}\r\n *Alert:* {{ .Labels.alertname }}{{ if .Labels.severity }} - {{ .Labels.severity }}{{ end }}\r\n\r\n *Summary:* {{ .Annotations.summary }}\r\n *Description:* {{ .Annotations.description }}\r\n *RelatedLogs:* {{ if gt (len .Annotations.related_logs) 0 -}} View in <{{ .Annotations.related_logs }}|logs explorer> {{- end}}\r\n *RelatedTraces:* {{ if gt (len .Annotations.related_traces) 0 -}} View in <{{ .Annotations.related_traces }}|traces explorer> {{- end}}\r\n\r\n *Details:*\r\n {{ range .Labels.SortedPairs -}}\r\n {{- if ne .Name "ruleId" -}}\r\n \u2022 *{{ .Name }}:* {{ .Value }}\r\n {{ end -}}\r\n {{ end -}}\r\n{{ end }}',
}
],
}
# pagerduty default config
pagerduty_default_config = {
"pagerduty_configs": [
{
"routing_key": "PagerDutyRoutingKey",
"url": "v2/enqueue", # base_url configured on runtime
"client": "SigNoz Alert Manager",
"client_url": "https://enter-signoz-host-n-port-here/alerts",
"description": '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }} for {{ .CommonLabels.job }}\n\t{{- if gt (len .CommonLabels) (len .GroupLabels) -}}\n\t {{" "}}(\n\t {{- with .CommonLabels.Remove .GroupLabels.Names }}\n\t\t{{- range $index, $label := .SortedPairs -}}\n\t\t {{ if $index }}, {{ end }}\n\t\t {{- $label.Name }}="{{ $label.Value -}}"\n\t\t{{- end }}\n\t {{- end -}}\n\t )\n\t{{- end }}',
"details": {
"firing": '{{ template "pagerduty.default.instances" .Alerts.Firing }}',
"num_firing": "{{ .Alerts.Firing | len }}",
"num_resolved": "{{ .Alerts.Resolved | len }}",
"resolved": '{{ template "pagerduty.default.instances" .Alerts.Resolved }}',
},
"source": "SigNoz Alert Manager",
"severity": "{{ (index .Alerts 0).Labels.severity }}",
}
],
}
# opsgenie default config
opsgenie_default_config = {
"opsgenie_configs": [
{
"api_key": "OpsGenieAPIKey",
"api_url": "/", # base_url configured on runtime
"description": '{{ if gt (len .Alerts.Firing) 0 -}}\r\n\tAlerts Firing:\r\n\t{{ range .Alerts.Firing }}\r\n\t - Message: {{ .Annotations.description }}\r\n\tLabels:\r\n\t{{ range .Labels.SortedPairs -}}\r\n\t\t{{- if ne .Name "ruleId" }} - {{ .Name }} = {{ .Value }}\r\n\t{{ end -}}\r\n\t{{- end }} Annotations:\r\n\t{{ range .Annotations.SortedPairs }} - {{ .Name }} = {{ .Value }}\r\n\t{{ end }} Source: {{ .GeneratorURL }}\r\n\t{{ end }}\r\n{{- end }}\r\n{{ if gt (len .Alerts.Resolved) 0 -}}\r\n\tAlerts Resolved:\r\n\t{{ range .Alerts.Resolved }}\r\n\t - Message: {{ .Annotations.description }}\r\n\tLabels:\r\n\t{{ range .Labels.SortedPairs -}}\r\n\t\t{{- if ne .Name "ruleId" }} - {{ .Name }} = {{ .Value }}\r\n\t{{ end -}}\r\n\t{{- end }} Annotations:\r\n\t{{ range .Annotations.SortedPairs }} - {{ .Name }} = {{ .Value }}\r\n\t{{ end }} Source: {{ .GeneratorURL }}\r\n\t{{ end }}\r\n{{- end }}',
"priority": '{{ if eq (index .Alerts 0).Labels.severity "critical" }}P1{{ else if eq (index .Alerts 0).Labels.severity "warning" }}P2{{ else if eq (index .Alerts 0).Labels.severity "info" }}P3{{ else }}P4{{ end }}',
"message": "{{ .CommonLabels.alertname }}",
"details": {},
}
]
}
# webhook default config
webhook_default_config = {
"webhook_configs": [
{
"url": "webhook/webhook_url", # base_url configured on runtime
}
],
}
# email default config
email_default_config = {
"email_configs": [
{
"to": "test@example.com",
"html": '<html><body>{{ range .Alerts -}}\r\n *Alert:* {{ .Labels.alertname }}{{ if .Labels.severity }} - {{ .Labels.severity }}{{ end }}\r\n\r\n *Summary:* {{ .Annotations.summary }}\r\n *Description:* {{ .Annotations.description }}\r\n *RelatedLogs:* {{ if gt (len .Annotations.related_logs) 0 -}} View in <{{ .Annotations.related_logs }}|logs explorer> {{- end}}\r\n *RelatedTraces:* {{ if gt (len .Annotations.related_traces) 0 -}} View in <{{ .Annotations.related_traces }}|traces explorer> {{- end}}\r\n\r\n *Details:*\r\n {{ range .Labels.SortedPairs -}}\r\n {{- if ne .Name "ruleId" -}}\r\n \u2022 *{{ .Name }}:* {{ .Value }}\r\n {{ end -}}\r\n {{ end -}}\r\n{{ end }}</body></html>',
"headers": {
"Subject": '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .CommonLabels.alertname }} for {{ .CommonLabels.job }}\n {{- if gt (len .CommonLabels) (len .GroupLabels) -}}\n {{" "}}(\n {{- with .CommonLabels.Remove .GroupLabels.Names }}\n {{- range $index, $label := .SortedPairs -}}\n {{ if $index }}, {{ end }}\n {{- $label.Name }}="{{ $label.Value -}}"\n {{- end }}\n {{- end -}}\n )\n {{- end }}'
},
}
],
}
@pytest.fixture(name="notification_channel", scope="package")
def notification_channel(
network: Network,
@@ -67,6 +149,27 @@ def notification_channel(
)
@pytest.fixture(name="create_notification_channel", scope="function")
def create_notification_channel(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> Callable[[dict], str]:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
def _create_notification_channel(channel_config: dict) -> str:
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/channels"),
json=channel_config,
headers={"Authorization": f"Bearer {admin_token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.CREATED, f"Failed to create channel, Response: {response.text} Response status: {response.status_code}"
return response.json()["data"]["id"]
return _create_notification_channel
@pytest.fixture(name="create_webhook_notification_channel", scope="function")
def create_webhook_notification_channel(
signoz: types.SigNoz,

View File

@@ -613,7 +613,7 @@ def build_order_by(name: str, direction: str = "desc") -> dict:
return {"key": {"name": name}, "direction": direction}
def build_logs_aggregation(expression: str, alias: str | None = None) -> dict:
def build_aggregation(expression: str, alias: str | None = None) -> dict:
agg: dict[str, Any] = {"expression": expression}
if alias:
agg["alias"] = alias

View File

@@ -192,3 +192,40 @@ class AlertTestCase:
alert_data: list[AlertData]
# list of alert expectations for the test case
alert_expectation: AlertExpectation
@dataclass(frozen=True)
class NotificationValidation:
# destination type of the notification, either webhook or email
# slack, msteams, pagerduty, opsgenie, webhook channels send notifications through webhook
# email channels send notifications through email
destination_type: Literal["webhook", "email"]
# validation data for validating the received notification payload
validation_data: dict[str, any]
@dataclass(frozen=True)
class AMNotificationExpectation:
# whether we expect any notifications to be fired or not, false when testing downtime scenarios
# or don't expect any notifications to be fired in given time period
should_notify: bool
# seconds to wait for the notifications to be fired, if no
# notifications are fired in the expected time, the test will fail
wait_time_seconds: int
# list of notifications to expect, as a single rule can trigger multiple notifications
# spanning across different notifiers
notification_validations: list[NotificationValidation]
@dataclass(frozen=True)
class AlertManagerNotificationTestCase:
# name of the test case
name: str
# path to the rule file in testdata directory
rule_path: str
# list of alert data that will be inserted into the database for the rule to be triggered
alert_data: list[AlertData]
# configuration for the notification channel
channel_config: dict[str, any]
# notification expectations for the test case
notification_expectation: AMNotificationExpectation

View File

@@ -39,5 +39,7 @@ def test_teardown(
idp: types.TestContainerIDP, # pylint: disable=unused-argument
create_user_admin: types.Operation, # pylint: disable=unused-argument
migrator: types.Operation, # pylint: disable=unused-argument
maildev: types.TestContainerDocker, # pylint: disable=unused-argument
notification_channel: types.TestContainerDocker, # pylint: disable=unused-argument
) -> None:
pass

View File

@@ -0,0 +1,20 @@
{ "timestamp": "2026-01-29T10:00:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "User login successful", "severity_text": "INFO" }
{ "timestamp": "2026-01-29T10:00:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: gateway timeout", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:01:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: card declined", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:01:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "Database connection established", "severity_text": "INFO" }
{ "timestamp": "2026-01-29T10:02:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: insufficient funds", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:02:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: invalid token", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:03:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "API request received", "severity_text": "INFO" }
{ "timestamp": "2026-01-29T10:03:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: gateway timeout", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:04:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: card declined", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:04:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: invalid token", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:05:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: gateway timeout", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:05:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: card declined", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:06:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: gateway timeout", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:06:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: insufficient funds", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:07:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: card declined", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:07:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: gateway timeout", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:08:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "Response sent to client", "severity_text": "INFO" }
{ "timestamp": "2026-01-29T10:08:30.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: invalid token", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:09:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: card declined", "severity_text": "ERROR" }
{ "timestamp": "2026-01-29T10:10:00.000000Z", "resources": { "service.name": "payment-service" }, "attributes": { "code.file": "payment_handler.py" }, "body": "payment failure: gateway timeout", "severity_text": "ERROR" }

View File

@@ -0,0 +1,69 @@
{
"alert": "content_templating_logs",
"ruleType": "threshold_rule",
"alertType": "LOGS_BASED_ALERT",
"condition": {
"thresholds": {
"kind": "basic",
"spec": [
{
"name": "critical",
"target": 0,
"matchType": "1",
"op": "1",
"channels": [
"test channel"
]
}
]
},
"compositeQuery": {
"queryType": "builder",
"panelType": "graph",
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "logs",
"filter": {
"expression": "body CONTAINS 'payment failure'"
},
"aggregations": [
{
"expression": "count()"
}
],
"groupBy": [
{"name": "service.name", "fieldContext": "resource"}
]
}
}
]
},
"selectedQueryName": "A"
},
"evaluation": {
"kind": "rolling",
"spec": {
"evalWindow": "5m0s",
"frequency": "15s"
}
},
"labels": {},
"annotations": {
"description": "Payment failure spike detected on $service_name",
"summary": "Payment failures elevated on $service_name"
},
"notificationSettings": {
"groupBy": [],
"usePolicy": false,
"renotify": {
"enabled": false,
"interval": "30m",
"alertStates": []
}
},
"version": "v5",
"schemaVersion": "v2alpha1"
}

View File

@@ -0,0 +1,12 @@
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:01:00+00:00","value":80,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:02:00+00:00","value":95,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:03:00+00:00","value":110,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:04:00+00:00","value":120,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:05:00+00:00","value":125,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:06:00+00:00","value":130,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:07:00+00:00","value":135,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:08:00+00:00","value":140,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:09:00+00:00","value":145,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:10:00+00:00","value":150,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:11:00+00:00","value":155,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}
{"metric_name":"container_memory_bytes_content_templating","labels":{"namespace":"production","pod":"checkout-7d9c8b5f4-x2k9p","container":"checkout","node":"ip-10-0-1-23","severity":"critical","service":"checkout"},"timestamp":"2026-01-29T10:12:00+00:00","value":160,"temporality":"Unspecified","type_":"Gauge","is_monotonic":false,"flags":0,"description":"","unit":"bytes","env":"default","resource_attrs":{},"scope_attrs":{}}

View File

@@ -0,0 +1,72 @@
{
"alert": "content_templating_metrics",
"ruleType": "threshold_rule",
"alertType": "METRIC_BASED_ALERT",
"condition": {
"thresholds": {
"kind": "basic",
"spec": [
{
"name": "critical",
"target": 100,
"matchType": "1",
"op": "1",
"channels": [
"test channel"
]
}
]
},
"compositeQuery": {
"queryType": "builder",
"panelType": "graph",
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "metrics",
"aggregations": [
{
"metricName": "container_memory_bytes_content_templating",
"timeAggregation": "avg",
"spaceAggregation": "max"
}
],
"groupBy": [
{"name": "namespace", "fieldContext": "attribute", "fieldDataType": "string"},
{"name": "pod", "fieldContext": "attribute", "fieldDataType": "string"},
{"name": "container", "fieldContext": "attribute", "fieldDataType": "string"},
{"name": "node", "fieldContext": "attribute", "fieldDataType": "string"},
{"name": "severity", "fieldContext": "attribute", "fieldDataType": "string"}
]
}
}
]
},
"selectedQueryName": "A"
},
"evaluation": {
"kind": "rolling",
"spec": {
"evalWindow": "5m0s",
"frequency": "15s"
}
},
"labels": {},
"annotations": {
"description": "Container $container in pod $pod ($namespace) exceeded memory threshold",
"summary": "High container memory in $namespace/$pod"
},
"notificationSettings": {
"groupBy": [],
"usePolicy": false,
"renotify": {
"enabled": false,
"interval": "30m",
"alertStates": []
}
},
"version": "v5",
"schemaVersion": "v2alpha1"
}

View File

@@ -0,0 +1,20 @@
{ "timestamp": "2026-01-29T10:00:00.000000Z", "duration": "PT1.2S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a1", "span_id": "c1b2c3d4e5f6a7b8", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:00:30.000000Z", "duration": "PT1.4S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a2", "span_id": "c2b3c4d5e6f7a8b9", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:01:00.000000Z", "duration": "PT1.6S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a3", "span_id": "c3b4c5d6e7f8a9b0", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:01:30.000000Z", "duration": "PT1.8S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a4", "span_id": "c4b5c6d7e8f9a0b1", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:02:00.000000Z", "duration": "PT2.1S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a5", "span_id": "c5b6c7d8e9f0a1b2", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:02:30.000000Z", "duration": "PT2.3S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a6", "span_id": "c6b7c8d9e0f1a2b3", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:03:00.000000Z", "duration": "PT2.5S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a7", "span_id": "c7b8c9d0e1f2a3b4", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:03:30.000000Z", "duration": "PT2.7S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a8", "span_id": "c8b9c0d1e2f3a4b5", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:04:00.000000Z", "duration": "PT2.9S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6a9", "span_id": "c9b0c1d2e3f4a5b6", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:04:30.000000Z", "duration": "PT3.1S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b1", "span_id": "d1c2d3e4f5a6b7c8", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:05:00.000000Z", "duration": "PT3.3S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b2", "span_id": "d2c3d4e5f6a7b8c9", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:05:30.000000Z", "duration": "PT3.5S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b3", "span_id": "d3c4d5e6f7a8b9c0", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:06:00.000000Z", "duration": "PT3.7S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b4", "span_id": "d4c5d6e7f8a9b0c1", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:06:30.000000Z", "duration": "PT3.9S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b5", "span_id": "d5c6d7e8f9a0b1c2", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:07:00.000000Z", "duration": "PT4.1S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b6", "span_id": "d6c7d8e9f0a1b2c3", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:07:30.000000Z", "duration": "PT4.3S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b7", "span_id": "d7c8d9e0f1a2b3c4", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:08:00.000000Z", "duration": "PT4.5S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b8", "span_id": "d8c9d0e1f2a3b4c5", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:08:30.000000Z", "duration": "PT4.7S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6b9", "span_id": "d9c0d1e2f3a4b5c6", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:09:00.000000Z", "duration": "PT4.9S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6c1", "span_id": "e1d2e3f4a5b6c7d8", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }
{ "timestamp": "2026-01-29T10:10:00.000000Z", "duration": "PT5.1S", "trace_id": "591f6d3d6b0a1f9e8a71b2c3d4e5f6c2", "span_id": "e2d3e4f5a6b7c8d9", "parent_span_id": "", "name": "POST /checkout", "kind": 2, "status_code": 1, "status_message": "", "resources": { "deployment.environment": "production", "service.name": "checkout-service", "os.type": "linux", "host.name": "ip-10-0-1-23" }, "attributes": { "net.transport": "IP.TCP", "http.scheme": "http", "http.user_agent": "Integration Test", "http.request.method": "POST", "http.response.status_code": "200", "http.request.path": "/checkout" } }

View File

@@ -0,0 +1,71 @@
{
"alert": "content_templating_traces",
"ruleType": "threshold_rule",
"alertType": "TRACES_BASED_ALERT",
"condition": {
"thresholds": {
"kind": "basic",
"spec": [
{
"name": "critical",
"target": 1,
"matchType": "1",
"op": "1",
"channels": [
"test channel"
],
"targetUnit": "s"
}
]
},
"compositeQuery": {
"queryType": "builder",
"unit": "ns",
"panelType": "graph",
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "traces",
"filter": {
"expression": "http.request.path = '/checkout'"
},
"aggregations": [
{
"expression": "p90(duration_nano)"
}
],
"groupBy": [
{"name": "service.name", "fieldContext": "resource"}
]
}
}
]
},
"selectedQueryName": "A"
},
"evaluation": {
"kind": "rolling",
"spec": {
"evalWindow": "5m0s",
"frequency": "15s"
}
},
"labels": {},
"annotations": {
"description": "p90 latency high on $service_name",
"summary": "p90 latency exceeded threshold on $service_name"
},
"notificationSettings": {
"groupBy": [],
"usePolicy": false,
"renotify": {
"enabled": false,
"interval": "30m",
"alertStates": []
}
},
"version": "v5",
"schemaVersion": "v2alpha1"
}

View File

@@ -0,0 +1,372 @@
import json
import uuid
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
import pytest
from wiremock.client import HttpMethods, Mapping, MappingRequest, MappingResponse
from fixtures import types
from fixtures.alerts import (
get_testdata_file_path,
update_raw_channel_config,
update_rule_channel_name,
verify_notification_expectation,
)
from fixtures.logger import setup_logger
from fixtures.maildev import delete_all_mails
from fixtures.notification_channel import (
email_default_config,
msteams_default_config,
opsgenie_default_config,
pagerduty_default_config,
slack_default_config,
webhook_default_config,
)
# tests to verify the notifiers sending out the notifications with expected content
NOTIFIERS_TEST = [
types.AlertManagerNotificationTestCase(
name="slack_notifier_default_templating",
rule_path="alerts/test_scenarios/threshold_above_at_least_once/rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alerts/test_scenarios/threshold_above_at_least_once/alert_data.jsonl",
),
],
channel_config=slack_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
# extra wait for alertmanager server setup
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/services/TEAM_ID/BOT_ID/TOKEN_ID",
"json_body": {
"username": "Alertmanager",
"attachments": [
{
"color": "danger",
"mrkdwn_in": ["fallback", "pretext", "text"],
}
],
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="msteams_notifier_default_templating",
rule_path="alerts/test_scenarios/threshold_above_at_least_once/rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alerts/test_scenarios/threshold_above_at_least_once/alert_data.jsonl",
),
],
channel_config=msteams_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/msteams/webhook_url",
"json_body": {
"type": "message",
"attachments": [
{
"contentType": "application/vnd.microsoft.card.adaptive",
"content": {
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.2",
"body": [
{
"type": "TextBlock",
"text": "Alerts",
"weight": "Bolder",
"size": "Medium",
"wrap": True,
"color": "Attention",
},
{
"type": "TextBlock",
"text": "Labels",
"weight": "Bolder",
"size": "Medium",
},
{
"type": "FactSet",
"text": "",
"facts": [
{
"title": "threshold.name",
"value": "critical",
}
],
},
{
"type": "TextBlock",
"text": "Annotations",
"weight": "Bolder",
"size": "Medium",
},
{
"type": "FactSet",
"text": "",
"facts": [
{
"title": "description",
"value": "This alert is fired when the defined metric (current value: 15) crosses the threshold (10)",
},
],
},
],
"msteams": {"width": "full"},
"actions": [
{
"type": "Action.OpenUrl",
"title": "View Alert",
}
],
},
}
],
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="pagerduty_notifier_default_templating",
rule_path="alerts/test_scenarios/threshold_above_at_least_once/rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alerts/test_scenarios/threshold_above_at_least_once/alert_data.jsonl",
),
],
channel_config=pagerduty_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/v2/enqueue",
"json_body": {
"routing_key": "PagerDutyRoutingKey",
"event_action": "trigger",
"payload": {
"source": "SigNoz Alert Manager",
"severity": "critical",
"custom_details": {
"firing": {
"Annotations": [
{"description = This alert is fired when the defined metric (current value": "15) crosses the threshold (10)"},
],
"Labels": [
"alertname = threshold_above_at_least_once",
"severity = critical",
"threshold.name = critical",
],
}
},
},
"client": "SigNoz Alert Manager",
"client_url": "https://enter-signoz-host-n-port-here/alerts",
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="opsgenie_notifier_default_templating",
rule_path="alerts/test_scenarios/threshold_above_at_least_once/rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alerts/test_scenarios/threshold_above_at_least_once/alert_data.jsonl",
),
],
channel_config=opsgenie_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/v2/alerts",
"json_body": {
"message": "threshold_above_at_least_once",
"details": {
"alertname": "threshold_above_at_least_once",
"severity": "critical",
"threshold.name": "critical",
},
"priority": "P1",
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="webhook_notifier_default_templating",
rule_path="alerts/test_scenarios/threshold_above_at_least_once/rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alerts/test_scenarios/threshold_above_at_least_once/alert_data.jsonl",
),
],
channel_config=webhook_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/webhook/webhook_url",
"json_body": {
"status": "firing",
"alerts": [
{
"status": "firing",
"labels": {
"alertname": "threshold_above_at_least_once",
"severity": "critical",
"threshold.name": "critical",
},
"annotations": {
"description": "This alert is fired when the defined metric (current value: 15) crosses the threshold (10)",
"summary": "This alert is fired when the defined metric (current value: 15) crosses the threshold (10)",
},
}
],
"commonLabels": {
"alertname": "threshold_above_at_least_once",
"severity": "critical",
"threshold.name": "critical",
},
"commonAnnotations": {
"description": "This alert is fired when the defined metric (current value: 15) crosses the threshold (10)",
"summary": "This alert is fired when the defined metric (current value: 15) crosses the threshold (10)",
},
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="email_notifier_default_templating",
rule_path="alerts/test_scenarios/threshold_above_at_least_once/rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alerts/test_scenarios/threshold_above_at_least_once/alert_data.jsonl",
),
],
channel_config=email_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="email",
validation_data={
"subject": '[FIRING:1] threshold_above_at_least_once for (alertname="threshold_above_at_least_once", severity="critical", threshold.name="critical")',
},
),
],
),
),
]
logger = setup_logger(__name__)
@pytest.mark.parametrize(
"notifier_test_case",
NOTIFIERS_TEST,
ids=lambda notifier_test_case: notifier_test_case.name,
)
def test_notifier_templating(
# wiremock container for webhook notifications
notification_channel: types.TestContainerDocker,
# function to create wiremock mocks
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
create_notification_channel: Callable[[dict], str],
# function to create alert rule
create_alert_rule: Callable[[dict], str],
# Alert data insertion related fixture
insert_alert_data: Callable[[list[types.AlertData], datetime], None],
# Mail dev container for email verification
maildev: types.TestContainerDocker,
# test case from parametrize
notifier_test_case: types.AlertManagerNotificationTestCase,
):
# generate unique channel name
channel_name = str(uuid.uuid4())
# update channel config: set name and rewrite URLs to wiremock
channel_config = update_raw_channel_config(notifier_test_case.channel_config, channel_name, notification_channel, maildev)
logger.info("Channel config: %s", {"channel_config": channel_config})
# setup wiremock mocks for webhook-based notification validations
webhook_validations = [v for v in notifier_test_case.notification_expectation.notification_validations if v.destination_type == "webhook"]
if len(webhook_validations) > 0:
mock_mappings = [
Mapping(
request=MappingRequest(method=HttpMethods.POST, url=v.validation_data["path"]),
response=MappingResponse(status=200, json_body={}),
persistent=False,
)
for v in webhook_validations
]
make_http_mocks(notification_channel, mock_mappings)
logger.info("Mock mappings created")
# clear mails if any destination is email
if any(v.destination_type == "email" for v in notifier_test_case.notification_expectation.notification_validations):
delete_all_mails(maildev)
logger.info("Mails deleted")
# create notification channel
create_notification_channel(channel_config)
logger.info("Channel created with name: %s", {"channel_name": channel_name})
# insert alert data
insert_alert_data(
notifier_test_case.alert_data,
base_time=datetime.now(tz=UTC) - timedelta(minutes=5),
)
# create alert rule
rule_path = get_testdata_file_path(notifier_test_case.rule_path)
with open(rule_path, encoding="utf-8") as f:
rule_data = json.loads(f.read())
update_rule_channel_name(rule_data, channel_name)
rule_id = create_alert_rule(rule_data)
logger.info("rule created: %s", {"rule_id": rule_id, "rule_name": rule_data["alert"]})
# verify notification expectations
verify_notification_expectation(
notification_channel,
maildev,
notifier_test_case.notification_expectation,
)

View File

@@ -0,0 +1,341 @@
import json
import uuid
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
import pytest
from wiremock.client import HttpMethods, Mapping, MappingRequest, MappingResponse
from fixtures import types
from fixtures.alerts import (
get_testdata_file_path,
update_raw_channel_config,
update_rule_channel_name,
verify_notification_expectation,
)
from fixtures.logger import setup_logger
from fixtures.maildev import delete_all_mails
from fixtures.notification_channel import (
msteams_default_config,
opsgenie_default_config,
pagerduty_default_config,
slack_default_config,
webhook_default_config,
)
logger = setup_logger(__name__)
# Test cases verifying default notification content
CONTENT_TEMPLATING_TEST = [
types.AlertManagerNotificationTestCase(
name="msteams_metrics_default_templating",
rule_path="alertmanager/content_templating/metrics_rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alertmanager/content_templating/metrics_data.jsonl",
),
],
channel_config=msteams_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/msteams/webhook_url",
"json_body": {
"type": "message",
"attachments": [
{
"contentType": "application/vnd.microsoft.card.adaptive",
"content": {
"type": "AdaptiveCard",
"body": [
{
"type": "TextBlock",
"text": '[FIRING:1] content_templating_metrics for (alertname="content_templating_metrics", container="checkout", namespace="production", node="ip-10-0-1-23", pod="checkout-7d9c8b5f4-x2k9p", severity="critical", threshold.name="critical")',
},
],
},
}
],
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="opsgenie_metrics_default_templating",
rule_path="alertmanager/content_templating/metrics_rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alertmanager/content_templating/metrics_data.jsonl",
),
],
channel_config=opsgenie_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/v2/alerts",
"json_body": {
"message": "content_templating_metrics",
"details": {
"alertname": "content_templating_metrics",
"container": "checkout",
"namespace": "production",
"node": "ip-10-0-1-23",
"pod": "checkout-7d9c8b5f4-x2k9p",
"severity": "critical",
"threshold.name": "critical",
},
"priority": "P1",
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="pagerduty_metrics_default_templating",
rule_path="alertmanager/content_templating/metrics_rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alertmanager/content_templating/metrics_data.jsonl",
),
],
channel_config=pagerduty_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/v2/enqueue",
"json_body": {
"routing_key": "PagerDutyRoutingKey",
"payload": {
"severity": "critical",
"custom_details": {
"firing": {
"Labels": [
"alertname = content_templating_metrics",
"container = checkout",
"namespace = production",
"node = ip-10-0-1-23",
"pod = checkout-7d9c8b5f4-x2k9p",
"severity = critical",
"threshold.name = critical",
],
}
},
},
"client": "SigNoz Alert Manager",
"client_url": "https://enter-signoz-host-n-port-here/alerts",
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="slack_logs_default_templating",
rule_path="alertmanager/content_templating/logs_rule.json",
alert_data=[
types.AlertData(
type="logs",
data_path="alertmanager/content_templating/logs_data.jsonl",
),
],
channel_config=slack_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/services/TEAM_ID/BOT_ID/TOKEN_ID",
"json_body": {
"username": "Alertmanager",
"attachments": [
{
"color": "danger",
"mrkdwn_in": ["fallback", "pretext", "text"],
}
],
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="slack_metrics_default_templating",
rule_path="alertmanager/content_templating/metrics_rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alertmanager/content_templating/metrics_data.jsonl",
),
],
channel_config=slack_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/services/TEAM_ID/BOT_ID/TOKEN_ID",
"json_body": {
"username": "Alertmanager",
"attachments": [
{
"color": "danger",
"mrkdwn_in": ["fallback", "pretext", "text"],
}
],
},
},
),
],
),
),
types.AlertManagerNotificationTestCase(
name="webhook_metrics_default_templating",
rule_path="alertmanager/content_templating/metrics_rule.json",
alert_data=[
types.AlertData(
type="metrics",
data_path="alertmanager/content_templating/metrics_data.jsonl",
),
],
channel_config=webhook_default_config,
notification_expectation=types.AMNotificationExpectation(
should_notify=True,
wait_time_seconds=120,
notification_validations=[
types.NotificationValidation(
destination_type="webhook",
validation_data={
"path": "/webhook/webhook_url",
"json_body": {
"status": "firing",
"alerts": [
{
"status": "firing",
"labels": {
"alertname": "content_templating_metrics",
"container": "checkout",
"namespace": "production",
"node": "ip-10-0-1-23",
"pod": "checkout-7d9c8b5f4-x2k9p",
"severity": "critical",
"threshold.name": "critical",
},
"annotations": {
"description": "Container checkout in pod checkout-7d9c8b5f4-x2k9p (production) exceeded memory threshold",
"summary": "High container memory in production/checkout-7d9c8b5f4-x2k9p",
},
}
],
"commonLabels": {
"alertname": "content_templating_metrics",
"container": "checkout",
"namespace": "production",
"node": "ip-10-0-1-23",
"pod": "checkout-7d9c8b5f4-x2k9p",
"severity": "critical",
"threshold.name": "critical",
},
},
},
),
],
),
),
]
@pytest.mark.parametrize(
"default_templating_test_case",
CONTENT_TEMPLATING_TEST,
ids=lambda default_templating_test_case: default_templating_test_case.name,
)
def test_content_templating(
# wiremock container for webhook notifications
notification_channel: types.TestContainerDocker,
# function to create wiremock mocks
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
create_notification_channel: Callable[[dict], str],
# function to create alert rule
create_alert_rule: Callable[[dict], str],
# Alert data insertion related fixture
insert_alert_data: Callable[[list[types.AlertData], datetime], None],
# Mail dev container for email verification
maildev: types.TestContainerDocker,
# test case from parametrize
default_templating_test_case: types.AlertManagerNotificationTestCase,
):
# generate unique channel name
channel_name = str(uuid.uuid4())
# update channel config: set name and rewrite URLs to wiremock
channel_config = update_raw_channel_config(default_templating_test_case.channel_config, channel_name, notification_channel, maildev)
logger.info("Channel config: %s", {"channel_config": channel_config})
# setup wiremock mocks for webhook-based notification validations
webhook_validations = [v for v in default_templating_test_case.notification_expectation.notification_validations if v.destination_type == "webhook"]
if len(webhook_validations) > 0:
mock_mappings = [
Mapping(
request=MappingRequest(method=HttpMethods.POST, url=v.validation_data["path"]),
response=MappingResponse(status=200, json_body={}),
persistent=False,
)
for v in webhook_validations
]
make_http_mocks(notification_channel, mock_mappings)
logger.info("Mock mappings created")
# clear mails if any destination is email
if any(v.destination_type == "email" for v in default_templating_test_case.notification_expectation.notification_validations):
delete_all_mails(maildev)
logger.info("Mails deleted")
# create notification channel
create_notification_channel(channel_config)
logger.info("Channel created with name: %s", {"channel_name": channel_name})
# insert alert data
insert_alert_data(
default_templating_test_case.alert_data,
base_time=datetime.now(tz=UTC) - timedelta(minutes=10),
)
# create alert rule
rule_path = get_testdata_file_path(default_templating_test_case.rule_path)
with open(rule_path, encoding="utf-8") as f:
rule_data = json.loads(f.read())
update_rule_channel_name(rule_data, channel_name)
rule_id = create_alert_rule(rule_data)
logger.info("rule created: %s", {"rule_id": rule_id, "rule_name": rule_data["alert"]})
# verify notification expectations
verify_notification_expectation(
notification_channel,
maildev,
default_templating_test_case.notification_expectation,
)

View File

@@ -0,0 +1,42 @@
import pytest
from testcontainers.core.container import Network
from fixtures import types
from fixtures.signoz import create_signoz
@pytest.fixture(name="signoz", scope="package")
def signoz( # pylint: disable=too-many-arguments,too-many-positional-arguments
network: Network,
zeus: types.TestContainerDocker,
gateway: types.TestContainerDocker,
sqlstore: types.TestContainerSQL,
clickhouse: types.TestContainerClickhouse,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
maildev: types.TestContainerDocker,
notification_channel: types.TestContainerDocker,
) -> types.SigNoz:
"""
Package-scoped fixture for setting up SigNoz.
Overrides SMTP, PagerDuty, and OpsGenie URLs to point to test containers.
"""
return create_signoz(
network=network,
zeus=zeus,
gateway=gateway,
sqlstore=sqlstore,
clickhouse=clickhouse,
request=request,
pytestconfig=pytestconfig,
env_overrides={
# SMTP config for email notifications via maildev
"SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__SMARTHOST": f"{maildev.container_configs['1025'].address}:{maildev.container_configs['1025'].port}",
"SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__REQUIRE__TLS": "false",
"SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__FROM": "alertmanager@signoz.io",
# PagerDuty API URL -> wiremock (default: https://events.pagerduty.com/v2/enqueue)
"SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_PAGERDUTY__URL": notification_channel.container_configs["8080"].get("/v2/enqueue"),
# OpsGenie API URL -> wiremock (default: https://api.opsgenie.com/)
"SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_OPSGENIE__API__URL": notification_channel.container_configs["8080"].get("/"),
},
)

View File

@@ -9,7 +9,7 @@ from fixtures.audit import AuditLog
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.querier import (
BuilderQuery,
build_logs_aggregation,
build_aggregation,
build_order_by,
build_scalar_query,
make_query_request,
@@ -365,7 +365,7 @@ def test_audit_scalar_count_failures(
name="A",
signal="logs",
source="audit",
aggregations=[build_logs_aggregation("count()")],
aggregations=[build_aggregation("count()")],
filter_expression="signoz.audit.outcome = 'failure'",
)
],

View File

@@ -9,7 +9,7 @@ from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.logs import Logs
from fixtures.querier import (
build_logs_aggregation,
build_aggregation,
build_order_by,
build_raw_query,
build_scalar_query,
@@ -40,7 +40,7 @@ def _run_query_case(signoz: types.SigNoz, token: str, now: datetime, case: dict[
else:
aggregation = case.get("aggregation")
if aggregation and not isinstance(aggregation, list):
aggregations = [build_logs_aggregation(aggregation)]
aggregations = [build_aggregation(aggregation)]
elif aggregation:
aggregations = aggregation
else:

View File

@@ -9,8 +9,8 @@ from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.logs import Logs
from fixtures.querier import (
build_aggregation,
build_group_by_field,
build_logs_aggregation,
build_order_by,
build_raw_query,
build_scalar_query,
@@ -58,7 +58,7 @@ def _scalar(
q = build_scalar_query(
name=name,
signal="logs",
aggregations=[build_logs_aggregation(aggregation)],
aggregations=[build_aggregation(aggregation)],
filter_expression=expression,
group_by=group_by,
step_interval=60,

View File

@@ -6,9 +6,9 @@ from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.logs import Logs
from fixtures.querier import (
build_aggregation,
build_formula_query,
build_group_by_field,
build_logs_aggregation,
build_order_by,
build_scalar_query,
find_named_result,
@@ -99,7 +99,7 @@ def test_logs_formula_orderby_and_limit(
build_scalar_query(
name="A",
signal="logs",
aggregations=[build_logs_aggregation("count()")],
aggregations=[build_aggregation("count()")],
group_by=[build_group_by_field("service.name")],
filter_expression="severity_text = 'ERROR'",
disabled=True,
@@ -107,7 +107,7 @@ def test_logs_formula_orderby_and_limit(
build_scalar_query(
name="B",
signal="logs",
aggregations=[build_logs_aggregation("count()")],
aggregations=[build_aggregation("count()")],
group_by=[build_group_by_field("service.name")],
filter_expression="severity_text = 'INFO'",
disabled=True,

Some files were not shown because too many files have changed in this diff Show More