Compare commits

..

34 Commits

Author SHA1 Message Date
Gaurav Tewari
9c1e01d3e6 feat(llm-attribute-mapping): add Test tab with sample-span runner
Add the Test tab to the Attribute Mapping page: paste a sample span,
run it through the working draft's mappers, and see which target
attributes get populated.

- Wire useTestSpanMappers (POST /api/v1/span_mapper_groups/test) generated
  client from #11795 into a TestTab + useTestSpanMapper hook + TestResult.
- Build the request from the staged draft, sending a group's mappers only
  when it is new or its mapper set changed vs the saved snapshot; otherwise
  omit them so the backend backfills saved mappers by group name.
- Expose the saved snapshot from useAttributeMappingStore for the diff.
2026-06-23 10:49:10 +05:30
Gaurav Tewari
a358fc717b fix(llm-attribute-mapping): de-duplicate mapper sources by context+key
A key read from a span attribute vs the resource is a distinct source, so
both should be allowed to coexist as separate priorities. getCleanSources
de-duped by key alone, silently dropping the second context. Dedupe by the
(context, key) pair instead, keeping the higher-priority row.
2026-06-22 10:57:27 +05:30
Gaurav Tewari
feb6edc057 fix(llm-attribute-mapping): preserve unsaved mappers across lazy hydration
hydrateGroupMappers replaced a group's mappers with the fetched server list.
The Add-mapping drawer renders while the lazy fetch is in flight, so a mapper
staged before the response arrived (serverId === null) was clobbered on
resolve. Keep those local rows and fold the server mappers in ahead of them.
2026-06-22 10:52:51 +05:30
Gaurav Tewari
c9afc7a375 feat(llm-attribute-mapping): mapper create/edit/delete via drawer
Port the mapper-drawer feature onto the rewritten group-drawer (PR 3) base:
TanStack tables + CSS modules + config-based columns. Mapper rows are lazily
fetched on group expand, hydrated into the editable draft store, and managed
(create/edit/delete/toggle/reorder) through the drawer.
2026-06-22 00:06:43 +05:30
Gaurav Tewari
074915e22b chore(llm-attribute-mapping): remove orphaned IndexBadge
The TanStack mappers table dropped the leading index column, leaving
IndexBadge and its .indexBadge style unused. Remove both.
2026-06-21 00:06:15 +05:30
Gaurav Tewari
eaef369eb1 Merge branch 'feat/llm-attr-mapping-listing-2' into feat/llm-attr-mapping-group-drawer-3
# Conflicts:
#	frontend/src/container/LLMObservabilityAttributeMapping/AttributeMappingsTab.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/LLMObservabilityAttributeMapping.module.scss
#	frontend/src/container/LLMObservabilityAttributeMapping/LLMObservabilityAttributeMapping.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/MapperGroupsTable.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/MappersTable.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/useAttributeMappingStore.ts
#	frontend/src/container/LLMObservabilityAttributeMapping/utils.ts
2026-06-21 00:03:07 +05:30
Gaurav Tewari
a13969c8bc chore: migrate tanstack table 2026-06-20 23:45:23 +05:30
Gaurav Tewari
3f0d07c28c feat(llm-attribute-mapping): read-only listing on CSS modules [2/5]
Rebase the listing slice onto the foundation's CSS-module refactor
(which deleted the global stylesheet) and migrate it accordingly:

- Merge listing styles into LLMObservabilityAttributeMapping.module.scss
  (groups/mappers tables, source chips, index badge, error/footer).
- Convert all listing components from global BEM classNames to
  styles.* module access; drop dead/style-less classes (am-table,
  am-row-actions, am-add-row, *_edited, mappers-table__error).
- Adopt theme-aware semantic tokens (--l2/l3-*, --accent-primary,
  --callout-error-*) in place of --bg-* primitives.
2026-06-20 23:08:37 +05:30
Gaurav Tewari
f2b3c4125c refactor: css module 2026-06-20 23:08:24 +05:30
Gaurav Tewari
96cc6ec588 fix: css styling 2026-06-20 23:08:24 +05:30
Gaurav Tewari
c6705a72a7 feat(llm-attribute-mapping): add attribute mapping foundation (route, permission, page shell) 2026-06-20 23:08:24 +05:30
Gaurav Tewari
e1cb822091 chore(deps): bump @grafana/data and pin transitive deps to patched versions (#11796)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
- @grafana/data ^11.6.14 -> ^11.6.15
- http-proxy-middleware 4.0.0 -> 4.1.1 (dep + resolution)
- form-data 4.0.4 -> 4.0.6
- tmp 0.2.4 -> 0.2.7
- add js-cookie ^3.0.7 resolution pin (forces react-use's transitive copy to a patched range)

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-06-20 10:26:40 +00:00
Gaurav Tewari
44d76c1306 Merge branch 'feat/llm-attr-mapping-listing-2' into feat/llm-attr-mapping-group-drawer-3
# Conflicts:
#	frontend/src/container/LLMObservabilityAttributeMapping/AttributeMappingsTab.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/LLMObservabilityAttributeMapping.module.scss
#	frontend/src/container/LLMObservabilityAttributeMapping/LLMObservabilityAttributeMapping.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/MapperGroupsTable.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/MappersTable.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/useAttributeMappingStore.ts
#	frontend/src/container/LLMObservabilityAttributeMapping/utils.ts
2026-06-19 17:33:54 +05:30
Gaurav Tewari
b9ac79898e refactor(llm-attribute-mapping): migrate styles to CSS modules 2026-06-19 17:16:00 +05:30
Gaurav Tewari
c136834ab5 feat(llm-attribute-mapping): read-only listing on CSS modules [2/5]
Rebase the listing slice onto the foundation's CSS-module refactor
(which deleted the global stylesheet) and migrate it accordingly:

- Merge listing styles into LLMObservabilityAttributeMapping.module.scss
  (groups/mappers tables, source chips, index badge, error/footer).
- Convert all listing components from global BEM classNames to
  styles.* module access; drop dead/style-less classes (am-table,
  am-row-actions, am-add-row, *_edited, mappers-table__error).
- Adopt theme-aware semantic tokens (--l2/l3-*, --accent-primary,
  --callout-error-*) in place of --bg-* primitives.
2026-06-19 16:59:32 +05:30
Gaurav Tewari
dd0155065f refactor: css module 2026-06-19 16:43:00 +05:30
Gaurav Tewari
38f1506472 feat: update draft and mapper 2026-06-19 01:23:44 +05:30
Gaurav Tewari
7692f99f30 fix: update drawer 2026-06-19 01:05:35 +05:30
Gaurav Tewari
c3ba316ccf fix: update logic 2026-06-19 00:43:36 +05:30
Gaurav Tewari
eeb8353a17 fix: input 2026-06-19 00:28:16 +05:30
Gaurav Tewari
47b5fff3f6 Merge branch 'feat/llm-attr-mapping-listing-2' into feat/llm-attr-mapping-group-drawer-3
# Conflicts:
#	frontend/src/container/LLMObservabilityAttributeMapping/LLMObservabilityAttributeMapping.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/MapperGroupsTable.tsx
#	frontend/src/container/LLMObservabilityAttributeMapping/useAttributeMappingStore.ts
2026-06-19 00:09:48 +05:30
Gaurav Tewari
a0940af68f chore: review comments 2026-06-18 23:56:37 +05:30
Gaurav Tewari
72713d996d fix: update col count 2026-06-18 23:25:11 +05:30
Gaurav Tewari
93a1caa9cc feat: add attribute mapping tab 2026-06-18 23:13:33 +05:30
Gaurav Tewari
bb8900113a fix: fetch data only when tab is open 2026-06-18 22:50:58 +05:30
Gaurav Tewari
1fb195b383 fix: text overflow in attribute mapping 2026-06-18 22:24:51 +05:30
Gaurav Tewari
c2761c24df fix(llm-attribute-mapping): unlock filters column shrink so long keys wrap within the cell 2026-06-18 22:11:54 +05:30
Gaurav Tewari
a6b77993c6 fix(llm-attribute-mapping): wrap long condition keys so the context badge isn't clipped 2026-06-18 22:04:24 +05:30
Gaurav Tewari
59d2468fb1 style(llm-attribute-mapping): use design-token css variables for spacing and typography 2026-06-18 21:54:27 +05:30
Gaurav Tewari
44d8d55b2f Merge branch 'feat/llm-attr-mapping-foundation-1' into feat/llm-attr-mapping-listing-2 2026-06-18 21:35:09 +05:30
Gaurav Tewari
8fdce2b9ad fix: css styling 2026-06-18 21:20:45 +05:30
Gaurav Tewari
1c06a57361 feat(llm-attribute-mapping): group create/edit/delete via drawer with diff-save 2026-06-18 20:13:40 +05:30
Gaurav Tewari
dfd3bce670 feat(llm-attribute-mapping): read-only groups and mappers listing 2026-06-18 19:39:57 +05:30
Gaurav Tewari
9725a268cc feat(llm-attribute-mapping): add attribute mapping foundation (route, permission, page shell) 2026-06-18 19:33:02 +05:30
49 changed files with 3596 additions and 610 deletions

View File

@@ -43,7 +43,7 @@
"@dnd-kit/modifiers": "7.0.0",
"@dnd-kit/sortable": "8.0.0",
"@dnd-kit/utilities": "3.2.2",
"@grafana/data": "^11.6.14",
"@grafana/data": "^11.6.15",
"@monaco-editor/react": "^4.7.0",
"@sentry/react": "10.57.0",
"@sentry/vite-plugin": "5.3.0",
@@ -79,7 +79,7 @@
"event-source-polyfill": "1.0.31",
"eventemitter3": "5.0.1",
"history": "4.10.1",
"http-proxy-middleware": "4.0.0",
"http-proxy-middleware": "4.1.1",
"http-status-codes": "2.3.0",
"i18next": "^21.6.12",
"i18next-browser-languagedetector": "^6.1.3",
@@ -231,16 +231,17 @@
"xml2js": "0.5.0",
"phin": "^3.7.1",
"body-parser": "1.20.3",
"http-proxy-middleware": "4.0.0",
"http-proxy-middleware": "4.1.1",
"cross-spawn": "7.0.5",
"cookie": "^0.7.1",
"serialize-javascript": "6.0.2",
"prismjs": "1.30.0",
"got": "11.8.5",
"form-data": "4.0.4",
"form-data": "4.0.6",
"brace-expansion": "^2.0.2",
"on-headers": "^1.1.0",
"tmp": "0.2.4",
"js-cookie": "^3.0.7",
"tmp": "0.2.7",
"vite": "npm:rolldown-vite@7.3.1"
}
}

View File

@@ -12,16 +12,17 @@ overrides:
xml2js: 0.5.0
phin: ^3.7.1
body-parser: 1.20.3
http-proxy-middleware: 4.0.0
http-proxy-middleware: 4.1.1
cross-spawn: 7.0.5
cookie: ^0.7.1
serialize-javascript: 6.0.2
prismjs: 1.30.0
got: 11.8.5
form-data: 4.0.4
form-data: 4.0.6
brace-expansion: ^2.0.2
on-headers: ^1.1.0
tmp: 0.2.4
js-cookie: ^3.0.7
tmp: 0.2.7
vite: npm:rolldown-vite@7.3.1
importers:
@@ -56,8 +57,8 @@ importers:
specifier: 3.2.2
version: 3.2.2(react@18.2.0)
'@grafana/data':
specifier: ^11.6.14
version: 11.6.14(react-dom@18.2.0(react@18.2.0))(react@18.2.0)
specifier: ^11.6.15
version: 11.6.15(react-dom@18.2.0(react@18.2.0))(react@18.2.0)
'@monaco-editor/react':
specifier: ^4.7.0
version: 4.7.0(monaco-editor@0.55.1)(react-dom@18.2.0(react@18.2.0))(react@18.2.0)
@@ -164,8 +165,8 @@ importers:
specifier: 4.10.1
version: 4.10.1
http-proxy-middleware:
specifier: 4.0.0
version: 4.0.0
specifier: 4.1.1
version: 4.1.1
http-status-codes:
specifier: 2.3.0
version: 2.3.0
@@ -1636,14 +1637,14 @@ packages:
'@gerrit0/mini-shiki@3.23.0':
resolution: {integrity: sha512-bEMORlG0cqdjVyCEuU0cDQbORWX+kYCeo0kV1lbxF5bt4r7SID2l9bqsxJEM0zndaxpOUT7riCyIVEuqq/Ynxg==}
'@grafana/data@11.6.14':
resolution: {integrity: sha512-Nsjq1A9m6LbsKsKvOgvAk9Wq7RGjy0V4N9d5YsSnzMwCiw/ov2wblR2bcDpy95uF8KaDTIR2Gf40nJaOYksPMA==}
'@grafana/data@11.6.15':
resolution: {integrity: sha512-q2Zbjr0N9iEGY/zKHm4Z4X5x64806E17W58y7mnvwc0MlbyGPPVulcp/rWA2Nd190mZeafZQPer9u+MaO+0HUQ==}
peerDependencies:
react: ^18.0.0
react-dom: ^18.0.0
'@grafana/schema@11.6.14':
resolution: {integrity: sha512-YTqgYekb7kiu5NEoQxKF8czJ6QIARmMkCi9cNcynHqYpcDLOv5pg5Q0QtKgiiqHjlYoEeCV6iejdB4hXxzB+VA==}
'@grafana/schema@11.6.15':
resolution: {integrity: sha512-MPIvGAp9uzkswnH6e+Fmzu+WBTqWMgbv93/8iu56gb+sjCB2LciZLz4KvrPFdw32bWCGSMAGqsML9mgmeJZtGQ==}
'@humanfs/core@0.19.2':
resolution: {integrity: sha512-UhXNm+CFMWcbChXywFwkmhqjs3PRCmcSa/hfBgLIb7oQ5HNb1wS0icWsGtSAUNgefHeI+eBrA8I1fxmbHsGdvA==}
@@ -5167,8 +5168,8 @@ packages:
resolution: {integrity: sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw==}
engines: {node: '>=14'}
form-data@4.0.4:
resolution: {integrity: sha512-KrGhL9Q4zjj0kiUt5OO4Mr/A/jlI2jDYs5eHBpYHPcBEVSiipAvn2Ko2HnPe20rmcuuvMHNdZFp+4IlGTMF0Ow==}
form-data@4.0.6:
resolution: {integrity: sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==}
engines: {node: '>= 6'}
format@0.2.2:
@@ -5381,6 +5382,10 @@ packages:
resolution: {integrity: sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==}
engines: {node: '>= 0.4'}
hasown@2.0.4:
resolution: {integrity: sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A==}
engines: {node: '>= 0.4'}
hast-util-from-parse5@8.0.1:
resolution: {integrity: sha512-Er/Iixbc7IEa7r/XLtuG52zoqn/b3Xng/w6aZQ0xGVxzhw5xUFxcRqdPzP6yFi/4HBYRaifaI5fQ1RH8n0ZeOQ==}
@@ -5456,8 +5461,8 @@ packages:
resolution: {integrity: sha512-n2hY8YdoRE1i7r6M0w9DIw5GgZN0G25P8zLCRQ8rjXtTU3vsNFBI/vWK/UIeE6g5MUUz6avwAPXmL6Fy9D/90w==}
engines: {node: '>= 6'}
http-proxy-middleware@4.0.0:
resolution: {integrity: sha512-wuHwaUtmC0XzJNHqRp41zXtt5ojpHbusXGhq6781VvnjWUYPu7opmOF3eomGNujT07kEOnHWZyV9UZzKimVCKA==}
http-proxy-middleware@4.1.1:
resolution: {integrity: sha512-KX5ZofGXLFXqFAkQoOWZ+rTtaLTut7m0gyL+QzJrdejtIZ+F4bPPDoe7reISg2+v0CAz5OfVwEJEhty7X+e57g==}
engines: {node: ^22.15.0 || ^24.0.0 || >=26.0.0}
http-status-codes@2.3.0:
@@ -5467,8 +5472,8 @@ packages:
resolution: {integrity: sha512-dFcAjpTQFgoLMzC2VwU+C/CbS7uRL0lWmxDITmqm7C+7F0Odmj6s9l6alZc6AELXhrnggM2CeWSXHGOdX2YtwA==}
engines: {node: '>= 6'}
httpxy@0.5.1:
resolution: {integrity: sha512-JPhqYiixe1A1I+MXDewWDZqeudBGU8Q9jCHYN8ML+779RQzLjTi78HBvWz4jMxUD6h2/vUL12g4q/mFM0OUw1A==}
httpxy@0.5.3:
resolution: {integrity: sha512-SMS9V6Sn7VWaS11lYhoAr0ceoaiolTWf4jYdJn0NJhCdKMu9R2H9Fh0LBDWBHQF6HRLI1PmaePYsjanSpE5PEw==}
human-signals@2.1.0:
resolution: {integrity: sha512-B4FFZ6q/T2jhhksgkbEW3HBvWIfDW85snkQgawt07S7J5QXTk6BkNV+0yAeZrM5QpMAdYlocGoljn0sJ/WQkFw==}
@@ -6041,8 +6046,8 @@ packages:
js-base64@3.7.5:
resolution: {integrity: sha512-3MEt5DTINKqfScXKfJFrRbxkrnk2AxPWGBL/ycjz4dK8iqiSJ06UxD8jh8xuh6p10TX4t2+7FsBYVxxQbMg+qA==}
js-cookie@2.2.1:
resolution: {integrity: sha512-HvdH2LzI/EAZcUwA8+0nKNtWHqS+ZmijLA30RwZA0bo7ToCckjK5MkGhjED9KoRcXO6BaGI3I9UIzSA1FKFPOQ==}
js-cookie@3.0.8:
resolution: {integrity: sha512-yeJd4aNAdYZQjaon2bpD/Gb0B/omw7HQOsynXXcOiWVCacbBcPlgn8S/d1X6blFSaHao7ozqtW7NZW19xpCtIw==}
js-levenshtein@1.1.6:
resolution: {integrity: sha512-X2BB11YZtrRqY4EnQcLX5Rh373zbK4alC1FW7D7MBhL2gtcC17cTnr6DmfHZeS0s2rTHjUTMMHfG7gO8SSdw+g==}
@@ -8394,8 +8399,8 @@ packages:
resolution: {integrity: sha512-Pugqs6M0m7Lv1I7FtxN4aoyToKg1C4tu+/381vH35y8oENM/Ai7f7C4StcoK4/+BSw9ebcS8jRiVrORFKCALLw==}
engines: {node: ^20.0.0 || >=22.0.0}
tmp@0.2.4:
resolution: {integrity: sha512-UdiSoX6ypifLmrfQ/XfiawN6hkjSBpCjhKxxZcWlUUmoXLaCKQU0bx4HF/tdDK2uzRuchf1txGvrWBzYREssoQ==}
tmp@0.2.7:
resolution: {integrity: sha512-e0votIpp4Uo2AJYSzVHV6xCcawuiez3DzqDAbrTc3YxBkplN6e+dM13ZeIcZnDg/QpSuU2zfZ3rzwY8ukEnaXw==}
engines: {node: '>=14.14'}
tmpl@1.0.5:
@@ -10318,10 +10323,10 @@ snapshots:
'@shikijs/types': 3.23.0
'@shikijs/vscode-textmate': 10.0.2
'@grafana/data@11.6.14(react-dom@18.2.0(react@18.2.0))(react@18.2.0)':
'@grafana/data@11.6.15(react-dom@18.2.0(react@18.2.0))(react@18.2.0)':
dependencies:
'@braintree/sanitize-url': 7.0.1
'@grafana/schema': 11.6.14
'@grafana/schema': 11.6.15
'@types/d3-interpolate': 3.0.1
'@types/string-hash': 1.1.3
d3-interpolate: 3.0.1
@@ -10347,7 +10352,7 @@ snapshots:
uplot: 1.6.31
xss: 1.0.14
'@grafana/schema@11.6.14':
'@grafana/schema@11.6.15':
dependencies:
tslib: 2.8.1
@@ -12886,7 +12891,7 @@ snapshots:
axios@1.16.0:
dependencies:
follow-redirects: 1.16.0
form-data: 4.0.4
form-data: 4.0.6
proxy-from-env: 2.1.0
transitivePeerDependencies:
- debug
@@ -13833,7 +13838,7 @@ snapshots:
es-errors: 1.3.0
get-intrinsic: 1.3.0
has-tostringtag: 1.0.2
hasown: 2.0.2
hasown: 2.0.4
es-toolkit@1.46.1: {}
@@ -14031,7 +14036,7 @@ snapshots:
dependencies:
chardet: 0.7.0
iconv-lite: 0.4.24
tmp: 0.2.4
tmp: 0.2.7
fast-deep-equal@3.1.3: {}
@@ -14164,12 +14169,12 @@ snapshots:
cross-spawn: 7.0.5
signal-exit: 4.1.0
form-data@4.0.4:
form-data@4.0.6:
dependencies:
asynckit: 0.4.0
combined-stream: 1.0.8
es-set-tostringtag: 2.1.0
hasown: 2.0.2
hasown: 2.0.4
mime-types: 2.1.35
format@0.2.2: {}
@@ -14248,7 +14253,7 @@ snapshots:
get-proto: 1.0.1
gopd: 1.2.0
has-symbols: 1.1.0
hasown: 2.0.2
hasown: 2.0.4
math-intrinsics: 1.1.0
get-nonce@1.0.1: {}
@@ -14386,6 +14391,10 @@ snapshots:
dependencies:
function-bind: 1.1.2
hasown@2.0.4:
dependencies:
function-bind: 1.1.2
hast-util-from-parse5@8.0.1:
dependencies:
'@types/hast': 3.0.4
@@ -14506,10 +14515,10 @@ snapshots:
transitivePeerDependencies:
- supports-color
http-proxy-middleware@4.0.0:
http-proxy-middleware@4.1.1:
dependencies:
debug: 4.3.4(supports-color@5.5.0)
httpxy: 0.5.1
httpxy: 0.5.3
is-glob: 4.0.3
is-plain-obj: 4.1.0
micromatch: 4.0.8
@@ -14525,7 +14534,7 @@ snapshots:
transitivePeerDependencies:
- supports-color
httpxy@0.5.1: {}
httpxy@0.5.3: {}
human-signals@2.1.0: {}
@@ -15339,7 +15348,7 @@ snapshots:
js-base64@3.7.5: {}
js-cookie@2.2.1: {}
js-cookie@3.0.8: {}
js-levenshtein@1.1.6: {}
@@ -15367,7 +15376,7 @@ snapshots:
decimal.js: 10.6.0
domexception: 4.0.0
escodegen: 2.1.0
form-data: 4.0.4
form-data: 4.0.6
html-encoding-sniffer: 3.0.0
http-proxy-agent: 5.0.0
https-proxy-agent: 5.0.1
@@ -17336,7 +17345,7 @@ snapshots:
copy-to-clipboard: 3.3.3
fast-deep-equal: 3.1.3
fast-shallow-equal: 1.0.0
js-cookie: 2.2.1
js-cookie: 3.0.8
nano-css: 5.6.2(react-dom@18.2.0(react@18.2.0))(react@18.2.0)
react: 18.2.0
react-dom: 18.2.0(react@18.2.0)
@@ -17355,7 +17364,7 @@ snapshots:
copy-to-clipboard: 3.3.3
fast-deep-equal: 3.1.3
fast-shallow-equal: 1.0.0
js-cookie: 2.2.1
js-cookie: 3.0.8
nano-css: 5.6.2(react-dom@18.2.0(react@18.2.0))(react@18.2.0)
react: 18.2.0
react-dom: 18.2.0(react@18.2.0)
@@ -18103,7 +18112,7 @@ snapshots:
tinypool@2.1.0: {}
tmp@0.2.4: {}
tmp@0.2.7: {}
tmpl@1.0.5: {}

View File

@@ -330,3 +330,10 @@ export const AIAssistantPage = Loadable(
/* webpackChunkName: "AI Assistant Page" */ 'pages/AIAssistantPage/AIAssistantPage'
),
);
export const LLMObservabilityAttributeMappingPage = Loadable(
() =>
import(
/* webpackChunkName: "LLM Observability Attribute Mapping Page" */ 'pages/LLMObservabilityAttributeMapping'
),
);

View File

@@ -22,6 +22,7 @@ import {
IntegrationsDetailsPage,
LicensePage,
ListAllALertsPage,
LLMObservabilityAttributeMappingPage,
LiveLogs,
Login,
Logs,
@@ -505,6 +506,13 @@ const routes: AppRoutes[] = [
key: 'AI_ASSISTANT',
isPrivate: true,
},
{
path: ROUTES.LLM_OBSERVABILITY_ATTRIBUTE_MAPPING,
exact: true,
component: LLMObservabilityAttributeMappingPage,
key: 'LLM_OBSERVABILITY_ATTRIBUTE_MAPPING',
isPrivate: true,
},
];
export const SUPPORT_ROUTE: AppRoutes = {

View File

@@ -8135,6 +8135,44 @@ export interface SpantypesGettableSpanMapperGroupsDTO {
items: SpantypesSpanMapperGroupDTO[];
}
export type SpantypesSpanMapperTestSpanDTOAttributesAnyOf = {
[key: string]: unknown;
};
/**
* @nullable
*/
export type SpantypesSpanMapperTestSpanDTOAttributes =
SpantypesSpanMapperTestSpanDTOAttributesAnyOf | null;
export type SpantypesSpanMapperTestSpanDTOResourceAnyOf = {
[key: string]: unknown;
};
/**
* @nullable
*/
export type SpantypesSpanMapperTestSpanDTOResource =
SpantypesSpanMapperTestSpanDTOResourceAnyOf | null;
export interface SpantypesSpanMapperTestSpanDTO {
/**
* @type object,null
*/
attributes?: SpantypesSpanMapperTestSpanDTOAttributes;
/**
* @type object,null
*/
resource?: SpantypesSpanMapperTestSpanDTOResource;
}
export interface SpantypesGettableSpanMapperTestDTO {
/**
* @type array,null
*/
spans: SpantypesSpanMapperTestSpanDTO[] | null;
}
export enum SpantypesSpanAggregationTypeDTO {
span_count = 'span_count',
execution_time_percentage = 'execution_time_percentage',
@@ -8430,6 +8468,33 @@ export interface SpantypesPostableSpanMapperGroupDTO {
name: string;
}
export interface SpantypesPostableSpanMapperTestGroupDTO {
condition: SpantypesSpanMapperGroupConditionDTO | null;
/**
* @type boolean
*/
enabled?: boolean;
/**
* @type array,null
*/
mappers?: SpantypesPostableSpanMapperDTO[] | null;
/**
* @type string
*/
name: string;
}
export interface SpantypesPostableSpanMapperTestDTO {
/**
* @type array,null
*/
groups: SpantypesPostableSpanMapperTestGroupDTO[] | null;
/**
* @type array,null
*/
spans: SpantypesSpanMapperTestSpanDTO[] | null;
}
export interface SpantypesSpanAggregationDTO {
aggregation: SpantypesSpanAggregationTypeDTO;
field: TelemetrytypesTelemetryFieldKeyDTO;
@@ -9798,6 +9863,14 @@ export type UpdateSpanMapperPathParameters = {
groupId: string;
mapperId: string;
};
export type TestSpanMappers200 = {
data: SpantypesGettableSpanMapperTestDTO;
/**
* @type string
*/
status: string;
};
export type GetStats200Data = { [key: string]: unknown };
export type GetStats200 = {

View File

@@ -30,8 +30,10 @@ import type {
RenderErrorResponseDTO,
SpantypesPostableSpanMapperDTO,
SpantypesPostableSpanMapperGroupDTO,
SpantypesPostableSpanMapperTestDTO,
SpantypesUpdatableSpanMapperDTO,
SpantypesUpdatableSpanMapperGroupDTO,
TestSpanMappers200,
UpdateSpanMapperGroupPathParameters,
UpdateSpanMapperPathParameters,
} from '../sigNoz.schemas';
@@ -780,3 +782,86 @@ export const useUpdateSpanMapper = <
> => {
return useMutation(getUpdateSpanMapperMutationOptions(options));
};
/**
* Tests how span mappers would transform sample spans
* @summary Test span mappers against sample spans
*/
export const testSpanMappers = (
spantypesPostableSpanMapperTestDTO?: BodyType<SpantypesPostableSpanMapperTestDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<TestSpanMappers200>({
url: `/api/v1/span_mapper_groups/test`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: spantypesPostableSpanMapperTestDTO,
signal,
});
};
export const getTestSpanMappersMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof testSpanMappers>>,
TError,
{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof testSpanMappers>>,
TError,
{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
TContext
> => {
const mutationKey = ['testSpanMappers'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof testSpanMappers>>,
{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> }
> = (props) => {
const { data } = props ?? {};
return testSpanMappers(data);
};
return { mutationFn, ...mutationOptions };
};
export type TestSpanMappersMutationResult = NonNullable<
Awaited<ReturnType<typeof testSpanMappers>>
>;
export type TestSpanMappersMutationBody =
| BodyType<SpantypesPostableSpanMapperTestDTO>
| undefined;
export type TestSpanMappersMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Test span mappers against sample spans
*/
export const useTestSpanMappers = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof testSpanMappers>>,
TError,
{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof testSpanMappers>>,
TError,
{ data?: BodyType<SpantypesPostableSpanMapperTestDTO> },
TContext
> => {
return useMutation(getTestSpanMappersMutationOptions(options));
};

View File

@@ -91,6 +91,8 @@ const ROUTES = {
AI_ASSISTANT_BASE: '/ai-assistant',
AI_ASSISTANT_ICON_PREVIEW: '/ai-assistant-icon-preview',
MCP_SERVER: '/settings/mcp-server',
LLM_OBSERVABILITY_ATTRIBUTE_MAPPING:
'/llm-observability/settings/attribute-mapping',
} as const;
export default ROUTES;

View File

@@ -0,0 +1,56 @@
import { Button } from '@signozhq/ui/button';
import styles from './LLMObservabilityAttributeMapping.module.scss';
interface AttributeMappingHeaderProps {
isDirty: boolean;
isSaving: boolean;
onDiscard: () => void;
onSave: () => void;
}
function AttributeMappingHeader({
isDirty,
isSaving,
onDiscard,
onSave,
}: AttributeMappingHeaderProps): JSX.Element {
return (
<header className={styles.pageHeader}>
<div className={styles.pageHeaderTitle}>
<h1 className={styles.title}>Attribute Mapping</h1>
<p className={styles.description}>
Configure source-to-target attribute remapping for LLM traces
</p>
</div>
<div className={styles.pageHeaderActions}>
{isDirty && (
<span className={styles.unsavedChanges} data-testid="unsaved-changes">
Unsaved changes
</span>
)}
<Button
variant="outlined"
color="secondary"
onClick={onDiscard}
disabled={!isDirty || isSaving}
testId="discard-changes-btn"
>
Discard
</Button>
<Button
variant="solid"
color="primary"
onClick={onSave}
loading={isSaving}
disabled={!isDirty || isSaving}
testId="save-changes-btn"
>
{isSaving ? 'Saving…' : 'Save changes'}
</Button>
</div>
</header>
);
}
export default AttributeMappingHeader;

View File

@@ -0,0 +1,41 @@
import styles from './LLMObservabilityAttributeMapping.module.scss';
import MapperGroupsTable from './MapperGroupsTable';
import { DraftGroup } from './types';
import { AttributeMappingStore } from './useAttributeMappingStore';
interface AttributeMappingsTabProps {
store: AttributeMappingStore;
onEditGroup: (group: DraftGroup) => void;
onAddGroup: () => void;
}
// "Attribute mappings" tab: the mapping-groups listing, its error state and
// footer summary. The store is owned by the container (the header's save/
// discard share it), so it's passed in rather than created here.
function AttributeMappingsTab({
store,
onEditGroup,
onAddGroup,
}: AttributeMappingsTabProps): JSX.Element {
return (
<div data-testid="attribute-mappings-tab">
{store.isError && (
<div className={styles.pageError} role="alert">
Failed to load mapping groups. Please try again.
</div>
)}
<MapperGroupsTable
store={store}
onEditGroup={onEditGroup}
onAddGroup={onAddGroup}
/>
<footer className={styles.pageFooter}>
Showing {store.groups.length} group{store.groups.length === 1 ? '' : 's'}
</footer>
</div>
);
}
export default AttributeMappingsTab;

View File

@@ -0,0 +1,93 @@
import { Button } from '@signozhq/ui/button';
import { Plus, X } from '@signozhq/icons';
import styles from './GroupFormDrawer.module.scss';
import KeySearchInput from './KeySearchInput';
import { FieldContextValue } from './types';
interface ConditionKeyListProps {
label: string;
labelHint?: string;
keys: string[];
placeholder: string;
addLabel: string;
testIdPrefix: string;
fieldContext: FieldContextValue;
onChange: (keys: string[]) => void;
}
// Editor for one list of condition keys (the group's span-attribute or
// resource gating keys). Substring "contains" match, order irrelevant.
function ConditionKeyList({
label,
labelHint,
keys,
placeholder,
addLabel,
testIdPrefix,
fieldContext,
onChange,
}: ConditionKeyListProps): JSX.Element {
const updateKey = (index: number, value: string): void => {
onChange(keys.map((key, i) => (i === index ? value : key)));
};
const addKey = (): void => {
onChange([...keys, '']);
};
const removeKey = (index: number): void => {
onChange(keys.filter((_, i) => i !== index));
};
return (
<div className={styles.groupFormField}>
<span className={styles.groupFormLabel}>
{label}
{labelHint && (
<span className={styles.groupFormLabelHint}> {labelHint}</span>
)}
</span>
{keys.length > 0 && (
<div className={styles.groupFormKeys}>
{keys.map((key, index) => (
// eslint-disable-next-line react/no-array-index-key
<div className={styles.groupFormKey} key={index}>
<KeySearchInput
className={styles.groupFormKeyInput}
placeholder={placeholder}
value={key}
fieldContext={fieldContext}
onChange={(next): void => updateKey(index, next)}
testId={`${testIdPrefix}-${index}`}
/>
<Button
variant="ghost"
color="secondary"
size="icon"
aria-label="Remove key"
onClick={(): void => removeKey(index)}
testId={`${testIdPrefix}-remove-${index}`}
>
<X size={14} />
</Button>
</div>
))}
</div>
)}
<Button
variant="dashed"
color="secondary"
prefix={<Plus size={14} />}
onClick={addKey}
testId={`${testIdPrefix}-add`}
>
{addLabel}
</Button>
</div>
);
}
export default ConditionKeyList;

View File

@@ -0,0 +1,76 @@
.groupForm {
display: flex;
flex-direction: column;
gap: var(--spacing-10);
padding: var(--spacing-2) 0;
}
.groupFormField {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
}
.groupFormFieldRow {
flex-direction: row;
align-items: center;
justify-content: space-between;
}
.groupFormLabel {
font-size: var(--font-size-xs);
font-weight: var(--font-weight-semibold);
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--l3-foreground);
}
.groupFormLabelHint {
font-weight: var(--font-weight-normal);
text-transform: none;
letter-spacing: normal;
}
.groupFormHint {
font-size: var(--font-size-xs);
color: var(--l3-foreground);
}
.groupFormKeys {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
}
.groupFormKey {
display: flex;
align-items: center;
gap: var(--spacing-3);
}
.groupFormKeyInput {
flex: 1;
font-family: 'Geist Mono', monospace;
}
.groupFormError {
padding: var(--spacing-5) var(--spacing-6);
border-radius: var(--radius-2);
background: var(--callout-error-background);
color: var(--callout-error-title);
font-size: var(--periscope-font-size-base);
}
.groupFormFooter {
display: flex;
align-items: center;
justify-content: space-between;
width: 100%;
gap: var(--spacing-4);
}
.groupFormFooterActions {
display: flex;
gap: var(--spacing-4);
margin-left: auto;
}

View File

@@ -0,0 +1,147 @@
import { Button } from '@signozhq/ui/button';
import { DrawerWrapper } from '@signozhq/ui/drawer';
import { Input } from '@signozhq/ui/input';
import { Switch } from '@signozhq/ui/switch';
import { Trash2 } from '@signozhq/icons';
import ConditionKeyList from './ConditionKeyList';
import styles from './GroupFormDrawer.module.scss';
import { FieldContext, GroupDraft, MapperDraftMode } from './types';
import { isGroupDraftValid } from './utils';
interface GroupFormDrawerProps {
isOpen: boolean;
mode: MapperDraftMode;
draft: GroupDraft;
setDraft: (next: GroupDraft) => void;
onClose: () => void;
onSave: () => void;
onDelete: () => void;
isSaving: boolean;
isDeleting: boolean;
saveError: string | null;
}
function GroupFormDrawer({
isOpen,
mode,
draft,
setDraft,
onClose,
onSave,
onDelete,
isSaving,
isDeleting,
saveError,
}: GroupFormDrawerProps): JSX.Element {
const isEdit = mode === 'edit';
const isValid = isGroupDraftValid(draft);
return (
<DrawerWrapper
open={isOpen}
onOpenChange={(open): void => {
if (!open) {
onClose();
}
}}
title={isEdit ? 'Edit group' : 'New group'}
subTitle="A group gates which spans its mappings run on"
width="wide"
testId="group-form-drawer"
footer={
<div className={styles.groupFormFooter}>
{isEdit && (
<Button
variant="ghost"
color="destructive"
prefix={<Trash2 size={14} />}
onClick={onDelete}
disabled={isDeleting}
testId="group-form-delete"
>
{isDeleting ? 'Deleting…' : 'Delete'}
</Button>
)}
<div className={styles.groupFormFooterActions}>
<Button
variant="ghost"
color="secondary"
onClick={onClose}
testId="group-form-cancel"
>
Cancel
</Button>
<Button
variant="solid"
color="primary"
onClick={onSave}
disabled={!isValid || isSaving}
testId="group-form-save"
>
{/* eslint-disable-next-line no-nested-ternary */}
{isSaving ? 'Saving…' : isEdit ? 'Save group' : 'Create group'}
</Button>
</div>
</div>
}
>
<div className={styles.groupForm}>
<div className={styles.groupFormField}>
<span className={styles.groupFormLabel}>Group name</span>
<Input
placeholder="e.g. OpenAI gateway"
value={draft.name}
onChange={(event): void =>
setDraft({ ...draft, name: event.target.value })
}
testId="group-form-name"
/>
</div>
<div className={`${styles.groupFormField} ${styles.groupFormFieldRow}`}>
<span className={styles.groupFormLabel}>Enabled</span>
<Switch
value={draft.enabled}
onChange={(checked): void => setDraft({ ...draft, enabled: checked })}
testId="group-form-enabled"
/>
</div>
<ConditionKeyList
label="Condition · span attribute keys"
labelHint="· runs when a span attribute key contains any of these"
keys={draft.attributes}
placeholder="e.g. gen_ai."
addLabel="Add attribute key"
testIdPrefix="group-form-attribute"
fieldContext={FieldContext.attribute}
onChange={(attributes): void => setDraft({ ...draft, attributes })}
/>
<ConditionKeyList
label="Condition · resource keys"
labelHint="· or when a resource key contains any of these"
keys={draft.resource}
placeholder="e.g. service.name"
addLabel="Add resource key"
testIdPrefix="group-form-resource"
fieldContext={FieldContext.resource}
onChange={(resource): void => setDraft({ ...draft, resource })}
/>
<span className={styles.groupFormHint}>
Leave both empty to run this group on every span.
</span>
{saveError && (
<div className={styles.groupFormError} role="alert">
{saveError}
</div>
)}
</div>
</DrawerWrapper>
);
}
export default GroupFormDrawer;

View File

@@ -0,0 +1,51 @@
.keySearch {
position: relative;
flex: 1;
min-width: 0;
}
.keySearchDropdown {
position: absolute;
top: calc(100% + var(--spacing-2));
left: 0;
z-index: 20;
min-width: 100%;
width: max-content;
max-width: 420px;
max-height: 240px;
overflow-x: hidden;
overflow-y: auto;
padding: var(--spacing-2);
border: 1px solid var(--l2-border);
border-radius: 6px;
background: var(--l2-background);
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.4);
}
.keySearchDropdownEmpty {
padding: var(--spacing-4) var(--spacing-5);
font-size: var(--font-size-xs);
color: var(--l3-foreground);
}
.keySearchOption {
display: block;
width: 100%;
max-width: 100%;
padding: var(--spacing-3) var(--spacing-5);
border: none;
border-radius: 3px;
background: transparent;
color: var(--l1-foreground);
font-family: 'Geist Mono', monospace;
font-size: var(--font-size-xs);
text-align: left;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
cursor: pointer;
&:hover {
background: var(--l2-background-hover);
}
}

View File

@@ -0,0 +1,117 @@
import { useMemo, useState } from 'react';
import { Input } from '@signozhq/ui/input';
import { useGetFieldsKeys } from 'api/generated/services/fields';
import {
TelemetrytypesFieldContextDTO,
TelemetrytypesSignalDTO,
} from 'api/generated/services/sigNoz.schemas';
import cx from 'classnames';
import Spinner from 'components/Spinner';
import useDebounce from 'hooks/useDebounce';
import styles from './KeySearchInput.module.scss';
import { FieldContext, FieldContextValue } from './types';
const SUGGESTION_LIMIT = 50;
const DEBOUNCE_MS = 300;
interface KeySearchInputProps {
value: string;
fieldContext: FieldContextValue;
placeholder?: string;
className?: string;
disabled?: boolean;
testId?: string;
onChange: (value: string) => void;
}
// Maps the mapper's attribute/resource context to the fields-endpoint context.
function toFieldsContext(
context: FieldContextValue,
): TelemetrytypesFieldContextDTO {
return context === FieldContext.resource
? TelemetrytypesFieldContextDTO.resource
: TelemetrytypesFieldContextDTO.attribute;
}
// Free-text input with span/resource key suggestions from /api/v1/fields/keys
// (signal=traces). Typing keeps the custom value; suggestions are assistive.
function KeySearchInput({
value,
fieldContext,
placeholder,
className,
disabled,
testId,
onChange,
}: KeySearchInputProps): JSX.Element {
const [isOpen, setIsOpen] = useState(false);
const debouncedSearch = useDebounce(value, DEBOUNCE_MS);
const { data, isFetching } = useGetFieldsKeys(
{
signal: TelemetrytypesSignalDTO.traces,
fieldContext: toFieldsContext(fieldContext),
searchText: debouncedSearch,
limit: SUGGESTION_LIMIT,
},
{ query: { enabled: isOpen && !disabled, keepPreviousData: true } },
);
const suggestions = useMemo(() => {
const keys = data?.data?.keys ?? {};
return Object.keys(keys)
.filter((key) => key !== value)
.slice(0, SUGGESTION_LIMIT);
}, [data, value]);
return (
<div className={cx(styles.keySearch, className)}>
<Input
placeholder={placeholder}
value={value}
disabled={disabled}
autoComplete="off"
onChange={(event): void => {
onChange(event.target.value);
setIsOpen(true);
}}
onFocus={(): void => setIsOpen(true)}
onBlur={(): void => setIsOpen(false)}
testId={testId}
/>
{isOpen && suggestions.length > 0 && (
<div
className={styles.keySearchDropdown}
data-testid={`${testId}-dropdown`}
>
{suggestions.map((name) => (
<button
type="button"
key={name}
className={styles.keySearchOption}
// onMouseDown (not onClick) so selection runs before the input blur.
onMouseDown={(event): void => {
event.preventDefault();
onChange(name);
setIsOpen(false);
}}
data-testid={`${testId}-option-${name}`}
>
{name}
</button>
))}
</div>
)}
{isOpen && isFetching && suggestions.length === 0 && (
<div
className={cx(styles.keySearchDropdown, styles.keySearchDropdownEmpty)}
>
<Spinner size="small" height="auto" />
</div>
)}
</div>
);
}
export default KeySearchInput;

View File

@@ -0,0 +1,161 @@
.llmObservabilityAttributeMapping {
display: flex;
flex-direction: column;
gap: var(--spacing-8);
padding: var(--spacing-12);
}
.pageHeader {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: var(--spacing-8);
}
.pageHeaderTitle {
display: flex;
flex-direction: column;
}
.title {
margin: 0;
font-size: var(--periscope-font-size-large);
font-weight: var(--font-weight-semibold);
}
.description {
margin: var(--spacing-2) 0 0;
font-size: var(--periscope-font-size-base);
color: var(--l3-foreground);
}
.pageHeaderActions {
display: flex;
align-items: center;
gap: var(--spacing-6);
}
.unsavedChanges {
font-size: var(--periscope-font-size-base);
color: var(--accent-amber);
}
.tableEmpty {
padding: var(--spacing-12) var(--spacing-6);
text-align: center;
color: var(--l3-foreground);
font-size: var(--periscope-font-size-base);
}
.muted {
color: var(--l3-foreground);
}
.pageError {
padding: var(--padding-3) var(--padding-4);
border-radius: var(--radius-2);
background: var(--callout-error-background);
color: var(--callout-error-title);
font-size: var(--periscope-font-size-base);
}
.pageFooter {
margin-top: var(--spacing-8);
font-size: var(--font-size-xs);
color: var(--l3-foreground);
}
.rowActions {
display: flex;
align-items: center;
gap: var(--spacing-3);
}
.addRow {
align-self: flex-start;
}
.groupsTableWrapper {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
}
.groupsTableNameCell {
display: flex;
align-items: center;
gap: var(--spacing-4);
}
.groupsTableName {
font-weight: var(--font-weight-semibold);
}
.groupsTableFilters {
display: flex;
flex-direction: column;
gap: var(--spacing-3);
// Allow this column to shrink within the cell so long keys wrap
// instead of forcing the cell wider than its allotted width.
min-width: 0;
}
.groupsTableFilter {
display: flex;
align-items: flex-start;
gap: var(--spacing-4);
font-size: var(--periscope-font-size-base);
min-width: 0;
// Keep the context badge at full width; only the key text flexes.
> *:first-child {
flex-shrink: 0;
}
}
.groupsTableFilterKey {
min-width: 0;
font-family: 'Geist Mono', monospace;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.mappersTableWrapper {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
margin: var(--margin-1) 0;
}
.mappersTableTarget {
font-family: 'Geist Mono', monospace;
font-weight: var(--font-weight-semibold);
}
.mappersTableSources {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: var(--spacing-3);
}
.mappersTableSourceChip {
display: inline-flex;
align-items: center;
max-width: 220px;
padding: 2px var(--padding-2);
border: 1px solid var(--l2-border);
border-radius: 6px;
background: var(--l3-background);
font-family: 'Geist Mono', monospace;
font-size: var(--font-size-xs);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.mappersTableSourceMore {
font-size: var(--font-size-xs);
white-space: nowrap;
}

View File

@@ -0,0 +1,88 @@
import { useCallback } from 'react';
import { Tabs } from '@signozhq/ui/tabs';
import AttributeMappingHeader from './AttributeMappingHeader';
import AttributeMappingsTab from './AttributeMappingsTab';
import GroupFormDrawer from './GroupFormDrawer';
import styles from './LLMObservabilityAttributeMapping.module.scss';
import TestTab from './TestTab';
import { useAttributeMappingStore } from './useAttributeMappingStore';
import { useGroupFormDrawer } from './useGroupFormDrawer';
function LLMObservabilityAttributeMapping(): JSX.Element {
const store = useAttributeMappingStore();
const groupDrawer = useGroupFormDrawer();
const handleGroupSave = useCallback((): void => {
store.upsertGroup(groupDrawer.draft);
groupDrawer.close();
}, [store, groupDrawer]);
const handleGroupDelete = useCallback((): void => {
if (groupDrawer.draft.id) {
store.removeGroup(groupDrawer.draft.id);
}
groupDrawer.close();
}, [store, groupDrawer]);
const tabItems = [
{
key: 'attribute-mappings',
label: 'Attribute mappings',
children: (
<AttributeMappingsTab
store={store}
onEditGroup={groupDrawer.openForEdit}
onAddGroup={groupDrawer.openForAdd}
/>
),
},
{
key: 'test',
label: 'Test',
children: <TestTab store={store} />,
},
];
return (
<div
className={styles.llmObservabilityAttributeMapping}
data-testid="llm-observability-attribute-mapping-page"
>
<AttributeMappingHeader
isDirty={store.isDirty}
isSaving={store.isSaving}
onDiscard={store.discard}
onSave={store.save}
/>
{store.saveError && (
<div className={styles.pageError} role="alert">
{store.saveError}
</div>
)}
<Tabs
testId="attribute-mapping-tabs"
defaultValue="attribute-mappings"
items={tabItems}
/>
{groupDrawer.isOpen && (
<GroupFormDrawer
isOpen={groupDrawer.isOpen}
mode={groupDrawer.mode}
draft={groupDrawer.draft}
setDraft={groupDrawer.setDraft}
onClose={groupDrawer.close}
onSave={handleGroupSave}
onDelete={handleGroupDelete}
isSaving={false}
isDeleting={false}
saveError={null}
/>
)}
</div>
);
}
export default LLMObservabilityAttributeMapping;

View File

@@ -0,0 +1,104 @@
.form {
display: flex;
flex-direction: column;
gap: 20px;
padding: 4px 0;
}
.field {
display: flex;
flex-direction: column;
gap: 8px;
}
.label {
font-size: 12px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--bg-vanilla-400);
}
.labelHint {
font-weight: 400;
text-transform: none;
letter-spacing: normal;
}
.hint {
font-size: 12px;
color: var(--bg-vanilla-400);
}
.fieldContext {
width: 200px;
}
.sources {
display: flex;
flex-direction: column;
gap: 8px;
}
.source {
display: flex;
align-items: center;
gap: 6px;
}
.sourceHandle {
display: inline-flex;
align-items: center;
justify-content: center;
padding: 4px;
border: none;
background: transparent;
color: var(--bg-vanilla-400);
cursor: grab;
touch-action: none;
user-select: none;
&:active {
cursor: grabbing;
}
}
.sourceIndex {
width: 20px;
text-align: center;
font-size: 12px;
color: var(--bg-vanilla-400);
}
.sourceInput {
flex: 1;
min-width: 0;
font-family: 'Geist Mono', monospace;
}
.sourceSelect {
flex: 0 0 auto;
width: 120px;
}
.error {
padding: 10px 12px;
border-radius: 4px;
background: var(--callout-error-background);
color: var(--callout-error-title);
font-size: 13px;
}
.footer {
display: flex;
align-items: center;
justify-content: space-between;
width: 100%;
gap: 8px;
}
.footerActions {
display: flex;
gap: 8px;
margin-left: auto;
}

View File

@@ -0,0 +1,257 @@
import { useState } from 'react';
import { Button } from '@signozhq/ui/button';
import { DrawerWrapper } from '@signozhq/ui/drawer';
import { SelectSimple } from '@signozhq/ui/select';
import {
closestCenter,
DndContext,
DragEndEvent,
PointerSensor,
useSensor,
useSensors,
} from '@dnd-kit/core';
import { restrictToVerticalAxis } from '@dnd-kit/modifiers';
import {
arrayMove,
SortableContext,
verticalListSortingStrategy,
} from '@dnd-kit/sortable';
import { Plus, Trash2 } from '@signozhq/icons';
import { v4 as uuid } from 'uuid';
import KeySearchInput from './KeySearchInput';
import styles from './MapperFormDrawer.module.scss';
import SourceAttributeRow from './SourceAttributeRow';
import {
FieldContext,
FieldContextValue,
MapperDraft,
MapperDraftMode,
SourceConfig,
} from './types';
import { createEmptySource, isMapperDraftValid } from './utils';
const FIELD_CONTEXT_OPTIONS = [
{ value: FieldContext.attribute, label: 'Span attribute' },
{ value: FieldContext.resource, label: 'Resource' },
];
interface MapperFormDrawerProps {
isOpen: boolean;
mode: MapperDraftMode;
draft: MapperDraft;
setDraft: (next: MapperDraft) => void;
onClose: () => void;
onSave: () => void;
onDelete: () => void;
isSaving: boolean;
isDeleting: boolean;
saveError: string | null;
}
function MapperFormDrawer({
isOpen,
mode,
draft,
setDraft,
onClose,
onSave,
onDelete,
isSaving,
isDeleting,
saveError,
}: MapperFormDrawerProps): JSX.Element {
const isEdit = mode === 'edit';
const isValid = isMapperDraftValid(draft);
// Stable per-row ids for the sortable list. These are UI-only (never sent to
// the API and excluded from the draft), so dnd-kit can track rows reliably
// even though sources are stored as a plain array. Re-seeded each time the
// drawer opens; kept in lockstep with the sources array on add/remove/drag.
const [rowIds, setRowIds] = useState<string[]>(() =>
draft.sources.map(() => uuid()),
);
const sourceIds = draft.sources.map(
(_, index) => rowIds[index] ?? `pending-${index}`,
);
// 5px activation distance so clicking into the input never starts a drag.
const sensors = useSensors(
useSensor(PointerSensor, { activationConstraint: { distance: 5 } }),
);
const updateSource = (index: number, patch: Partial<SourceConfig>): void => {
const sources = draft.sources.map((source, i) =>
i === index ? { ...source, ...patch } : source,
);
setDraft({ ...draft, sources });
};
const addSource = (): void => {
setDraft({ ...draft, sources: [...draft.sources, createEmptySource()] });
setRowIds((prev) => [...prev, uuid()]);
};
const removeSource = (index: number): void => {
const sources = draft.sources.filter((_, i) => i !== index);
if (sources.length === 0) {
setDraft({ ...draft, sources: [createEmptySource()] });
setRowIds([uuid()]);
return;
}
setDraft({ ...draft, sources });
setRowIds((prev) => prev.filter((_, i) => i !== index));
};
const handleDragEnd = (event: DragEndEvent): void => {
const { active, over } = event;
if (!over || active.id === over.id) {
return;
}
const from = sourceIds.indexOf(String(active.id));
const to = sourceIds.indexOf(String(over.id));
if (from === -1 || to === -1) {
return;
}
setDraft({ ...draft, sources: arrayMove(draft.sources, from, to) });
setRowIds((prev) => arrayMove(prev, from, to));
};
return (
<DrawerWrapper
open={isOpen}
onOpenChange={(open): void => {
if (!open) {
onClose();
}
}}
title={isEdit ? 'Edit mapping' : 'New custom mapping'}
subTitle="Map source attributes onto a canonical target attribute"
width="wide"
testId="mapper-form-drawer"
footer={
<div className={styles.footer}>
{isEdit && (
<Button
variant="ghost"
color="destructive"
prefix={<Trash2 size={14} />}
onClick={onDelete}
disabled={isDeleting}
testId="mapper-form-delete"
>
{isDeleting ? 'Deleting…' : 'Delete'}
</Button>
)}
<div className={styles.footerActions}>
<Button
variant="ghost"
color="secondary"
onClick={onClose}
testId="mapper-form-cancel"
>
Cancel
</Button>
<Button
variant="solid"
color="primary"
onClick={onSave}
disabled={!isValid || isSaving}
testId="mapper-form-save"
>
{/* eslint-disable-next-line no-nested-ternary */}
{isSaving ? 'Saving…' : isEdit ? 'Save mapping' : 'Create mapping'}
</Button>
</div>
</div>
}
>
<div className={styles.form}>
<div className={styles.field}>
<span className={styles.label}>Target attribute</span>
<KeySearchInput
placeholder="e.g. gen_ai.content.prompt"
value={draft.name}
fieldContext={draft.fieldContext}
disabled={isEdit}
onChange={(name): void => setDraft({ ...draft, name })}
testId="mapper-form-target"
/>
{isEdit && (
<span className={styles.hint}>
The target attribute can&apos;t be changed after creation.
</span>
)}
</div>
<div className={styles.field}>
<span className={styles.label}>Write target to</span>
<SelectSimple
className={styles.fieldContext}
items={FIELD_CONTEXT_OPTIONS}
value={draft.fieldContext}
withPortal={false}
onChange={(next): void =>
setDraft({ ...draft, fieldContext: next as FieldContextValue })
}
testId="mapper-form-field-context"
/>
<span className={styles.hint}>
Where the standardized attribute is written.
</span>
</div>
<div className={styles.field}>
<span className={styles.label}>
Source attributes
<span className={styles.labelHint}>
{' '}
· priority: top bottom · drag to reorder
</span>
</span>
<DndContext
sensors={sensors}
collisionDetection={closestCenter}
modifiers={[restrictToVerticalAxis]}
onDragEnd={handleDragEnd}
>
<SortableContext items={sourceIds} strategy={verticalListSortingStrategy}>
<div className={styles.sources}>
{draft.sources.map((source, index) => (
<SourceAttributeRow
key={sourceIds[index]}
id={sourceIds[index]}
index={index}
value={source}
canRemove={draft.sources.length > 1}
onChange={updateSource}
onRemove={removeSource}
/>
))}
</div>
</SortableContext>
</DndContext>
<Button
variant="dashed"
color="secondary"
prefix={<Plus size={14} />}
onClick={addSource}
testId="mapper-form-add-source"
>
Add another source
</Button>
</div>
{saveError && (
<div className={styles.error} role="alert">
{saveError}
</div>
)}
</div>
</DrawerWrapper>
);
}
export default MapperFormDrawer;

View File

@@ -0,0 +1,80 @@
import { useMemo } from 'react';
import { Button } from '@signozhq/ui/button';
import { Plus } from '@signozhq/icons';
import TanStackTable from 'components/TanStackTableView';
import styles from './LLMObservabilityAttributeMapping.module.scss';
import { getMapperGroupsColumns } from './mapperGroups.config';
import MappersTable from './MappersTable';
import { DraftGroup } from './types';
import { AttributeMappingStore } from './useAttributeMappingStore';
const SKELETON_ROW_COUNT = 5;
interface MapperGroupsTableProps {
store: AttributeMappingStore;
onEditGroup: (group: DraftGroup) => void;
onAddGroup: () => void;
}
// Top-level listing of mapping groups. Each row expands to reveal its mappers,
// which MappersTable fetches lazily on first open. Built on the shared
// TanStackTable with virtual scroll disabled — this is a small, content-height
// list, and nested expanded tables need to grow with their content rather than
// live inside a fixed-height viewport. Row actions (edit/delete/toggle) live in
// the column config; the add-group affordance sits below the table.
function MapperGroupsTable({
store,
onEditGroup,
onAddGroup,
}: MapperGroupsTableProps): JSX.Element {
const columns = useMemo(
() =>
getMapperGroupsColumns({
onEditGroup,
onRemoveGroup: store.removeGroup,
onToggleGroup: store.toggleGroup,
}),
[onEditGroup, store.removeGroup, store.toggleGroup],
);
const isEmpty = !store.isLoading && store.groups.length === 0;
return (
<div className={styles.groupsTableWrapper}>
{isEmpty ? (
<div className={styles.tableEmpty} data-testid="mapper-groups-empty">
No mapping groups yet.
</div>
) : (
<TanStackTable<DraftGroup>
data={store.groups}
columns={columns}
isLoading={store.isLoading}
skeletonRowCount={SKELETON_ROW_COUNT}
getRowKey={(row): string => row.localId}
getRowCanExpand={(): boolean => true}
renderExpandedRow={(row): JSX.Element => (
<MappersTable group={row} store={store} />
)}
disableVirtualScroll
testId="mapper-groups-table"
/>
)}
<Button
variant="ghost"
color="primary"
prefix={<Plus size={14} />}
className={styles.addRow}
onClick={onAddGroup}
testId="add-group-row"
disabled={store.isLoading}
>
Add a new group
</Button>
</div>
);
}
export default MapperGroupsTable;

View File

@@ -0,0 +1,138 @@
import { useCallback, useEffect, useMemo } from 'react';
import { Button } from '@signozhq/ui/button';
import { Plus } from '@signozhq/icons';
import { useListSpanMappers } from 'api/generated/services/spanmapper';
import TanStackTable from 'components/TanStackTableView';
import styles from './LLMObservabilityAttributeMapping.module.scss';
import MapperFormDrawer from './MapperFormDrawer';
import { getMappersColumns } from './mappers.config';
import { DraftGroup, DraftMapper, Mapper } from './types';
import { AttributeMappingStore } from './useAttributeMappingStore';
import { useMapperFormDrawer } from './useMapperFormDrawer';
const SKELETON_ROW_COUNT = 3;
interface MappersTableProps {
group: DraftGroup;
store: AttributeMappingStore;
}
// Nested table of a group's mappers, rendered inside the group's expanded row.
// This component only mounts when its group row is expanded, so the fetch is
// lazy by construction — a group's mappers load on first open, are folded into
// the store's draft so they're editable, and are then cached by react-query.
// New (unsaved) groups have no serverId, so skip the fetch.
function MappersTable({ group, store }: MappersTableProps): JSX.Element {
const drawer = useMapperFormDrawer();
const { hydrateGroupMappers, upsertMapper, removeMapper, toggleMapper } = store;
const hasServerId = group.serverId !== null;
const { data, isLoading, isError } = useListSpanMappers(
{ groupId: group.serverId ?? '' },
{ query: { enabled: hasServerId } },
);
useEffect(() => {
const items = data?.data?.items;
if (group.serverId && items) {
// The generated schema mis-types this list response with the groups DTO;
// the runtime payload is mappers.
hydrateGroupMappers(group.serverId, items as unknown as Mapper[]);
}
}, [group.serverId, data, hydrateGroupMappers]);
const isLoadingMappers = hasServerId && isLoading;
const isErrorMappers = hasServerId && isError;
const handleSave = useCallback((): void => {
upsertMapper(group.localId, drawer.draft);
drawer.close();
}, [upsertMapper, group.localId, drawer]);
const handleDelete = useCallback((): void => {
if (drawer.draft.id) {
removeMapper(group.localId, drawer.draft.id);
}
drawer.close();
}, [removeMapper, group.localId, drawer]);
const columns = useMemo(
() =>
getMappersColumns({
onEdit: drawer.openForEdit,
onRemove: (localId): void => removeMapper(group.localId, localId),
onToggle: (localId, enabled): void =>
toggleMapper(group.localId, localId, enabled),
}),
[drawer.openForEdit, removeMapper, toggleMapper, group.localId],
);
let content: JSX.Element;
if (isErrorMappers) {
content = (
<div
className={styles.tableEmpty}
data-testid={`mappers-error-${group.localId}`}
>
Failed to load mappings. Please try again.
</div>
);
} else if (!isLoadingMappers && group.mappers.length === 0) {
content = (
<div
className={styles.tableEmpty}
data-testid={`mappers-empty-${group.localId}`}
>
No mappings in this group yet.
</div>
);
} else {
content = (
<TanStackTable<DraftMapper>
data={group.mappers}
columns={columns}
isLoading={isLoadingMappers}
skeletonRowCount={SKELETON_ROW_COUNT}
getRowKey={(row): string => row.localId}
disableVirtualScroll
testId={`mappers-table-${group.localId}`}
/>
);
}
return (
<div className={styles.mappersTableWrapper}>
{content}
<Button
variant="ghost"
color="primary"
size="sm"
prefix={<Plus size={14} />}
className={styles.addRow}
onClick={drawer.openForAdd}
testId={`add-mapper-${group.localId}`}
>
Add mapping
</Button>
{drawer.isOpen && (
<MapperFormDrawer
isOpen={drawer.isOpen}
mode={drawer.mode}
draft={drawer.draft}
setDraft={drawer.setDraft}
onClose={drawer.close}
onSave={handleSave}
onDelete={handleDelete}
isSaving={false}
isDeleting={false}
saveError={null}
/>
)}
</div>
);
}
export default MappersTable;

View File

@@ -0,0 +1,116 @@
import { Button } from '@signozhq/ui/button';
import { SelectSimple } from '@signozhq/ui/select';
import { useSortable } from '@dnd-kit/sortable';
import { CSS } from '@dnd-kit/utilities';
import { GripVertical, X } from '@signozhq/icons';
import KeySearchInput from './KeySearchInput';
import styles from './MapperFormDrawer.module.scss';
import {
FieldContext,
FieldContextValue,
MapperOperation,
MapperOperationValue,
SourceConfig,
} from './types';
const CONTEXT_OPTIONS = [
{ value: FieldContext.attribute, label: 'Attribute' },
{ value: FieldContext.resource, label: 'Resource' },
];
const OPERATION_OPTIONS = [
{ value: MapperOperation.move, label: 'Move' },
{ value: MapperOperation.copy, label: 'Copy' },
];
interface SourceAttributeRowProps {
id: string;
index: number;
value: SourceConfig;
canRemove: boolean;
onChange: (index: number, patch: Partial<SourceConfig>) => void;
onRemove: (index: number) => void;
}
// A single draggable source row. Order = priority (top wins). Each source can
// be read from a span attribute or the resource, and moved (delete source) or
// copied (keep source). Only the grip is a drag handle.
function SourceAttributeRow({
id,
index,
value,
canRemove,
onChange,
onRemove,
}: SourceAttributeRowProps): JSX.Element {
const {
attributes,
listeners,
setNodeRef,
transform,
transition,
isDragging,
} = useSortable({ id });
const style = {
transform: CSS.Transform.toString(transform),
transition,
opacity: isDragging ? 0.6 : 1,
};
return (
<div className={styles.source} ref={setNodeRef} style={style}>
<div
className={styles.sourceHandle}
data-testid={`mapper-form-source-handle-${index}`}
{...attributes}
{...listeners}
>
<GripVertical size={14} />
</div>
<span className={styles.sourceIndex}>{index + 1}</span>
<KeySearchInput
className={styles.sourceInput}
placeholder="Source attribute key"
value={value.key}
fieldContext={value.context}
onChange={(key): void => onChange(index, { key })}
testId={`mapper-form-source-${index}`}
/>
<SelectSimple
className={styles.sourceSelect}
items={CONTEXT_OPTIONS}
value={value.context}
withPortal={false}
onChange={(next): void =>
onChange(index, { context: next as FieldContextValue })
}
testId={`mapper-form-source-context-${index}`}
/>
<SelectSimple
className={styles.sourceSelect}
items={OPERATION_OPTIONS}
value={value.operation}
withPortal={false}
onChange={(next): void =>
onChange(index, { operation: next as MapperOperationValue })
}
testId={`mapper-form-source-operation-${index}`}
/>
<Button
variant="ghost"
color="secondary"
size="icon"
aria-label="Remove source"
disabled={!canRemove}
onClick={(): void => onRemove(index)}
testId={`mapper-form-source-remove-${index}`}
>
<X size={14} />
</Button>
</div>
);
}
export default SourceAttributeRow;

View File

@@ -0,0 +1,88 @@
import { Badge } from '@signozhq/ui/badge';
import { SpantypesSpanMapperTestSpanDTO } from 'api/generated/services/sigNoz.schemas';
import { useMemo } from 'react';
import { AttrChangeStatus, diffSpanAttributes } from './testPayload';
import styles from './TestTab.module.scss';
interface TestResultProps {
index: number;
span: SpantypesSpanMapperTestSpanDTO;
inputAttributes: Record<string, unknown>;
}
const STATUS_BADGE: Partial<
Record<
AttrChangeStatus,
{ color: 'success' | 'robin' | 'sienna'; label: string }
>
> = {
added: { color: 'success', label: 'populated' },
changed: { color: 'robin', label: 'remapped' },
removed: { color: 'sienna', label: 'moved out' },
};
// Only added/removed rows carry a background treatment; the rest render plain.
// Kept as an explicit map so we never do dynamic `styles[...]` access.
const ROW_CLASS: Partial<Record<AttrChangeStatus, string>> = {
added: styles.added,
removed: styles.removed,
};
function formatValue(value: unknown): string {
if (typeof value === 'string') {
return value;
}
return JSON.stringify(value);
}
// Renders one transformed span as a key/value list, highlighting which target
// attributes the mappers populated and which source keys a move consumed.
function TestResult({
index,
span,
inputAttributes,
}: TestResultProps): JSX.Element {
const entries = useMemo(
() => diffSpanAttributes(inputAttributes, span.attributes ?? {}),
[inputAttributes, span.attributes],
);
return (
<div className={styles.resultCard} data-testid={`test-result-${index}`}>
<div className={styles.resultTitle}>Resulting attributes</div>
{entries.length === 0 ? (
<div className={styles.resultEmpty}>This span has no attributes.</div>
) : (
<div className={styles.attrRows}>
{entries.map((entry) => {
const badge = STATUS_BADGE[entry.status];
return (
<div
key={entry.key}
className={`${styles.attrRow} ${ROW_CLASS[entry.status] ?? ''}`}
>
<span className={styles.attrKey} title={entry.key}>
{entry.key}
</span>
<span className={styles.attrValue} title={formatValue(entry.value)}>
{formatValue(entry.value)}
</span>
{badge ? (
<Badge color={badge.color} variant="outline">
{badge.label}
</Badge>
) : (
<span />
)}
</div>
);
})}
</div>
)}
</div>
);
}
export default TestResult;

View File

@@ -0,0 +1,129 @@
.testTab {
display: flex;
flex-direction: column;
gap: var(--spacing-6);
max-width: 860px;
}
.heading {
display: flex;
align-items: center;
gap: var(--spacing-3);
margin: 0;
font-size: var(--periscope-font-size-base);
font-weight: var(--font-weight-semibold);
}
.description {
margin: 0;
font-size: var(--periscope-font-size-base);
color: var(--l3-foreground);
}
.editor {
width: 100%;
min-height: 260px;
padding: var(--padding-4);
border: 1px solid var(--l2-border);
border-radius: var(--radius-2);
background: var(--l1-background);
color: var(--l1-foreground);
font-family: 'Geist Mono', monospace;
font-size: var(--font-size-xs);
line-height: 1.6;
resize: vertical;
&:focus {
outline: none;
border-color: var(--robin-500);
}
}
.actions {
display: flex;
align-items: center;
gap: var(--spacing-4);
}
.error {
padding: var(--padding-3) var(--padding-4);
border-radius: var(--radius-2);
background: var(--callout-error-background);
color: var(--callout-error-title);
font-size: var(--periscope-font-size-base);
}
.results {
display: flex;
flex-direction: column;
gap: var(--spacing-6);
}
.resultCard {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
padding: var(--padding-4);
border: 1px solid var(--l2-border);
border-radius: var(--radius-2);
background: var(--l1-background);
}
.resultTitle {
display: flex;
align-items: center;
gap: var(--spacing-3);
font-size: var(--periscope-font-size-base);
font-weight: var(--font-weight-semibold);
}
.resultEmpty {
color: var(--l3-foreground);
font-size: var(--periscope-font-size-base);
}
.attrRows {
display: flex;
flex-direction: column;
gap: var(--spacing-2);
}
.attrRow {
display: grid;
grid-template-columns: minmax(0, 1fr) minmax(0, 1.4fr) auto;
align-items: center;
gap: var(--spacing-4);
padding: var(--spacing-2) var(--padding-3);
border-radius: var(--radius-2);
font-family: 'Geist Mono', monospace;
font-size: var(--font-size-xs);
&.added {
background: var(--callout-success-background);
}
&.removed {
opacity: 0.65;
.attrKey,
.attrValue {
text-decoration: line-through;
}
}
}
.attrKey,
.attrValue {
min-width: 0;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.attrKey {
font-weight: var(--font-weight-medium);
}
.attrValue {
color: var(--l3-foreground);
}

View File

@@ -0,0 +1,80 @@
import { Play } from '@signozhq/icons';
import { Button } from '@signozhq/ui/button';
import styles from './TestTab.module.scss';
import TestResult from './TestResult';
import { AttributeMappingStore } from './useAttributeMappingStore';
import { useTestSpanMapper } from './useTestSpanMapper';
interface TestTabProps {
store: AttributeMappingStore;
}
// "Test" tab: paste a sample span, run it through the working draft's mappers,
// and see which target attributes get populated. Only groups whose mappers
// changed are sent in full — unchanged groups are backfilled server-side.
function TestTab({ store }: TestTabProps): JSX.Element {
const { input, setInput, run, isRunning, result, testedAttributes, error } =
useTestSpanMapper(store.snapshot, store.groups);
return (
<div className={styles.testTab} data-testid="test-tab">
<h3 className={styles.heading}>Test with sample span</h3>
<p className={styles.description}>
Paste a JSON span object to see which target attributes get populated and
which source key matched.
</p>
<textarea
className={styles.editor}
data-testid="test-span-input"
value={input}
onChange={(event): void => setInput(event.target.value)}
spellCheck={false}
aria-label="Sample span JSON"
/>
<div className={styles.actions}>
<Button
testId="run-test-button"
variant="solid"
color="primary"
prefix={<Play size={14} />}
onClick={run}
loading={isRunning}
disabled={isRunning}
>
Run Test
</Button>
</div>
{error && (
<div className={styles.error} role="alert" data-testid="test-error">
{error}
</div>
)}
{result && (
<div className={styles.results} data-testid="test-results">
{result.length === 0 ? (
<div className={styles.resultEmpty}>
No spans returned. The mappers produced no output for this input.
</div>
) : (
result.map((span, index) => (
<TestResult
// eslint-disable-next-line react/no-array-index-key
key={index}
index={index}
span={span}
inputAttributes={testedAttributes ?? {}}
/>
))
)}
</div>
)}
</div>
);
}
export default TestTab;

View File

@@ -0,0 +1,128 @@
import { Badge } from '@signozhq/ui/badge';
import { Button } from '@signozhq/ui/button';
import { Switch } from '@signozhq/ui/switch';
import { ChevronDown, ChevronRight, Pencil, Trash2 } from '@signozhq/icons';
import type { TableColumnDef } from 'components/TanStackTableView';
import styles from './LLMObservabilityAttributeMapping.module.scss';
import type { DraftGroup } from './types';
import { conditionFiltersFromGroup } from './utils';
interface ColumnsConfig {
onEditGroup: (group: DraftGroup) => void;
onRemoveGroup: (localId: string) => void;
onToggleGroup: (localId: string, enabled: boolean) => void;
}
// Column definitions for the mapping-groups TanStackTable. Sorting is off across
// the board — the groups list API returns the full set unordered, so there's no
// server-side ordering to back a sortable header yet.
export function getMapperGroupsColumns({
onEditGroup,
onRemoveGroup,
onToggleGroup,
}: ColumnsConfig): TableColumnDef<DraftGroup>[] {
return [
{
id: 'name',
header: 'Group name',
accessorFn: (row): string => row.name,
width: { min: 240, default: '100%' },
enableMove: false,
enableRemove: false,
cell: ({ row, isExpanded, toggleExpanded }): JSX.Element => (
<div className={styles.groupsTableNameCell}>
<Button
variant="ghost"
color="secondary"
size="icon"
aria-label={isExpanded ? 'Collapse group' : 'Expand group'}
onClick={(): void => toggleExpanded()}
testId={`group-expand-${row.localId}`}
>
{isExpanded ? <ChevronDown size={14} /> : <ChevronRight size={14} />}
</Button>
<span
className={styles.groupsTableName}
data-testid={`group-name-${row.localId}`}
>
{row.name}
</span>
</div>
),
},
{
id: 'filters',
header: 'Filters',
width: { min: 200, default: '100%' },
enableMove: false,
cell: ({ row }): JSX.Element => {
const filters = conditionFiltersFromGroup(row);
if (filters.length === 0) {
return <span className={styles.muted}>No condition · always runs</span>;
}
return (
<div
className={styles.groupsTableFilters}
data-testid={`group-filters-${row.localId}`}
>
{filters.map((filter) => (
<div
className={styles.groupsTableFilter}
key={`${filter.context}:${filter.key}`}
>
<Badge
color={filter.context === 'resource' ? 'amber' : 'robin'}
variant="outline"
>
{filter.context}
</Badge>
<span className={styles.groupsTableFilterKey}>
contains {filter.key}
</span>
</div>
))}
</div>
);
},
},
{
id: 'actions',
header: 'Actions',
// Compact, right-aligned action cluster — opt out of the "last column
// fills 100%" rule so the spare width flows into Group name / Filters.
width: { fixed: '160px', ignoreLastColumnFill: true },
enableMove: false,
enableRemove: false,
cell: ({ row }): JSX.Element => (
<div className={styles.rowActions}>
<Button
variant="ghost"
color="secondary"
size="icon"
aria-label="Edit group"
onClick={(): void => onEditGroup(row)}
testId={`group-edit-${row.localId}`}
>
<Pencil size={14} />
</Button>
<Button
variant="ghost"
color="destructive"
size="icon"
aria-label="Delete group"
onClick={(): void => onRemoveGroup(row.localId)}
testId={`group-delete-${row.localId}`}
>
<Trash2 size={14} />
</Button>
<Switch
value={row.enabled}
onChange={(checked): void => onToggleGroup(row.localId, checked)}
testId={`group-enabled-${row.localId}`}
/>
</div>
),
},
];
}

View File

@@ -0,0 +1,132 @@
import { Badge } from '@signozhq/ui/badge';
import { Button } from '@signozhq/ui/button';
import { Switch } from '@signozhq/ui/switch';
import { Pencil, Trash2 } from '@signozhq/icons';
import type { TableColumnDef } from 'components/TanStackTableView';
import cx from 'classnames';
import styles from './LLMObservabilityAttributeMapping.module.scss';
import { DraftMapper, FieldContext } from './types';
const MAX_VISIBLE_SOURCES = 3;
interface ColumnsConfig {
onEdit: (mapper: DraftMapper) => void;
onRemove: (localId: string) => void;
onToggle: (localId: string, enabled: boolean) => void;
}
// Column definitions for the per-group mappers TanStackTable (rendered inside an
// expanded group row). Sorting is off — priority order is positional (top wins).
export function getMappersColumns({
onEdit,
onRemove,
onToggle,
}: ColumnsConfig): TableColumnDef<DraftMapper>[] {
return [
{
id: 'target',
header: 'Target',
accessorFn: (row): string => row.name,
width: { min: 200, default: '100%' },
enableMove: false,
cell: ({ row }): JSX.Element => (
<span
className={styles.mappersTableTarget}
data-testid={`mapper-target-${row.localId}`}
>
{row.name}
</span>
),
},
{
id: 'sources',
header: 'Sources',
width: { min: 220, default: '100%' },
enableMove: false,
cell: ({ row }): JSX.Element => {
// Skeleton placeholder rows reach the cell before real data, so
// `sources` can be undefined — default to empty.
const sources = row.sources ?? [];
if (sources.length === 0) {
return <span className={styles.muted}></span>;
}
const visible = sources.slice(0, MAX_VISIBLE_SOURCES);
const remaining = sources.length - visible.length;
return (
<div
className={styles.mappersTableSources}
data-testid={`mapper-sources-${row.localId}`}
>
{visible.map((source) => (
<span
className={styles.mappersTableSourceChip}
key={`${source.context}:${source.key}`}
title={source.key}
>
{source.key}
</span>
))}
{remaining > 0 && (
<span className={cx(styles.mappersTableSourceMore, styles.muted)}>
+{remaining} more
</span>
)}
</div>
);
},
},
{
id: 'writesTo',
header: 'Writes to',
width: { min: 130 },
enableMove: false,
cell: ({ row }): JSX.Element => (
<Badge
color={row.fieldContext === FieldContext.resource ? 'amber' : 'robin'}
variant="outline"
>
{row.fieldContext}
</Badge>
),
},
{
id: 'actions',
header: 'Actions',
// Compact, right-aligned action cluster — opt out of the "last column
// fills 100%" rule so the spare width flows into Target / Sources.
width: { fixed: '160px', ignoreLastColumnFill: true },
enableMove: false,
enableRemove: false,
cell: ({ row }): JSX.Element => (
<div className={styles.rowActions}>
<Button
variant="ghost"
color="secondary"
size="icon"
aria-label="Edit mapping"
onClick={(): void => onEdit(row)}
testId={`mapper-edit-${row.localId}`}
>
<Pencil size={14} />
</Button>
<Button
variant="ghost"
color="destructive"
size="icon"
aria-label="Delete mapping"
onClick={(): void => onRemove(row.localId)}
testId={`mapper-delete-${row.localId}`}
>
<Trash2 size={14} />
</Button>
<Switch
value={row.enabled}
onChange={(checked): void => onToggle(row.localId, checked)}
testId={`mapper-enabled-${row.localId}`}
/>
</div>
),
},
];
}

View File

@@ -0,0 +1,190 @@
import {
SpantypesPostableSpanMapperDTO,
SpantypesPostableSpanMapperGroupDTO,
SpantypesUpdatableSpanMapperDTO,
SpantypesUpdatableSpanMapperGroupDTO,
} from 'api/generated/services/sigNoz.schemas';
import { DraftGroup, DraftMapper, SourceConfig } from './types';
import {
buildPostableGroup,
buildPostableMapper,
buildUpdatableGroup,
buildUpdatableMapper,
} from './utils';
// Thin persistence surface the store wires to the generated mutations.
// createGroup returns the new server id so its mappers can be created under it.
export interface SaveMutations {
createGroup: (data: SpantypesPostableSpanMapperGroupDTO) => Promise<string>;
updateGroup: (
groupId: string,
data: SpantypesUpdatableSpanMapperGroupDTO,
) => Promise<void>;
deleteGroup: (groupId: string) => Promise<void>;
createMapper: (
groupId: string,
data: SpantypesPostableSpanMapperDTO,
) => Promise<void>;
updateMapper: (
groupId: string,
mapperId: string,
data: SpantypesUpdatableSpanMapperDTO,
) => Promise<void>;
deleteMapper: (groupId: string, mapperId: string) => Promise<void>;
}
function arraysEqual(a: string[], b: string[]): boolean {
return a.length === b.length && a.every((value, index) => value === b[index]);
}
function sourcesEqual(a: SourceConfig[], b: SourceConfig[]): boolean {
return (
a.length === b.length &&
a.every(
(source, index) =>
source.key === b[index].key &&
source.context === b[index].context &&
source.operation === b[index].operation,
)
);
}
function groupChanged(snapshot: DraftGroup, draft: DraftGroup): boolean {
return (
snapshot.name !== draft.name ||
snapshot.enabled !== draft.enabled ||
!arraysEqual(snapshot.attributes, draft.attributes) ||
!arraysEqual(snapshot.resource, draft.resource)
);
}
function mapperChanged(snapshot: DraftMapper, draft: DraftMapper): boolean {
return (
snapshot.enabled !== draft.enabled ||
snapshot.fieldContext !== draft.fieldContext ||
!sourcesEqual(snapshot.sources, draft.sources)
);
}
function groupDraftOf(
node: DraftGroup,
): Parameters<typeof buildPostableGroup>[0] {
return {
id: node.serverId,
name: node.name,
attributes: node.attributes,
resource: node.resource,
enabled: node.enabled,
};
}
function mapperDraftOf(
node: DraftMapper,
): Parameters<typeof buildPostableMapper>[0] {
return {
id: node.serverId,
name: node.name,
fieldContext: node.fieldContext,
sources: node.sources,
enabled: node.enabled,
};
}
async function persistMappers(
groupServerId: string,
snapshotMappers: DraftMapper[],
draftMappers: DraftMapper[],
m: SaveMutations,
): Promise<void> {
const snapById = new Map(
snapshotMappers
.filter((mapper) => mapper.serverId)
.map((mapper) => [mapper.serverId as string, mapper]),
);
const draftServerIds = new Set(
draftMappers
.filter((mapper) => mapper.serverId)
.map((mapper) => mapper.serverId as string),
);
// Deleted mappers.
await Promise.all(
snapshotMappers
.filter((mapper) => mapper.serverId && !draftServerIds.has(mapper.serverId))
.map((mapper) => m.deleteMapper(groupServerId, mapper.serverId as string)),
);
// Created + updated mappers (sequential to keep ordering deterministic).
for (const mapper of draftMappers) {
if (!mapper.serverId) {
// eslint-disable-next-line no-await-in-loop
await m.createMapper(
groupServerId,
buildPostableMapper(mapperDraftOf(mapper)),
);
} else {
const snap = snapById.get(mapper.serverId);
if (!snap || mapperChanged(snap, mapper)) {
// eslint-disable-next-line no-await-in-loop
await m.updateMapper(
groupServerId,
mapper.serverId,
buildUpdatableMapper(mapperDraftOf(mapper)),
);
}
}
}
}
// Diffs the staged tree against the server snapshot and issues the minimal set
// of create/update/delete calls to reconcile them.
export async function persistDraft(
snapshot: DraftGroup[],
draft: DraftGroup[],
m: SaveMutations,
): Promise<void> {
const snapById = new Map(
snapshot
.filter((group) => group.serverId)
.map((group) => [group.serverId as string, group]),
);
const draftServerIds = new Set(
draft
.filter((group) => group.serverId)
.map((group) => group.serverId as string),
);
// Apply additive work (creates/updates) before deletes, so a failure here
// leaves at worst an incomplete set of additions rather than groups that
// were deleted with no replacement persisted. Deletes are irreversible
// (they cascade mappers server-side), so we do them last, once everything
// else has succeeded.
for (const group of draft) {
if (!group.serverId) {
// eslint-disable-next-line no-await-in-loop
const newId = await m.createGroup(buildPostableGroup(groupDraftOf(group)));
// eslint-disable-next-line no-await-in-loop
await persistMappers(newId, [], group.mappers, m);
continue;
}
const snap = snapById.get(group.serverId);
if (!snap || groupChanged(snap, group)) {
// eslint-disable-next-line no-await-in-loop
await m.updateGroup(
group.serverId,
buildUpdatableGroup(groupDraftOf(group)),
);
}
// eslint-disable-next-line no-await-in-loop
await persistMappers(group.serverId, snap?.mappers ?? [], group.mappers, m);
}
// Deleted groups (cascades mappers server-side).
await Promise.all(
snapshot
.filter((group) => group.serverId && !draftServerIds.has(group.serverId))
.map((group) => m.deleteGroup(group.serverId as string)),
);
}

View File

@@ -0,0 +1,135 @@
import {
SpantypesPostableSpanMapperTestDTO,
SpantypesPostableSpanMapperTestGroupDTO,
SpantypesSpanMapperTestSpanDTO,
} from 'api/generated/services/sigNoz.schemas';
import { DraftGroup } from './types';
import {
buildPostableGroup,
buildPostableMapper,
groupDraftFromNode,
mapperDraftFromNode,
} from './utils';
// A group's mappers must be sent in full when the group is new (the backend has
// no saved group of that name to backfill from) or when its mapper set has
// diverged from the saved baseline. Otherwise we omit them and let the backend
// load the saved mappers by group name — which also covers groups whose rows
// were never expanded (their draft carries no mappers yet).
function shouldSendMappers(
snap: DraftGroup | undefined,
group: DraftGroup,
): boolean {
if (!snap) {
return true;
}
return JSON.stringify(snap.mappers) !== JSON.stringify(group.mappers);
}
// Builds the `groups` portion of the test request from the working draft,
// sending each group's name/enabled/condition from the current draft and its
// `mappers` only when they changed (null otherwise → backend backfills).
export function buildTestGroups(
snapshot: DraftGroup[],
draft: DraftGroup[],
): SpantypesPostableSpanMapperTestGroupDTO[] {
const snapById = new Map(
snapshot
.filter((group) => group.serverId)
.map((group) => [group.serverId as string, group]),
);
return draft.map((group) => {
const base = buildPostableGroup(groupDraftFromNode(group));
const snap = group.serverId ? snapById.get(group.serverId) : undefined;
return {
...base,
mappers: shouldSendMappers(snap, group)
? group.mappers.map((mapper) =>
buildPostableMapper(mapperDraftFromNode(mapper)),
)
: null,
};
});
}
// Parses the pasted JSON into a single test span. The pasted object is treated
// as the span's attribute map (matching the sample shown to the user); resource
// is left empty. Throws a friendly error on anything that isn't a JSON object.
export function parseSpanInput(input: string): SpantypesSpanMapperTestSpanDTO {
const trimmed = input.trim();
if (!trimmed) {
throw new Error('Paste a JSON span object to run the test.');
}
let parsed: unknown;
try {
parsed = JSON.parse(trimmed);
} catch {
throw new Error(
'Invalid JSON — check for trailing commas or missing quotes.',
);
}
if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) {
throw new Error('Span must be a JSON object of attribute key-value pairs.');
}
return { attributes: parsed as Record<string, unknown>, resource: {} };
}
export function buildTestRequest(
snapshot: DraftGroup[],
draft: DraftGroup[],
input: string,
): SpantypesPostableSpanMapperTestDTO {
return {
groups: buildTestGroups(snapshot, draft),
spans: [parseSpanInput(input)],
};
}
export type AttrChangeStatus = 'added' | 'changed' | 'unchanged' | 'removed';
export interface AttrDiffEntry {
key: string;
value: unknown;
status: AttrChangeStatus;
}
function valuesEqual(a: unknown, b: unknown): boolean {
return JSON.stringify(a) === JSON.stringify(b);
}
// Diffs the span attributes a user pasted against the attributes the mappers
// produced, so the UI can highlight which target keys got populated ('added'),
// which source keys were consumed by a move ('removed'), and what stayed.
// Added keys (the populated targets) sort first as the primary signal.
export function diffSpanAttributes(
inputAttributes: Record<string, unknown>,
resultAttributes: Record<string, unknown>,
): AttrDiffEntry[] {
const added: AttrDiffEntry[] = [];
const changed: AttrDiffEntry[] = [];
const unchanged: AttrDiffEntry[] = [];
const removed: AttrDiffEntry[] = [];
Object.entries(resultAttributes).forEach(([key, value]) => {
if (!(key in inputAttributes)) {
added.push({ key, value, status: 'added' });
} else if (!valuesEqual(inputAttributes[key], value)) {
changed.push({ key, value, status: 'changed' });
} else {
unchanged.push({ key, value, status: 'unchanged' });
}
});
Object.entries(inputAttributes).forEach(([key, value]) => {
if (!(key in resultAttributes)) {
removed.push({ key, value, status: 'removed' });
}
});
return [...added, ...changed, ...unchanged, ...removed];
}

View File

@@ -0,0 +1,84 @@
import {
SpantypesFieldContextDTO,
SpantypesSpanMapperConfigDTO,
SpantypesSpanMapperDTO,
SpantypesSpanMapperGroupConditionDTO,
SpantypesSpanMapperGroupDTO,
SpantypesSpanMapperOperationDTO,
SpantypesSpanMapperSourceDTO,
} from 'api/generated/services/sigNoz.schemas';
export type MapperGroup = SpantypesSpanMapperGroupDTO;
export type MapperGroupCondition =
NonNullable<SpantypesSpanMapperGroupConditionDTO>;
export type Mapper = SpantypesSpanMapperDTO;
export type MapperConfig = SpantypesSpanMapperConfigDTO;
export type MapperSource = SpantypesSpanMapperSourceDTO;
export const FieldContext = SpantypesFieldContextDTO;
export const MapperOperation = SpantypesSpanMapperOperationDTO;
export type FieldContextValue = SpantypesFieldContextDTO;
export type MapperOperationValue = SpantypesSpanMapperOperationDTO;
// A single human-readable condition clause shown in the group's Filters column.
export interface ConditionFilter {
context: 'attribute' | 'resource';
key: string;
}
export type MapperDraftMode = 'add' | 'edit';
// One source candidate. `context` is where the key is read from (span
// attribute or resource); `operation` is move (delete source) or copy (keep).
// Priority is implicit in list order (top wins), derived on save.
export interface SourceConfig {
key: string;
context: SpantypesFieldContextDTO;
operation: SpantypesSpanMapperOperationDTO;
}
// Editable form state for a mapper. `sources` is ordered highest priority
// first; `fieldContext` is where the standardized target is written.
export interface MapperDraft {
id: string | null;
name: string;
fieldContext: SpantypesFieldContextDTO;
sources: SourceConfig[];
enabled: boolean;
}
// Editable form state for a group. The group runs when a span carries a
// span-attribute key matching `attributes` OR a resource key matching
// `resource` (plain substring match).
export interface GroupDraft {
id: string | null;
name: string;
attributes: string[];
resource: string[];
enabled: boolean;
}
// Working-copy node for a mapper. `localId` is a stable client key (the server
// id once persisted, or a temporary id for not-yet-saved rows). `serverId` is
// null until the row has been persisted.
export interface DraftMapper {
localId: string;
serverId: string | null;
name: string;
fieldContext: SpantypesFieldContextDTO;
sources: SourceConfig[];
enabled: boolean;
}
// Working-copy node for a group, holding its mappers inline so the whole tree
// can be staged locally and diffed against the server snapshot on save.
export interface DraftGroup {
localId: string;
serverId: string | null;
name: string;
attributes: string[];
resource: string[];
enabled: boolean;
mappers: DraftMapper[];
}

View File

@@ -0,0 +1,318 @@
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { toast } from '@signozhq/ui/sonner';
import { useQueryClient } from 'react-query';
import {
useCreateSpanMapper,
useCreateSpanMapperGroup,
useDeleteSpanMapper,
useDeleteSpanMapperGroup,
useListSpanMapperGroups,
useUpdateSpanMapper,
useUpdateSpanMapperGroup,
} from 'api/generated/services/spanmapper';
import { persistDraft, SaveMutations } from './saveDraft';
import {
DraftGroup,
GroupDraft,
Mapper,
MapperDraft,
MapperGroup,
} from './types';
import {
buildDraftGroup,
buildDraftMapper,
nodeFromGroupDraft,
nodeFromMapperDraft,
} from './utils';
const GROUPS_KEY_PREFIX = '/api/v1/span_mapper_groups';
function clone(groups: DraftGroup[]): DraftGroup[] {
return JSON.parse(JSON.stringify(groups)) as DraftGroup[];
}
export interface AttributeMappingStore {
groups: DraftGroup[];
// The last-saved server baseline the working `groups` are diffed against.
// Exposed so the Test tab can send only the groups whose mappers changed.
snapshot: DraftGroup[];
isLoading: boolean;
isError: boolean;
isDirty: boolean;
isSaving: boolean;
saveError: string | null;
upsertGroup: (draft: GroupDraft) => void;
removeGroup: (localId: string) => void;
toggleGroup: (localId: string, enabled: boolean) => void;
hydrateGroupMappers: (groupServerId: string, mappers: Mapper[]) => void;
upsertMapper: (groupLocalId: string, draft: MapperDraft) => void;
removeMapper: (groupLocalId: string, mapperLocalId: string) => void;
toggleMapper: (
groupLocalId: string,
mapperLocalId: string,
enabled: boolean,
) => void;
save: () => Promise<void>;
discard: () => void;
}
export function useAttributeMappingStore(): AttributeMappingStore {
const queryClient = useQueryClient();
const groupsQuery = useListSpanMapperGroups();
const serverGroups: MapperGroup[] = useMemo(
() => groupsQuery.data?.data?.items ?? [],
[groupsQuery.data],
);
const ready = !groupsQuery.isLoading;
// A group's mappers are fetched lazily when its row is first expanded (see
// MappersTable -> hydrateGroupMappers) and cached here, keyed by group server
// id. Page load stays a single groups request — never an N+1 fan-out across
// every group.
const [loadedMappers, setLoadedMappers] = useState<Record<string, Mapper[]>>(
{},
);
const loadedRef = useRef<Set<string>>(new Set());
const snapshot = useMemo<DraftGroup[]>(() => {
if (!ready) {
return [];
}
return serverGroups.map((group) =>
buildDraftGroup(group, loadedMappers[group.id] ?? []),
);
}, [ready, serverGroups, loadedMappers]);
const [draft, setDraft] = useState<DraftGroup[] | null>(null);
// Initialise the working copy once data is ready (and re-init after a save
// clears it). Never clobbers in-flight edits — only runs when draft is null.
useEffect(() => {
if (ready && draft === null) {
setDraft(clone(snapshot));
}
}, [ready, draft, snapshot]);
const [isSaving, setIsSaving] = useState(false);
const [saveError, setSaveError] = useState<string | null>(null);
const { mutateAsync: createGroup } = useCreateSpanMapperGroup();
const { mutateAsync: updateGroup } = useUpdateSpanMapperGroup();
const { mutateAsync: deleteGroup } = useDeleteSpanMapperGroup();
const { mutateAsync: createMapper } = useCreateSpanMapper();
const { mutateAsync: updateMapper } = useUpdateSpanMapper();
const { mutateAsync: deleteMapper } = useDeleteSpanMapper();
const mutations: SaveMutations = useMemo(
() => ({
createGroup: async (data): Promise<string> => {
const result = await createGroup({ data });
return result.data.id;
},
updateGroup: async (groupId, data): Promise<void> => {
await updateGroup({ pathParams: { groupId }, data });
},
deleteGroup: async (groupId): Promise<void> => {
await deleteGroup({ pathParams: { groupId } });
},
createMapper: async (groupId, data): Promise<void> => {
await createMapper({ pathParams: { groupId }, data });
},
updateMapper: async (groupId, mapperId, data): Promise<void> => {
await updateMapper({ pathParams: { groupId, mapperId }, data });
},
deleteMapper: async (groupId, mapperId): Promise<void> => {
await deleteMapper({ pathParams: { groupId, mapperId } });
},
}),
[
createGroup,
updateGroup,
deleteGroup,
createMapper,
updateMapper,
deleteMapper,
],
);
const upsertGroup = useCallback((groupDraft: GroupDraft): void => {
setDraft((prev) => {
const groups = prev ?? [];
if (groupDraft.id) {
return groups.map((group) =>
group.localId === groupDraft.id
? nodeFromGroupDraft(groupDraft, group)
: group,
);
}
return [...groups, nodeFromGroupDraft(groupDraft)];
});
}, []);
const removeGroup = useCallback((localId: string): void => {
setDraft((prev) => (prev ?? []).filter((group) => group.localId !== localId));
}, []);
const toggleGroup = useCallback((localId: string, enabled: boolean): void => {
setDraft((prev) =>
(prev ?? []).map((group) =>
group.localId === localId ? { ...group, enabled } : group,
),
);
}, []);
// Fold a group's freshly-fetched server mappers into both the snapshot
// baseline and the working draft, exactly once per group (the guard skips
// re-expands). The "Add mapping" affordance renders while the fetch is still
// in flight, so a user can stage a new mapper (serverId === null) before the
// server list arrives — those unsaved rows are preserved, with the server
// mappers folded in ahead of them, so a racing add isn't clobbered.
const hydrateGroupMappers = useCallback(
(groupServerId: string, mappers: Mapper[]): void => {
if (loadedRef.current.has(groupServerId)) {
return;
}
loadedRef.current.add(groupServerId);
setLoadedMappers((prev) => ({ ...prev, [groupServerId]: mappers }));
setDraft((prev) =>
prev === null
? prev
: prev.map((group) =>
group.serverId === groupServerId
? {
...group,
mappers: [
...mappers.map(buildDraftMapper),
...group.mappers.filter((mapper) => mapper.serverId === null),
],
}
: group,
),
);
},
[],
);
const upsertMapper = useCallback(
(groupLocalId: string, mapperDraft: MapperDraft): void => {
setDraft((prev) =>
(prev ?? []).map((group) => {
if (group.localId !== groupLocalId) {
return group;
}
if (mapperDraft.id) {
return {
...group,
mappers: group.mappers.map((mapper) =>
mapper.localId === mapperDraft.id
? nodeFromMapperDraft(mapperDraft, mapper)
: mapper,
),
};
}
return {
...group,
mappers: [...group.mappers, nodeFromMapperDraft(mapperDraft)],
};
}),
);
},
[],
);
const removeMapper = useCallback(
(groupLocalId: string, mapperLocalId: string): void => {
setDraft((prev) =>
(prev ?? []).map((group) =>
group.localId === groupLocalId
? {
...group,
mappers: group.mappers.filter(
(mapper) => mapper.localId !== mapperLocalId,
),
}
: group,
),
);
},
[],
);
const toggleMapper = useCallback(
(groupLocalId: string, mapperLocalId: string, enabled: boolean): void => {
setDraft((prev) =>
(prev ?? []).map((group) =>
group.localId === groupLocalId
? {
...group,
mappers: group.mappers.map((mapper) =>
mapper.localId === mapperLocalId ? { ...mapper, enabled } : mapper,
),
}
: group,
),
);
},
[],
);
const discard = useCallback((): void => {
setSaveError(null);
setDraft(clone(snapshot));
}, [snapshot]);
const save = useCallback(async (): Promise<void> => {
if (!draft) {
return;
}
setIsSaving(true);
setSaveError(null);
try {
await persistDraft(snapshot, draft, mutations);
await queryClient.invalidateQueries({
predicate: (query) =>
typeof query.queryKey?.[0] === 'string' &&
(query.queryKey[0] as string).startsWith(GROUPS_KEY_PREFIX),
});
// Drop lazily-loaded mappers and re-initialise the working copy from the
// freshly-fetched server data; expanded rows re-hydrate on next render.
loadedRef.current = new Set();
setLoadedMappers({});
setDraft(null);
toast.success('Attribute mapping changes saved');
} catch (error) {
const message = error instanceof Error ? error.message : 'Save failed';
setSaveError(message);
toast.error(`Failed to save changes: ${message}`);
} finally {
setIsSaving(false);
}
}, [draft, snapshot, mutations, queryClient]);
const isDirty = useMemo(
() => draft !== null && JSON.stringify(draft) !== JSON.stringify(snapshot),
[draft, snapshot],
);
return {
groups: draft ?? [],
snapshot,
isLoading: !ready || draft === null,
isError: groupsQuery.isError,
isDirty,
isSaving,
saveError,
upsertGroup,
removeGroup,
toggleGroup,
hydrateGroupMappers,
upsertMapper,
removeMapper,
toggleMapper,
save,
discard,
};
}

View File

@@ -0,0 +1,40 @@
import { useCallback, useState } from 'react';
import { DraftGroup, GroupDraft, MapperDraftMode } from './types';
import { EMPTY_GROUP_DRAFT, groupDraftFromNode } from './utils';
interface UseGroupFormDrawer {
isOpen: boolean;
mode: MapperDraftMode;
draft: GroupDraft;
setDraft: (next: GroupDraft) => void;
openForAdd: () => void;
openForEdit: (group: DraftGroup) => void;
close: () => void;
}
// Form state for the group drawer. Persistence is staged through the store,
// so this hook only owns open/draft/mode.
export function useGroupFormDrawer(): UseGroupFormDrawer {
const [isOpen, setIsOpen] = useState<boolean>(false);
const [mode, setMode] = useState<MapperDraftMode>('add');
const [draft, setDraft] = useState<GroupDraft>(EMPTY_GROUP_DRAFT);
const openForAdd = useCallback((): void => {
setMode('add');
setDraft(EMPTY_GROUP_DRAFT);
setIsOpen(true);
}, []);
const openForEdit = useCallback((group: DraftGroup): void => {
setMode('edit');
setDraft(groupDraftFromNode(group));
setIsOpen(true);
}, []);
const close = useCallback((): void => {
setIsOpen(false);
}, []);
return { isOpen, mode, draft, setDraft, openForAdd, openForEdit, close };
}

View File

@@ -0,0 +1,40 @@
import { useCallback, useState } from 'react';
import { DraftMapper, MapperDraft, MapperDraftMode } from './types';
import { EMPTY_MAPPER_DRAFT, mapperDraftFromNode } from './utils';
interface UseMapperFormDrawerResult {
isOpen: boolean;
mode: MapperDraftMode;
draft: MapperDraft;
setDraft: (next: MapperDraft) => void;
openForAdd: () => void;
openForEdit: (mapper: DraftMapper) => void;
close: () => void;
}
// Form state for the mapper drawer. Persistence is staged through the store,
// so this hook only owns open/draft/mode.
export function useMapperFormDrawer(): UseMapperFormDrawerResult {
const [isOpen, setIsOpen] = useState<boolean>(false);
const [mode, setMode] = useState<MapperDraftMode>('add');
const [draft, setDraft] = useState<MapperDraft>(EMPTY_MAPPER_DRAFT);
const openForAdd = useCallback((): void => {
setMode('add');
setDraft(EMPTY_MAPPER_DRAFT);
setIsOpen(true);
}, []);
const openForEdit = useCallback((mapper: DraftMapper): void => {
setMode('edit');
setDraft(mapperDraftFromNode(mapper));
setIsOpen(true);
}, []);
const close = useCallback((): void => {
setIsOpen(false);
}, []);
return { isOpen, mode, draft, setDraft, openForAdd, openForEdit, close };
}

View File

@@ -0,0 +1,109 @@
import { useCallback, useState } from 'react';
import {
RenderErrorResponseDTO,
SpantypesSpanMapperTestSpanDTO,
} from 'api/generated/services/sigNoz.schemas';
import { useTestSpanMappers } from 'api/generated/services/spanmapper';
import { AxiosError } from 'axios';
import { buildTestRequest } from './testPayload';
import { DraftGroup } from './types';
// Pre-filled sample so the tab is runnable on first open (mirrors the design).
export const SAMPLE_SPAN_JSON = `{
"my_company.llm.input": "What is quantum computing?",
"llm.input_messages": "What is quantum computing?",
"gen_ai.request.model": "gpt-4",
"gen_ai.usage.total_tokens": 1250,
"gen_ai.content.completion": "Quantum computing leverages..."
}`;
function apiErrorMessage(error: unknown): string {
const axiosError = error as AxiosError<RenderErrorResponseDTO>;
return (
axiosError?.response?.data?.error?.message ??
(error instanceof Error ? error.message : 'Test failed. Please try again.')
);
}
export interface UseTestSpanMapper {
input: string;
setInput: (value: string) => void;
run: () => void;
reset: () => void;
isRunning: boolean;
result: SpantypesSpanMapperTestSpanDTO[] | null;
// The attributes that were actually submitted with the last successful run,
// so the result diff stays stable even if the textarea is edited afterwards.
testedAttributes: Record<string, unknown> | null;
error: string | null;
}
// Owns the Test tab's local state: the pasted span JSON, the parsed/built
// request, and the result/error of the test mutation. Builds the request from
// the working draft (sending only changed groups' mappers — see testPayload).
export function useTestSpanMapper(
snapshot: DraftGroup[],
draft: DraftGroup[],
): UseTestSpanMapper {
const [input, setInput] = useState<string>(SAMPLE_SPAN_JSON);
const [error, setError] = useState<string | null>(null);
const [result, setResult] = useState<SpantypesSpanMapperTestSpanDTO[] | null>(
null,
);
const [testedAttributes, setTestedAttributes] = useState<Record<
string,
unknown
> | null>(null);
const { mutate, isLoading } = useTestSpanMappers();
const run = useCallback((): void => {
setError(null);
let body;
try {
body = buildTestRequest(snapshot, draft, input);
} catch (parseError) {
setResult(null);
setError(apiErrorMessage(parseError));
return;
}
const submittedAttributes = (body.spans?.[0]?.attributes ?? {}) as Record<
string,
unknown
>;
mutate(
{ data: body },
{
onSuccess: (response) => {
setTestedAttributes(submittedAttributes);
setResult(response.data?.spans ?? []);
},
onError: (mutationError) => {
setResult(null);
setError(apiErrorMessage(mutationError));
},
},
);
}, [snapshot, draft, input, mutate]);
const reset = useCallback((): void => {
setError(null);
setResult(null);
setTestedAttributes(null);
}, []);
return {
input,
setInput,
run,
reset,
isRunning: isLoading,
result,
testedAttributes,
error,
};
}

View File

@@ -0,0 +1,264 @@
import {
SpantypesPostableSpanMapperDTO,
SpantypesPostableSpanMapperGroupDTO,
SpantypesUpdatableSpanMapperDTO,
SpantypesUpdatableSpanMapperGroupDTO,
} from 'api/generated/services/sigNoz.schemas';
import { v4 as uuid } from 'uuid';
import {
ConditionFilter,
DraftGroup,
DraftMapper,
FieldContext,
GroupDraft,
Mapper,
MapperDraft,
MapperGroup,
MapperOperation,
SourceConfig,
} from './types';
// Client-side id for not-yet-persisted rows. Prefixed so it never collides
// with a server UUID and is easy to spot in logs.
export function genLocalId(prefix: 'group' | 'mapper'): string {
return `local-${prefix}-${uuid()}`;
}
// Trimmed, de-duplicated, non-empty keys preserving input order.
export function cleanKeys(keys: string[]): string[] {
const seen = new Set<string>();
const result: string[] = [];
keys.forEach((raw) => {
const key = raw.trim();
if (key && !seen.has(key)) {
seen.add(key);
result.push(key);
}
});
return result;
}
// Display clauses for a group's condition keys (span attribute keys first,
// then resource keys).
export function conditionFiltersFromGroup(group: {
attributes?: string[];
resource?: string[];
}): ConditionFilter[] {
// TanStackTable renders skeleton placeholder rows through the cells on first
// render, so these arrays can be undefined before real data lands — default
// to empty rather than crashing the cell.
return [
...(group.attributes ?? []).map((key) => ({
context: 'attribute' as const,
key,
})),
...(group.resource ?? []).map((key) => ({
context: 'resource' as const,
key,
})),
];
}
// Source configs for a mapper, highest priority first (first match wins at
// evaluation time).
export function getMapperSources(mapper: Mapper): SourceConfig[] {
const sources = mapper.config?.sources ?? [];
return [...sources]
.sort((a, b) => b.priority - a.priority)
.map((source) => ({
key: source.key,
context: source.context,
operation: source.operation,
}));
}
export function createEmptySource(): SourceConfig {
return {
key: '',
context: FieldContext.attribute,
operation: MapperOperation.copy,
};
}
export const EMPTY_MAPPER_DRAFT: MapperDraft = {
id: null,
name: '',
fieldContext: FieldContext.attribute,
sources: [createEmptySource()],
enabled: true,
};
// Trimmed, de-duplicated (by context+key), non-empty sources in priority order,
// preserving each source's context and operation. A key identifies a different
// source per context (span attribute vs resource), so both can coexist; only an
// exact (context, key) repeat is collapsed, keeping the higher-priority row.
export function getCleanSources(draft: MapperDraft): SourceConfig[] {
const seen = new Set<string>();
const result: SourceConfig[] = [];
draft.sources.forEach((source) => {
const key = source.key.trim();
const dedupeKey = `${source.context}:${key}`;
if (key && !seen.has(dedupeKey)) {
seen.add(dedupeKey);
result.push({ ...source, key });
}
});
return result;
}
export function isMapperDraftValid(draft: MapperDraft): boolean {
return draft.name.trim().length > 0 && getCleanSources(draft).length > 0;
}
// Priority is derived from list order so the first row wins.
function buildSources(
draft: MapperDraft,
): SpantypesPostableSpanMapperDTO['config']['sources'] {
const sources = getCleanSources(draft);
return sources.map((source, index) => ({
key: source.key,
context: source.context,
operation: source.operation,
priority: sources.length - index,
}));
}
export function buildPostableMapper(
draft: MapperDraft,
): SpantypesPostableSpanMapperDTO {
return {
name: draft.name.trim(),
fieldContext: draft.fieldContext,
enabled: draft.enabled,
config: { sources: buildSources(draft) },
};
}
// The target name is immutable on update (UpdatableSpanMapper has no name).
export function buildUpdatableMapper(
draft: MapperDraft,
): SpantypesUpdatableSpanMapperDTO {
return {
fieldContext: draft.fieldContext,
enabled: draft.enabled,
config: { sources: buildSources(draft) },
};
}
export const EMPTY_GROUP_DRAFT: GroupDraft = {
id: null,
name: '',
attributes: [''],
resource: [],
enabled: true,
};
export function isGroupDraftValid(draft: GroupDraft): boolean {
return draft.name.trim().length > 0;
}
export function buildPostableGroup(
draft: GroupDraft,
): SpantypesPostableSpanMapperGroupDTO {
return {
name: draft.name.trim(),
enabled: draft.enabled,
condition: {
attributes: cleanKeys(draft.attributes),
resource: cleanKeys(draft.resource),
},
};
}
// A full group payload is also a valid partial-update payload (all updatable
// fields are present), so we reuse the postable builder.
export function buildUpdatableGroup(
draft: GroupDraft,
): SpantypesUpdatableSpanMapperGroupDTO {
return buildPostableGroup(draft);
}
// ---- working-copy (draft tree) helpers ----
export function buildDraftMapper(mapper: Mapper): DraftMapper {
return {
localId: mapper.id,
serverId: mapper.id,
name: mapper.name,
fieldContext: mapper.fieldContext,
sources: getMapperSources(mapper),
enabled: mapper.enabled,
};
}
export function buildDraftGroup(
group: MapperGroup,
mappers: Mapper[],
): DraftGroup {
return {
localId: group.id,
serverId: group.id,
name: group.name,
attributes: group.condition?.attributes ?? [],
resource: group.condition?.resource ?? [],
enabled: group.enabled,
mappers: mappers.map(buildDraftMapper),
};
}
// DraftGroup -> editable form state (id carries the localId).
export function groupDraftFromNode(group: DraftGroup): GroupDraft {
return {
id: group.localId,
name: group.name,
attributes: group.attributes.length > 0 ? group.attributes : [''],
resource: group.resource,
enabled: group.enabled,
};
}
// DraftMapper -> editable form state (id carries the localId).
export function mapperDraftFromNode(mapper: DraftMapper): MapperDraft {
return {
id: mapper.localId,
name: mapper.name,
fieldContext: mapper.fieldContext,
sources:
mapper.sources.length > 0
? mapper.sources.map((source) => ({ ...source }))
: [createEmptySource()],
enabled: mapper.enabled,
};
}
// Form state -> working-copy node. Reuses cleanKeys/getCleanSourceKeys so the
// staged tree already holds normalized values.
export function nodeFromGroupDraft(
draft: GroupDraft,
existing?: DraftGroup,
): DraftGroup {
return {
localId: existing?.localId ?? genLocalId('group'),
serverId: existing?.serverId ?? null,
name: draft.name.trim(),
attributes: cleanKeys(draft.attributes),
resource: cleanKeys(draft.resource),
enabled: draft.enabled,
mappers: existing?.mappers ?? [],
};
}
export function nodeFromMapperDraft(
draft: MapperDraft,
existing?: DraftMapper,
): DraftMapper {
return {
localId: existing?.localId ?? genLocalId('mapper'),
serverId: existing?.serverId ?? null,
name: draft.name.trim(),
fieldContext: draft.fieldContext,
sources: getCleanSources(draft),
enabled: draft.enabled,
};
}

View File

@@ -206,6 +206,7 @@ export const routesToSkip = [
ROUTES.METER,
ROUTES.METER_EXPLORER_VIEWS,
ROUTES.SOMETHING_WENT_WRONG,
ROUTES.LLM_OBSERVABILITY_ATTRIBUTE_MAPPING,
];
export const routesToDisable = [ROUTES.LOGS_EXPLORER, ROUTES.LIVE_LOGS];

View File

@@ -0,0 +1,7 @@
import LLMObservabilityAttributeMapping from 'container/LLMObservabilityAttributeMapping/LLMObservabilityAttributeMapping';
function LLMObservabilityAttributeMappingPage(): JSX.Element {
return <LLMObservabilityAttributeMapping />;
}
export default LLMObservabilityAttributeMappingPage;

View File

@@ -136,4 +136,5 @@ export const routePermission: Record<keyof typeof ROUTES, ROLES[]> = {
AI_ASSISTANT_ICON_PREVIEW: ['ADMIN', 'EDITOR', 'VIEWER'],
MCP_SERVER: ['ADMIN', 'EDITOR', 'VIEWER'],
AI_ASSISTANT_BASE: ['ADMIN', 'EDITOR', 'VIEWER'],
LLM_OBSERVABILITY_ATTRIBUTE_MAPPING: ['ADMIN', 'EDITOR', 'VIEWER'],
};

View File

@@ -3,16 +3,15 @@ package flagger
import "github.com/SigNoz/signoz/pkg/types/featuretypes"
var (
FeatureUseSpanMetrics = featuretypes.MustNewName("use_span_metrics")
FeatureKafkaSpanEval = featuretypes.MustNewName("kafka_span_eval")
FeatureHideRootUser = featuretypes.MustNewName("hide_root_user")
FeatureGetMetersFromZeus = featuretypes.MustNewName("get_meters_from_zeus")
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureUseMeterReporter = featuretypes.MustNewName("use_meter_reporter")
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureUseFineGrainedAuthz = featuretypes.MustNewName("use_fine_grained_authz")
FeatureUseDashboardV2 = featuretypes.MustNewName("use_dashboard_v2")
FeatureEnableMetricsReduction = featuretypes.MustNewName("enable_metrics_reduction")
FeatureUseSpanMetrics = featuretypes.MustNewName("use_span_metrics")
FeatureKafkaSpanEval = featuretypes.MustNewName("kafka_span_eval")
FeatureHideRootUser = featuretypes.MustNewName("hide_root_user")
FeatureGetMetersFromZeus = featuretypes.MustNewName("get_meters_from_zeus")
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureUseMeterReporter = featuretypes.MustNewName("use_meter_reporter")
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureUseFineGrainedAuthz = featuretypes.MustNewName("use_fine_grained_authz")
FeatureUseDashboardV2 = featuretypes.MustNewName("use_dashboard_v2")
)
func MustNewRegistry() featuretypes.Registry {
@@ -89,14 +88,6 @@ func MustNewRegistry() featuretypes.Registry {
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureEnableMetricsReduction,
Kind: featuretypes.KindBoolean,
Stage: featuretypes.StageExperimental,
Description: "Controls whether metrics cardinality reduction (buffer/reduced tables) is read by the querier",
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
)
if err != nil {
panic(err)

View File

@@ -341,12 +341,12 @@ func alignedMetricWindow(startMs, endMs int64) (
}
tsAdjustedStartMs, _, distributedTSTable, localTSTable := telemetrymetrics.WhichTSTableToUse(
samplesAdjustedStartMs, flooredEndMs, false, nil,
samplesAdjustedStartMs, flooredEndMs, nil,
)
distributedSamplesTable, localSamplesTable := telemetrymetrics.WhichSamplesTableToUse(
samplesAdjustedStartMs, flooredEndMs,
metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil,
metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, nil,
)
return samplesAdjustedStartMs, flooredEndMs, tsAdjustedStartMs, distributedTSTable, localTSTable, distributedSamplesTable, localSamplesTable

View File

@@ -141,7 +141,7 @@ func (m *module) listMetrics(ctx context.Context, orgID valuer.UUID, params *met
sb.Select("DISTINCT metric_name")
if params.Start != nil && params.End != nil {
start, end, distributedTsTable, _ := telemetrymetrics.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), false, nil)
start, end, distributedTsTable, _ := telemetrymetrics.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), nil)
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
sb.Where(sb.Between("unix_milli", start, end))
} else {
@@ -527,7 +527,7 @@ func (m *module) InspectMetrics(
return nil, err
}
tsStart, _, tsTable, _ := telemetrymetrics.WhichTSTableToUse(start, end, false, nil)
tsStart, _, tsTable, _ := telemetrymetrics.WhichTSTableToUse(start, end, nil)
tsSb := sqlbuilder.NewSelectBuilder()
tsSb.Select("fingerprint", "labels")
tsSb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, tsTable))
@@ -971,8 +971,8 @@ func (m *module) fetchMetricsStatsWithSamples(
}
}
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, nil)
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
// Timeseries counts per metric
@@ -1100,7 +1100,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, req *metricsexplo
}
}
start, end, distributedTsTable, _ := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
start, end, distributedTsTable, _ := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), nil)
totalTSBuilder := sqlbuilder.NewSelectBuilder()
totalTSBuilder.Select("uniq(fingerprint) AS total_time_series")
@@ -1176,8 +1176,8 @@ func (m *module) computeSamplesTreemap(ctx context.Context, req *metricsexplorer
}
}
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), nil)
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, nil)
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
candidateLimit := req.Limit + 50

View File

@@ -91,22 +91,13 @@ func (q *builderQuery[T]) Fingerprint() string {
if a.ComparisonSpaceAggregationParam != nil {
spaceAggParamStr = a.ComparisonSpaceAggregationParam.StringValue()
}
part := fmt.Sprintf("%s:%s:%s:%s:%s",
aggParts = append(aggParts, fmt.Sprintf("%s:%s:%s:%s:%s",
a.MetricName,
a.Temporality.StringValue(),
a.TimeAggregation.StringValue(),
a.SpaceAggregation.StringValue(),
spaceAggParamStr,
)
if a.Reduced {
oneDay := uint64(24 * time.Hour.Milliseconds())
route := "reduced"
if q.toMS-q.fromMS < oneDay && q.fromMS >= uint64(time.Now().UnixMilli())-oneDay {
route = "buffer"
}
part += ":" + route
}
aggParts = append(aggParts, part)
))
}
}
parts = append(parts, fmt.Sprintf("aggs=[%s]", strings.Join(aggParts, ",")))

View File

@@ -119,7 +119,7 @@ func (q *querier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtype
queries := make(map[string]qbtypes.Query)
steps := make(map[string]qbtypes.Step)
missingMetricQueries, metricWarnings, err := q.resolveMetricMetadata(ctx, orgID, req.CompositeQuery.Queries, req.Start, req.End)
missingMetricQueries, metricWarnings, err := q.resolveMetricMetadata(ctx, req.CompositeQuery.Queries, req.Start, req.End)
if err != nil {
return nil, err
}
@@ -304,7 +304,7 @@ func (q *querier) populateQBEvent(event *qbtypes.QBEvent, queries []qbtypes.Quer
// resolved: never-seen metrics and dormant metrics (seen but no data in
// the query window).
// - err: Internal when a metadata fetch fails.
func (q *querier) resolveMetricMetadata(ctx context.Context, orgID valuer.UUID, queries []qbtypes.QueryEnvelope, start, end uint64) (missingMetricQueries []string, metricWarnings []string, err error) {
func (q *querier) resolveMetricMetadata(ctx context.Context, queries []qbtypes.QueryEnvelope, start, end uint64) (missingMetricQueries []string, metricWarnings []string, err error) {
metricNames := make([]string, 0)
for idx := range queries {
if queries[idx].Type != qbtypes.QueryTypeBuilder {
@@ -325,7 +325,7 @@ func (q *querier) resolveMetricMetadata(ctx context.Context, orgID valuer.UUID,
return nil, nil, nil
}
metricTemporality, metricTypes, reducedMetricsSet, err := q.metadataStore.FetchTemporalityAndTypeMulti(ctx, orgID, start, end, metricNames...)
metricTemporality, metricTypes, err := q.metadataStore.FetchTemporalityAndTypeMulti(ctx, start, end, metricNames...)
if err != nil {
q.logger.WarnContext(ctx, "failed to fetch metric temporality", errors.Attr(err), slog.Any("metrics", metricNames))
return nil, nil, errors.NewInternalf(errors.CodeInternal, "failed to fetch metrics temporality")
@@ -362,9 +362,6 @@ func (q *querier) resolveMetricMetadata(ctx context.Context, orgID valuer.UUID,
if err := spec.Aggregations[i].ValidateForType(); err != nil {
return nil, nil, err
}
if reducedMetricsSet[spec.Aggregations[i].MetricName] {
spec.Aggregations[i].Reduced = true
}
presentAggregations = append(presentAggregations, spec.Aggregations[i])
}
if len(presentAggregations) == 0 {

View File

@@ -2136,12 +2136,12 @@ func (t *telemetryMetaStore) GetAllValues(ctx context.Context, fieldValueSelecto
return values, complete, nil
}
func (t *telemetryMetaStore) FetchTemporality(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricName string) (metrictypes.Temporality, error) {
func (t *telemetryMetaStore) FetchTemporality(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricName string) (metrictypes.Temporality, error) {
if metricName == "" {
return metrictypes.Unknown, errors.Newf(errors.TypeInternal, errors.CodeInternal, "metric name cannot be empty")
}
temporalityMap, err := t.FetchTemporalityMulti(ctx, orgID, queryTimeRangeStartTs, queryTimeRangeEndTs, metricName)
temporalityMap, err := t.FetchTemporalityMulti(ctx, queryTimeRangeStartTs, queryTimeRangeEndTs, metricName)
if err != nil {
return metrictypes.Unknown, err
}
@@ -2154,27 +2154,25 @@ func (t *telemetryMetaStore) FetchTemporality(ctx context.Context, orgID valuer.
return temporality, nil
}
func (t *telemetryMetaStore) FetchTemporalityMulti(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, error) {
temporalities, _, _, err := t.FetchTemporalityAndTypeMulti(ctx, orgID, queryTimeRangeStartTs, queryTimeRangeEndTs, metricNames...)
func (t *telemetryMetaStore) FetchTemporalityMulti(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, error) {
temporalities, _, err := t.FetchTemporalityAndTypeMulti(ctx, queryTimeRangeStartTs, queryTimeRangeEndTs, metricNames...)
return temporalities, err
}
func (t *telemetryMetaStore) FetchTemporalityAndTypeMulti(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, map[string]metrictypes.Type, map[string]bool, error) {
func (t *telemetryMetaStore) FetchTemporalityAndTypeMulti(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, map[string]metrictypes.Type, error) {
if len(metricNames) == 0 {
return make(map[string]metrictypes.Temporality), make(map[string]metrictypes.Type), make(map[string]bool), nil
return make(map[string]metrictypes.Temporality), make(map[string]metrictypes.Type), nil
}
reductionEnabled := t.fl.BooleanOrEmpty(ctx, flagger.FeatureEnableMetricsReduction, featuretypes.NewFlaggerEvaluationContext(orgID))
temporalities := make(map[string]metrictypes.Temporality)
types := make(map[string]metrictypes.Type)
metricsTemporality, metricTypes, reduced, err := t.fetchMetricsTemporalityAndType(ctx, queryTimeRangeStartTs, queryTimeRangeEndTs, reductionEnabled, metricNames...)
metricsTemporality, metricTypes, err := t.fetchMetricsTemporalityAndType(ctx, queryTimeRangeStartTs, queryTimeRangeEndTs, metricNames...)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
meterMetricsTemporality, meterMetricsTypes, err := t.fetchMeterSourceMetricsTemporalityAndType(ctx, metricNames...)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
// For metrics not found in the database, set to Unknown
@@ -2199,10 +2197,10 @@ func (t *telemetryMetaStore) FetchTemporalityAndTypeMulti(ctx context.Context, o
}
}
return temporalities, types, reduced, nil
return temporalities, types, nil
}
func (t *telemetryMetaStore) fetchMetricsTemporalityAndType(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, reductionEnabled bool, metricNames ...string) (map[string][]metrictypes.Temporality, map[string]metrictypes.Type, map[string]bool, error) {
func (t *telemetryMetaStore) fetchMetricsTemporalityAndType(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string][]metrictypes.Temporality, map[string]metrictypes.Type, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalMetrics.StringValue(),
instrumentationtypes.CodeNamespace: "metadata",
@@ -2210,58 +2208,48 @@ func (t *telemetryMetaStore) fetchMetricsTemporalityAndType(ctx context.Context,
})
temporalities := make(map[string][]metrictypes.Temporality)
types := make(map[string]metrictypes.Type)
reduced := make(map[string]bool)
adjustedStartTs, adjustedEndTs, tsTableName, _ := telemetrymetrics.WhichTSTableToUse(queryTimeRangeStartTs, queryTimeRangeEndTs, false, nil)
adjustedStartTs, adjustedEndTs, tsTableName, _ := telemetrymetrics.WhichTSTableToUse(queryTimeRangeStartTs, queryTimeRangeEndTs, nil)
cols := []string{"metric_name", "temporality", "any(type) AS type", "any(is_monotonic) as is_monotonic"}
// Build query to fetch temporality for all metrics
// We use attr_string_value where attr_name = '__temporality__'
// Note: The columns are mixed in the current data - temporality column contains metric_name
// and metric_name column contains temporality value, so we use the correct mapping
sb := sqlbuilder.Select(
"metric_name",
"temporality",
"any(type) AS type",
"any(is_monotonic) as is_monotonic",
).
From(t.metricsDBName + "." + tsTableName)
// When reduction is enabled, fold the reduced-catalog presence check into the
// same query so a metric's reduced status comes back in one round trip.
var reducedArgs []any
if reductionEnabled {
rs := sqlbuilder.NewSelectBuilder()
rs.Select("metric_name")
rs.From(t.metricsDBName + "." + telemetrymetrics.TimeseriesV4ReducedTableName)
rs.Where(rs.In("metric_name", metricNames), rs.GTE("unix_milli", adjustedStartTs), rs.LT("unix_milli", adjustedEndTs))
rs.GroupBy("metric_name")
rsQuery, rsArgs := rs.BuildWithFlavor(sqlbuilder.ClickHouse)
cols = append(cols, fmt.Sprintf("metric_name GLOBAL IN (%s) AS reduced", rsQuery))
reducedArgs = rsArgs
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select(cols...)
sb.From(t.metricsDBName + "." + tsTableName)
// Filter by metric names (in the temporality column due to data mix-up)
sb.Where(
sb.In("metric_name", metricNames),
sb.GTE("unix_milli", adjustedStartTs),
sb.LT("unix_milli", adjustedEndTs),
)
sb.GroupBy("metric_name", "temporality")
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse, reducedArgs...)
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
t.logger.DebugContext(ctx, "fetching metric temporality", slog.String("query", query), slog.Any("args", args))
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, nil, nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to fetch metric temporality")
return nil, nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to fetch metric temporality")
}
defer rows.Close()
// Process results
for rows.Next() {
var metricName string
var temporality metrictypes.Temporality
var metricType metrictypes.Type
var isMonotonic bool
var isReduced uint8
dest := []any{&metricName, &temporality, &metricType, &isMonotonic}
if reductionEnabled {
dest = append(dest, &isReduced)
}
if err := rows.Scan(dest...); err != nil {
return nil, nil, nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to scan temporality result")
if err := rows.Scan(&metricName, &temporality, &metricType, &isMonotonic); err != nil {
return nil, nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to scan temporality result")
}
if temporality != metrictypes.Unknown {
temporalities[metricName] = append(temporalities[metricName], temporality)
@@ -2270,15 +2258,12 @@ func (t *telemetryMetaStore) fetchMetricsTemporalityAndType(ctx context.Context,
metricType = metrictypes.GaugeType
}
types[metricName] = metricType
if isReduced != 0 {
reduced[metricName] = true
}
}
if err := rows.Err(); err != nil {
return nil, nil, nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error iterating over metrics temporality rows")
return nil, nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "error iterating over metrics temporality rows")
}
return temporalities, types, reduced, nil
return temporalities, types, nil
}
func (t *telemetryMetaStore) fetchMeterSourceMetricsTemporalityAndType(ctx context.Context, metricNames ...string) (map[string]metrictypes.Temporality, map[string]metrictypes.Type, error) {

View File

@@ -1,157 +0,0 @@
package telemetrymetrics
import (
"context"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/require"
)
func reducedQuery(metric string, ty metrictypes.Type, temp metrictypes.Temporality, ta metrictypes.TimeAggregation, sa metrictypes.SpaceAggregation) qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation] {
return qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Signal: telemetrytypes.SignalMetrics,
StepInterval: qbtypes.Step{Duration: 5 * time.Minute},
Aggregations: []qbtypes.MetricAggregation{{
MetricName: metric,
Type: ty,
Temporality: temp,
TimeAggregation: ta,
SpaceAggregation: sa,
Reduced: true,
}},
}
}
func TestReducedStatementBuilder(t *testing.T) {
cases := []struct {
name string
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]
expected qbtypes.Statement
}{
{
name: "gauge_sum_latest",
query: reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationLatest, metrictypes.SpaceAggregationSum),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, anyLast(last) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, sum(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, argMax(value, unix_milli) AS per_series_value FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`sum_last`, computed_at) AS value FROM signoz_metrics.distributed_samples_v4_reduced_last_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, sum(per_series_value) AS value FROM __temporal_aggregation_cte GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "unspecified", false, "test.metric", uint64(1746999900000), uint64(1747172760000), 0, "test.metric", uint64(1746999900000), uint64(1747172760000), "test.metric", uint64(1746999900000), uint64(1747172760000), false},
},
},
{
name: "gauge_avg_avg",
query: reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationAvg, metrictypes.SpaceAggregationAvg),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, sum(sum) / sum(count) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, avg(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, avg(value) AS per_series_value, avg(weight) AS per_series_weight FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`sum_last`, computed_at) AS value, argMax(`count_series`, computed_at) AS weight FROM signoz_metrics.distributed_samples_v4_reduced_last_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, sum(per_series_value) / sum(per_series_weight) AS value FROM __temporal_aggregation_cte GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "unspecified", false, "test.metric", uint64(1746999900000), uint64(1747172760000), 0, "test.metric", uint64(1746999900000), uint64(1747172760000), "test.metric", uint64(1746999900000), uint64(1747172760000), false},
},
},
{
name: "gauge_min_min",
query: reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationMin, metrictypes.SpaceAggregationMin),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, min(min) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, min(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, min(value) AS per_series_value FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`min`, computed_at) AS value FROM signoz_metrics.distributed_samples_v4_reduced_last_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, min(per_series_value) AS value FROM __temporal_aggregation_cte GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "unspecified", false, "test.metric", uint64(1746999900000), uint64(1747172760000), 0, "test.metric", uint64(1746999900000), uint64(1747172760000), "test.metric", uint64(1746999900000), uint64(1747172760000), false},
},
},
{
name: "gauge_max_max",
query: reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationMax, metrictypes.SpaceAggregationMax),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, max(max) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, max(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, max(value) AS per_series_value FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`max`, computed_at) AS value FROM signoz_metrics.distributed_samples_v4_reduced_last_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, max(per_series_value) AS value FROM __temporal_aggregation_cte GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "unspecified", false, "test.metric", uint64(1746999900000), uint64(1747172760000), 0, "test.metric", uint64(1746999900000), uint64(1747172760000), "test.metric", uint64(1746999900000), uint64(1747172760000), false},
},
},
{
name: "counter_sum_rate",
query: reducedQuery("test.metric", metrictypes.SumType, metrictypes.Cumulative, metrictypes.TimeAggregationRate, metrictypes.SpaceAggregationSum),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT ts, multiIf(row_number() OVER rate_window = 1, nan, (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) < 0, per_series_value / (ts - lagInFrame(ts, 1) OVER rate_window), (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) / (ts - lagInFrame(ts, 1) OVER rate_window)) AS per_series_value FROM (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, max(max) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)), __spatial_aggregation_cte AS (SELECT ts, sum(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, sum(value) / 300 AS per_series_value FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`sum`, computed_at) AS value FROM signoz_metrics.distributed_samples_v4_reduced_sum_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, sum(per_series_value) AS value FROM __temporal_aggregation_cte GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "cumulative", false, "test.metric", uint64(1746999600000), uint64(1747172760000), 0, "test.metric", uint64(1746999600000), uint64(1747172760000), "test.metric", uint64(1746999600000), uint64(1747172760000), false},
},
},
{
name: "counter_avg_increase",
query: reducedQuery("test.metric", metrictypes.SumType, metrictypes.Cumulative, metrictypes.TimeAggregationIncrease, metrictypes.SpaceAggregationAvg),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT ts, multiIf(row_number() OVER rate_window = 1, nan, (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) < 0, per_series_value, per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) AS per_series_value FROM (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, max(max) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)), __spatial_aggregation_cte AS (SELECT ts, avg(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, sum(value) AS per_series_value, avg(weight) AS per_series_weight FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`sum`, computed_at) AS value, argMax(`count_series`, computed_at) AS weight FROM signoz_metrics.distributed_samples_v4_reduced_sum_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, sum(per_series_value) / sum(per_series_weight) AS value FROM __temporal_aggregation_cte GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "cumulative", false, "test.metric", uint64(1746999600000), uint64(1747172760000), 0, "test.metric", uint64(1746999600000), uint64(1747172760000), "test.metric", uint64(1746999600000), uint64(1747172760000), false},
},
},
{
name: "counter_min_omitted",
query: reducedQuery("test.metric", metrictypes.SumType, metrictypes.Cumulative, metrictypes.TimeAggregationRate, metrictypes.SpaceAggregationMin),
expected: qbtypes.Statement{
Query: "WITH __temporal_aggregation_cte AS (SELECT ts, multiIf(row_number() OVER rate_window = 1, nan, (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) < 0, per_series_value / (ts - lagInFrame(ts, 1) OVER rate_window), (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) / (ts - lagInFrame(ts, 1) OVER rate_window)) AS per_series_value FROM (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, max(max) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)), __spatial_aggregation_cte AS (SELECT ts, min(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "cumulative", false, "test.metric", uint64(1746999600000), uint64(1747172760000), 0},
},
},
{
name: "counter_max_omitted",
query: reducedQuery("test.metric", metrictypes.SumType, metrictypes.Cumulative, metrictypes.TimeAggregationRate, metrictypes.SpaceAggregationMax),
expected: qbtypes.Statement{
Query: "WITH __temporal_aggregation_cte AS (SELECT ts, multiIf(row_number() OVER rate_window = 1, nan, (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) < 0, per_series_value / (ts - lagInFrame(ts, 1) OVER rate_window), (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) / (ts - lagInFrame(ts, 1) OVER rate_window)) AS per_series_value FROM (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, max(max) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)), __spatial_aggregation_cte AS (SELECT ts, max(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "cumulative", false, "test.metric", uint64(1746999600000), uint64(1747172760000), 0},
},
},
{
name: "histogram_p99",
query: reducedQuery("test.metric", metrictypes.HistogramType, metrictypes.Cumulative, metrictypes.TimeAggregationUnspecified, metrictypes.SpaceAggregationPercentile99),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT ts, `le`, multiIf(row_number() OVER rate_window = 1, nan, (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) < 0, per_series_value / (ts - lagInFrame(ts, 1) OVER rate_window), (per_series_value - lagInFrame(per_series_value, 1) OVER rate_window) / (ts - lagInFrame(ts, 1) OVER rate_window)) AS per_series_value FROM (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, `le`, max(max) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint, JSONExtractString(labels, 'le') AS `le` FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint, `le`) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts, `le` ORDER BY fingerprint, ts) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)), __spatial_aggregation_cte AS (SELECT ts, `le`, sum(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts, `le`) SELECT ts, histogramQuantile(arrayMap(x -> toFloat64(x), groupArray(le)), groupArray(value), 0.990) AS value FROM __spatial_aggregation_cte GROUP BY ts ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, `le`, sum(value) / 300 AS per_series_value FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`sum`, computed_at) AS value FROM signoz_metrics.distributed_samples_v4_reduced_sum_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint, JSONExtractString(labels, 'le') AS `le` FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint, `le`) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts, `le`), __spatial_aggregation_cte AS (SELECT ts, `le`, sum(per_series_value) AS value FROM __temporal_aggregation_cte GROUP BY ts, `le`) SELECT ts, histogramQuantile(arrayMap(x -> toFloat64(x), groupArray(le)), groupArray(value), 0.990) AS value FROM __spatial_aggregation_cte GROUP BY ts ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "cumulative", false, "test.metric", uint64(1746999900000), uint64(1747172760000), 0, "test.metric", uint64(1746999900000), uint64(1747172760000), "test.metric", uint64(1746999900000), uint64(1747172760000), false},
},
},
{
name: "summary_avg",
query: reducedQuery("test.metric", metrictypes.SummaryType, metrictypes.Unspecified, metrictypes.TimeAggregationAvg, metrictypes.SpaceAggregationAvg),
expected: qbtypes.Statement{
Query: "SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, sum(sum) / sum(count) AS per_series_value FROM signoz_metrics.distributed_samples_v4_agg_5m AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.time_series_v4_1day WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND LOWER(temporality) LIKE LOWER(?) AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY fingerprint, ts ORDER BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, avg(per_series_value) AS value FROM __temporal_aggregation_cte WHERE isNaN(per_series_value) = ? GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) UNION ALL SELECT * FROM (WITH __temporal_aggregation_cte AS (SELECT fingerprint, toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(300)) AS ts, avg(value) AS per_series_value, avg(weight) AS per_series_weight FROM (SELECT reduced_fingerprint AS fingerprint, unix_milli, argMax(`sum_last`, computed_at) AS value, argMax(`count_series`, computed_at) AS weight FROM signoz_metrics.distributed_samples_v4_reduced_last_60s WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli < ? GROUP BY reduced_fingerprint, unix_milli) AS points INNER JOIN (SELECT fingerprint FROM signoz_metrics.distributed_time_series_v4_reduced WHERE metric_name IN (?) AND unix_milli >= ? AND unix_milli <= ? AND __normalized = ? GROUP BY fingerprint) AS filtered_time_series ON points.fingerprint = filtered_time_series.fingerprint GROUP BY fingerprint, ts), __spatial_aggregation_cte AS (SELECT ts, sum(per_series_value) / sum(per_series_weight) AS value FROM __temporal_aggregation_cte GROUP BY ts) SELECT * FROM __spatial_aggregation_cte ORDER BY ts) ORDER BY ts",
Args: []any{"test.metric", uint64(1746921600000), uint64(1747172760000), "unspecified", false, "test.metric", uint64(1746999900000), uint64(1747172760000), 0, "test.metric", uint64(1746999900000), uint64(1747172760000), "test.metric", uint64(1746999900000), uint64(1747172760000), false},
},
},
}
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fl, err := flagger.New(context.Background(), instrumentationtest.New().ToProviderSettings(), flagger.Config{}, flagger.MustNewRegistry())
require.NoError(t, err)
sb := NewMetricQueryStatementBuilder(instrumentationtest.New().ToProviderSettings(), telemetrytypestest.NewMockMetadataStore(), fm, cb, fl)
const start, end = uint64(1747000000000), uint64(1747172800000)
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
got, err := sb.Build(context.Background(), start, end, qbtypes.RequestTypeTimeSeries, c.query, nil)
require.NoError(t, err)
require.Equal(t, c.expected.Query, got.Query)
require.Equal(t, c.expected.Args, got.Args)
})
}
t.Run("buffer_recent_window", func(t *testing.T) {
now := time.Now().UnixMilli()
q := reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationLatest, metrictypes.SpaceAggregationSum)
got, err := sb.Build(context.Background(), uint64(now-2*time.Hour.Milliseconds()), uint64(now), qbtypes.RequestTypeTimeSeries, q, nil)
require.NoError(t, err)
require.Contains(t, got.Query, "signoz_metrics.distributed_samples_v4_buffer")
require.Contains(t, got.Query, "signoz_metrics.time_series_v4_buffer")
require.Contains(t, got.Query, "is_reduced")
require.NotContains(t, got.Query, "UNION ALL")
})
t.Run("not_reduced", func(t *testing.T) {
q := reducedQuery("test.metric", metrictypes.GaugeType, metrictypes.Unspecified, metrictypes.TimeAggregationLatest, metrictypes.SpaceAggregationSum)
q.Aggregations[0].Reduced = false
got, err := sb.Build(context.Background(), start, end, qbtypes.RequestTypeTimeSeries, q, nil)
require.NoError(t, err)
require.NotContains(t, got.Query, "UNION ALL")
require.NotContains(t, got.Query, "reduced")
require.NotContains(t, got.Query, "buffer")
})
}

View File

@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"log/slog"
"time"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
@@ -181,30 +180,19 @@ func (b *MetricQueryStatementBuilder) buildPipelineStatement(
query.Aggregations[0].SpaceAggregation = metrictypes.SpaceAggregationSum
}
agg := query.Aggregations[0]
// A reduced metric reads the raw buffer for recent short windows, and
// samples_v4/agg (unioned with the reduced tables) otherwise. The buffer is
// shaped exactly like samples_v4 / time_series_v4, so once the table names are
// chosen the rest of the pipeline is unchanged.
useBuffer := agg.Reduced &&
end-start < oneDayInMilliseconds &&
start >= uint64(time.Now().UnixMilli())-oneDayInMilliseconds
samplesTable, _ := WhichSamplesTableToUse(start, end, agg.Type, agg.TimeAggregation, useBuffer, agg.TableHints)
tsStart, tsEnd, _, tsTable := WhichTSTableToUse(start, end, useBuffer, agg.TableHints)
var timeSeriesCTE string
var timeSeriesCTEArgs []any
var err error
if timeSeriesCTE, timeSeriesCTEArgs, err = b.buildTimeSeriesCTE(ctx, tsStart, tsEnd, query, keys, variables, tsTable); err != nil {
// time_series_cte
// this is applicable for all the queries
if timeSeriesCTE, timeSeriesCTEArgs, err = b.buildTimeSeriesCTE(ctx, start, end, query, keys, variables); err != nil {
return nil, err
}
if qbtypes.CanShortCircuitDelta(query.Aggregations[0]) {
// spatial_aggregation_cte directly for certain delta queries
if frag, args, err := b.buildTemporalAggDeltaFastPath(start, end, query, samplesTable, timeSeriesCTE, timeSeriesCTEArgs); err != nil {
if frag, args, err := b.buildTemporalAggDeltaFastPath(start, end, query, timeSeriesCTE, timeSeriesCTEArgs); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
@@ -212,7 +200,7 @@ func (b *MetricQueryStatementBuilder) buildPipelineStatement(
}
} else {
// temporal_aggregation_cte
if frag, args, err := b.buildTemporalAggregationCTE(ctx, start, end, query, keys, samplesTable, timeSeriesCTE, timeSeriesCTEArgs); err != nil {
if frag, args, err := b.buildTemporalAggregationCTE(ctx, start, end, query, keys, timeSeriesCTE, timeSeriesCTEArgs); err != nil {
return nil, err
} else if frag != "" {
cteFragments = append(cteFragments, frag)
@@ -226,188 +214,18 @@ func (b *MetricQueryStatementBuilder) buildPipelineStatement(
}
}
var reducedFragments []string
var reducedArgs [][]any
if agg.Reduced && !useBuffer {
var tsCTE string
var tsArgs []any
if tsCTE, tsArgs, err = b.buildReducedTimeSeriesCTE(ctx, start, end, query, keys, variables); err != nil {
return nil, err
}
if temporalFrag, temporalArgs, ok := b.buildReducedTemporalAggregationCTE(start, end, query, tsCTE, tsArgs); ok {
spatialFrag, spatialArgs := b.buildReducedSpatialAggregationCTE(query)
reducedFragments = []string{temporalFrag, spatialFrag}
reducedArgs = [][]any{temporalArgs, spatialArgs}
}
}
// reset the query to the original state
query.Aggregations[0].SpaceAggregation = origSpaceAgg
query.Aggregations[0].TimeAggregation = origTimeAgg
query.GroupBy = origGroupBy
mainStmt, err := b.BuildFinalSelect(cteFragments, cteArgs, query)
if err != nil {
return nil, err
}
if reducedFragments == nil {
return mainStmt, nil
}
reducedStmt, err := b.BuildFinalSelect(reducedFragments, reducedArgs, query)
if err != nil {
return nil, err
}
return unionStatements(mainStmt, reducedStmt, query)
}
func unionStatements(main, reduced *qbtypes.Statement, query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]) (*qbtypes.Statement, error) {
orderBy := "ts"
for _, g := range query.GroupBy {
orderBy = fmt.Sprintf("`%s`, ", g.Name) + orderBy
}
q := fmt.Sprintf("SELECT * FROM (%s) UNION ALL SELECT * FROM (%s) ORDER BY %s", main.Query, reduced.Query, orderBy)
args := append(append([]any{}, main.Args...), reduced.Args...)
warnings := append(append([]string{}, main.Warnings...), reduced.Warnings...)
return &qbtypes.Statement{Query: q, Args: args, Warnings: warnings}, nil
}
func (b *MetricQueryStatementBuilder) buildReducedTimeSeriesCTE(
ctx context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
) (string, []any, error) {
sb := sqlbuilder.NewSelectBuilder()
var preparedWhereClause querybuilder.PreparedWhereClause
var err error
if query.Filter != nil && query.Filter.Expression != "" {
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,
FieldKeys: keys,
FullTextColumn: &telemetrytypes.TelemetryFieldKey{Name: "labels"},
Variables: variables,
StartNs: start,
EndNs: end,
})
if err != nil {
return "", nil, err
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, TimeseriesV4ReducedTableName))
sb.Select("fingerprint")
for _, g := range query.GroupBy {
col, err := b.fm.ColumnExpressionFor(ctx, start, end, &g.TelemetryFieldKey, keys)
if err != nil {
return "", nil, err
}
sb.SelectMore(col)
}
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
sb.GTE("unix_milli", start),
sb.LTE("unix_milli", end),
sb.EQ("__normalized", false),
)
if !preparedWhereClause.IsEmpty() {
sb.AddWhereClause(preparedWhereClause.WhereClause)
}
sb.GroupBy("fingerprint")
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
q, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("(%s) AS filtered_time_series", q), args, nil
}
func (b *MetricQueryStatementBuilder) buildReducedTemporalAggregationCTE(
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
timeSeriesCTE string,
timeSeriesCTEArgs []any,
) (string, []any, bool) {
agg := query.Aggregations[0]
stepSec := int64(query.StepInterval.Seconds())
value, weight, ok := ReducedValueColumn(agg.Type, agg.SpaceAggregation)
if !ok {
return "", nil, false
}
// dedup recomputed buckets: latest computed_at wins per (series, 60s bucket)
dedup := sqlbuilder.NewSelectBuilder()
dedup.Select("reduced_fingerprint AS fingerprint", "unix_milli")
dedup.SelectMore(fmt.Sprintf("argMax(%s, computed_at) AS value", value))
if weight != "" {
dedup.SelectMore(fmt.Sprintf("argMax(%s, computed_at) AS weight", weight))
}
dedup.From(fmt.Sprintf("%s.%s", DBName, WhichReducedSamplesTableToUse(agg.Type)))
dedup.Where(
dedup.In("metric_name", agg.MetricName),
dedup.GTE("unix_milli", start),
dedup.LT("unix_milli", end),
)
dedup.GroupBy("reduced_fingerprint", "unix_milli")
dedupQuery, dedupArgs := dedup.BuildWithFlavor(sqlbuilder.ClickHouse)
sb := sqlbuilder.NewSelectBuilder()
sb.Select("fingerprint")
sb.SelectMore(fmt.Sprintf("toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalSecond(%d)) AS ts", stepSec))
for _, g := range query.GroupBy {
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
sb.SelectMore(fmt.Sprintf("%s AS per_series_value", ReducedTimeAggregationColumn(agg.TimeAggregation, stepSec)))
if weight != "" {
// count_series is a series count, not additive over time, so the avg
// denominator is reduced with avg
sb.SelectMore("avg(weight) AS per_series_weight")
}
sb.From(fmt.Sprintf("(%s) AS points", dedupQuery))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
sb.GroupBy("fingerprint", "ts")
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
initArgs := append(append([]any{}, dedupArgs...), timeSeriesCTEArgs...)
q, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse, initArgs...)
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", q), args, true
}
func (b *MetricQueryStatementBuilder) buildReducedSpatialAggregationCTE(
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
) (string, []any) {
spatial := "sum(per_series_value)"
switch query.Aggregations[0].SpaceAggregation {
case metrictypes.SpaceAggregationAvg:
spatial = "sum(per_series_value) / sum(per_series_weight)"
case metrictypes.SpaceAggregationMin:
spatial = "min(per_series_value)"
case metrictypes.SpaceAggregationMax:
spatial = "max(per_series_value)"
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select("ts")
for _, g := range query.GroupBy {
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
sb.SelectMore(spatial + " AS value")
sb.From("__temporal_aggregation_cte")
sb.GroupBy("ts")
sb.GroupBy(querybuilder.GroupByKeys(query.GroupBy)...)
q, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return fmt.Sprintf("__spatial_aggregation_cte AS (%s)", q), args
// final SELECT
return b.BuildFinalSelect(cteFragments, cteArgs, query)
}
func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
samplesTable string,
timeSeriesCTE string,
timeSeriesCTEArgs []any,
) (string, []any, error) {
@@ -424,7 +242,8 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
}
aggCol, err := AggregationColumnForSamplesTable(
samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation,
start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality,
query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints,
)
if err != nil {
return "", nil, err
@@ -441,7 +260,8 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDeltaFastPath(
sb.SelectMore(fmt.Sprintf("%s AS value", aggCol))
sb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
tbl, _ := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
sb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
@@ -461,7 +281,6 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
tsTable string,
) (string, []any, error) {
sb := sqlbuilder.NewSelectBuilder()
@@ -485,7 +304,8 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
}
}
sb.From(fmt.Sprintf("%s.%s", DBName, tsTable))
start, end, _, tbl := WhichTSTableToUse(start, end, query.Aggregations[0].TableHints)
sb.From(fmt.Sprintf("%s.%s", DBName, tbl))
sb.Select("fingerprint")
for _, g := range query.GroupBy {
@@ -511,12 +331,6 @@ func (b *MetricQueryStatementBuilder) buildTimeSeriesCTE(
sb.EQ("__normalized", false),
)
// the buffer holds both raw rows and the reduced catalog rows; the raw read
// only wants the original series
if tsTable == TimeseriesV4BufferLocalTableName {
sb.Where(sb.EQ("is_reduced", false))
}
if !preparedWhereClause.IsEmpty() {
sb.AddWhereClause(preparedWhereClause.WhereClause)
}
@@ -533,23 +347,21 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggregationCTE(
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
_ map[string][]*telemetrytypes.TelemetryFieldKey,
samplesTable string,
timeSeriesCTE string,
timeSeriesCTEArgs []any,
) (string, []any, error) {
if query.Aggregations[0].Temporality == metrictypes.Delta {
return b.buildTemporalAggDelta(ctx, start, end, query, samplesTable, timeSeriesCTE, timeSeriesCTEArgs)
return b.buildTemporalAggDelta(ctx, start, end, query, timeSeriesCTE, timeSeriesCTEArgs)
} else if query.Aggregations[0].Temporality != metrictypes.Multiple {
return b.buildTemporalAggCumulativeOrUnspecified(ctx, start, end, query, samplesTable, timeSeriesCTE, timeSeriesCTEArgs)
return b.buildTemporalAggCumulativeOrUnspecified(ctx, start, end, query, timeSeriesCTE, timeSeriesCTEArgs)
}
return b.buildTemporalAggForMultipleTemporalities(ctx, start, end, query, samplesTable, timeSeriesCTE, timeSeriesCTEArgs)
return b.buildTemporalAggForMultipleTemporalities(ctx, start, end, query, timeSeriesCTE, timeSeriesCTEArgs)
}
func (b *MetricQueryStatementBuilder) buildTemporalAggDelta(
_ context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
samplesTable string,
timeSeriesCTE string,
timeSeriesCTEArgs []any,
) (string, []any, error) {
@@ -566,7 +378,7 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDelta(
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggCol, err := AggregationColumnForSamplesTable(samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation)
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
if err != nil {
return "", nil, err
}
@@ -577,7 +389,8 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggDelta(
sb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
sb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
tbl, _ := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
sb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),
@@ -596,7 +409,6 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
_ context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
samplesTable string,
timeSeriesCTE string,
timeSeriesCTEArgs []any,
) (string, []any, error) {
@@ -612,13 +424,14 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
baseSb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggCol, err := AggregationColumnForSamplesTable(samplesTable, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation)
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
if err != nil {
return "", nil, err
}
baseSb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
baseSb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
tbl, _ := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
baseSb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
baseSb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
baseSb.Where(
baseSb.In("metric_name", query.Aggregations[0].MetricName),
@@ -662,7 +475,6 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggForMultipleTemporalities(
_ context.Context,
start, end uint64,
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
samplesTable string,
timeSeriesCTE string,
timeSeriesCTEArgs []any,
) (string, []any, error) {
@@ -677,11 +489,11 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggForMultipleTemporalities(
sb.SelectMore(fmt.Sprintf("`%s`", g.Name))
}
aggForDeltaTemporality, err := AggregationColumnForSamplesTable(samplesTable, metrictypes.Delta, query.Aggregations[0].TimeAggregation)
aggForDeltaTemporality, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, metrictypes.Delta, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
if err != nil {
return "", nil, err
}
aggForCumulativeTemporality, err := AggregationColumnForSamplesTable(samplesTable, metrictypes.Cumulative, query.Aggregations[0].TimeAggregation)
aggForCumulativeTemporality, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, metrictypes.Cumulative, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
if err != nil {
return "", nil, err
}
@@ -709,7 +521,8 @@ func (b *MetricQueryStatementBuilder) buildTemporalAggForMultipleTemporalities(
sb.SelectMore(expr)
}
sb.From(fmt.Sprintf("%s.%s AS points", DBName, samplesTable))
tbl, _ := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
sb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
sb.JoinWithOption(sqlbuilder.InnerJoin, timeSeriesCTE, "points.fingerprint = filtered_time_series.fingerprint")
sb.Where(
sb.In("metric_name", query.Aggregations[0].MetricName),

View File

@@ -30,17 +30,6 @@ const (
TimeseriesV41weekLocalTableName = "time_series_v4_1week"
AttributesMetadataTableName = "distributed_metadata"
AttributesMetadataLocalTableName = "metadata"
// The buffer holds raw points for ~24h; the reduced tables hold 60s
// aggregates of dropped-label series.
SamplesV4BufferTableName = "distributed_samples_v4_buffer"
SamplesV4BufferLocalTableName = "samples_v4_buffer"
TimeseriesV4BufferTableName = "distributed_time_series_v4_buffer"
TimeseriesV4BufferLocalTableName = "time_series_v4_buffer"
SamplesV4ReducedLastTableName = "distributed_samples_v4_reduced_last_60s"
SamplesV4ReducedSumTableName = "distributed_samples_v4_reduced_sum_60s"
TimeseriesV4ReducedTableName = "distributed_time_series_v4_reduced"
TimeseriesV4ReducedLocalTableName = "time_series_v4_reduced"
)
var (
@@ -60,16 +49,8 @@ var (
// in that order.
func WhichTSTableToUse(
start, end uint64,
useBuffer bool,
tableHints *metrictypes.MetricTableHints,
) (uint64, uint64, string, string) {
// the buffer holds the recent raw window for reduced metrics and has the same
// shape as time_series_v4; round the start to the hour like the v4 table.
if useBuffer {
start = start - (start % (oneHourInMilliseconds))
return start, end, TimeseriesV4BufferTableName, TimeseriesV4BufferLocalTableName
}
// if we have a hint for the table, we need to use it
// the hint will be used to override the default table selection logic
if tableHints != nil {
@@ -168,20 +149,14 @@ func WhichSamplesTableToUse(
start, end uint64,
metricType metrictypes.Type,
timeAggregation metrictypes.TimeAggregation,
useBuffer bool,
tableHints *metrictypes.MetricTableHints,
) (string, string) {
// the buffer holds the recent raw window for reduced metrics; same shape as samples_v4
if useBuffer {
return SamplesV4BufferTableName, SamplesV4BufferLocalTableName
}
// if we have a hint for the table, we need to use it
// the hint will be used to override the default table selection logic.
// SamplesTableName is the distributed name; derive the local via switch.
if tableHints != nil && tableHints.SamplesTableName != "" {
switch tableHints.SamplesTableName {
case SamplesV4TableName, SamplesV4BufferTableName:
case SamplesV4TableName:
return SamplesV4TableName, SamplesV4LocalTableName
case SamplesV4Agg5mTableName:
return SamplesV4Agg5mTableName, SamplesV4Agg5mLocalTableName
@@ -213,10 +188,13 @@ func WhichSamplesTableToUse(
}
func AggregationColumnForSamplesTable(
tableName string,
start, end uint64,
metricType metrictypes.Type,
temporality metrictypes.Temporality,
timeAggregation metrictypes.TimeAggregation,
tableHints *metrictypes.MetricTableHints,
) (string, error) {
tableName, _ := WhichSamplesTableToUse(start, end, metricType, timeAggregation, tableHints)
var aggregationColumn string
switch temporality {
case metrictypes.Delta:
@@ -224,7 +202,7 @@ func AggregationColumnForSamplesTable(
// although it doesn't make sense to use anyLast, avg, min, max, count on delta metrics,
// we are keeping it here to make sure that query will not be invalid
switch tableName {
case SamplesV4TableName, SamplesV4BufferTableName:
case SamplesV4TableName:
switch timeAggregation {
case metrictypes.TimeAggregationLatest:
aggregationColumn = "anyLast(value)"
@@ -266,7 +244,7 @@ func AggregationColumnForSamplesTable(
// for cumulative metrics, we only support `RATE`/`INCREASE`. The max value in window is
// used to calculate the sum which is then divided by the window size to get the rate
switch tableName {
case SamplesV4TableName, SamplesV4BufferTableName:
case SamplesV4TableName:
switch timeAggregation {
case metrictypes.TimeAggregationLatest:
aggregationColumn = "anyLast(value)"
@@ -306,7 +284,7 @@ func AggregationColumnForSamplesTable(
}
case metrictypes.Unspecified:
switch tableName {
case SamplesV4TableName, SamplesV4BufferTableName:
case SamplesV4TableName:
switch timeAggregation {
case metrictypes.TimeAggregationLatest:
aggregationColumn = "anyLast(value)"
@@ -354,65 +332,6 @@ func AggregationColumnForSamplesTable(
return aggregationColumn, nil
}
// WhichReducedSamplesTableToUse returns the 60s reduced samples table for a metric
// type: the last_60s table for gauge-like series, the sum_60s table for counters
// and histograms.
func WhichReducedSamplesTableToUse(metricType metrictypes.Type) string {
if metricType == metrictypes.SumType || metricType == metrictypes.HistogramType {
return SamplesV4ReducedSumTableName
}
return SamplesV4ReducedLastTableName
}
// ReducedValueColumn returns the reduced value column (and the avg-denominator
// weight) for a space aggregation. The reduced columns are pre-aggregated across
// the original series, so the space aggregation picks the underlying value; the
// sum table only has `sum`, so min/max across series have no column (ok=false).
func ReducedValueColumn(metricType metrictypes.Type, space metrictypes.SpaceAggregation) (value, weight string, ok bool) {
if metricType == metrictypes.SumType || metricType == metrictypes.HistogramType {
switch space {
case metrictypes.SpaceAggregationSum:
return "`sum`", "", true
case metrictypes.SpaceAggregationAvg:
return "`sum`", "`count_series`", true
}
return "", "", false
}
switch space {
case metrictypes.SpaceAggregationSum:
return "`sum_last`", "", true
case metrictypes.SpaceAggregationAvg:
return "`sum_last`", "`count_series`", true
case metrictypes.SpaceAggregationMin:
return "`min`", "", true
case metrictypes.SpaceAggregationMax:
return "`max`", "", true
}
return "", "", false
}
// ReducedTimeAggregationColumn applies the time aggregation to the reduced `value`
// column over the step's 60s buckets. latest uses argMax over the bucket timestamp
// (the buckets have no read order); rate divides the per-step sum by the step.
func ReducedTimeAggregationColumn(timeAggregation metrictypes.TimeAggregation, stepSec int64) string {
switch timeAggregation {
case metrictypes.TimeAggregationLatest:
return "argMax(value, unix_milli)"
case metrictypes.TimeAggregationAvg:
return "avg(value)"
case metrictypes.TimeAggregationMin:
return "min(value)"
case metrictypes.TimeAggregationMax:
return "max(value)"
case metrictypes.TimeAggregationCount:
return "count(value)"
case metrictypes.TimeAggregationRate:
return fmt.Sprintf("sum(value) / %d", stepSec)
default: // sum, increase
return "sum(value)"
}
}
func AggregationQueryForHistogramCountWithParams(param *metrictypes.ComparisonSpaceAggregationParam) (string, error) {
if param == nil {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "no aggregation param provided for histogram count")

View File

@@ -480,9 +480,7 @@ type MetricAggregation struct {
// value filter to apply to the query
ValueFilter *metrictypes.MetricValueFilter `json:"-"`
// reduce to operator for metric scalar requests
ReduceTo ReduceTo `json:"reduceTo,omitzero"`
Reduced bool `json:"-"`
ReduceTo ReduceTo `json:"reduceTo,omitempty"`
}
// Copy creates a deep copy of MetricAggregation.

View File

@@ -4,7 +4,6 @@ import (
"context"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
// MetadataStore is the interface for the telemetry metadata store.
@@ -27,12 +26,12 @@ type MetadataStore interface {
GetAllValues(ctx context.Context, fieldValueSelector *FieldValueSelector) (*TelemetryFieldValues, bool, error)
// FetchTemporality fetches the temporality for metric
FetchTemporality(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricName string) (metrictypes.Temporality, error)
FetchTemporality(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricName string) (metrictypes.Temporality, error)
// FetchTemporalityMulti fetches the temporality for multiple metrics
FetchTemporalityMulti(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, error)
FetchTemporalityMulti(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, error)
FetchTemporalityAndTypeMulti(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, map[string]metrictypes.Type, map[string]bool, error)
FetchTemporalityAndTypeMulti(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, map[string]metrictypes.Type, error)
// ListLogsJSONIndexes lists the JSON indexes for the logs table.
ListLogsJSONIndexes(ctx context.Context, filters ...string) ([]TelemetryFieldKeySkipIndex, error)

View File

@@ -6,7 +6,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/metrictypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
// MockMetadataStore implements the MetadataStore interface for testing purposes.
@@ -17,7 +16,6 @@ type MockMetadataStore struct {
AllValuesMap map[string]*telemetrytypes.TelemetryFieldValues
TemporalityMap map[string]metrictypes.Temporality
TypeMap map[string]metrictypes.Type
ReducedMap map[string]bool
PromotedPathsMap map[string]bool
LogsJSONIndexes []telemetrytypes.TelemetryFieldKeySkipIndex
ColumnEvolutionMetadataMap map[string][]*telemetrytypes.EvolutionEntry
@@ -308,7 +306,7 @@ func (m *MockMetadataStore) SetAllValues(lookupKey string, values *telemetrytype
}
// FetchTemporality fetches the temporality for a metric.
func (m *MockMetadataStore) FetchTemporality(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricName string) (metrictypes.Temporality, error) {
func (m *MockMetadataStore) FetchTemporality(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricName string) (metrictypes.Temporality, error) {
if temporality, exists := m.TemporalityMap[metricName]; exists {
return temporality, nil
}
@@ -316,7 +314,7 @@ func (m *MockMetadataStore) FetchTemporality(ctx context.Context, orgID valuer.U
}
// FetchTemporalityMulti fetches the temporality for multiple metrics.
func (m *MockMetadataStore) FetchTemporalityMulti(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, error) {
func (m *MockMetadataStore) FetchTemporalityMulti(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, error) {
result := make(map[string]metrictypes.Temporality)
for _, metricName := range metricNames {
@@ -331,10 +329,9 @@ func (m *MockMetadataStore) FetchTemporalityMulti(ctx context.Context, orgID val
}
// FetchTemporalityMulti fetches the temporality for multiple metrics.
func (m *MockMetadataStore) FetchTemporalityAndTypeMulti(ctx context.Context, orgID valuer.UUID, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, map[string]metrictypes.Type, map[string]bool, error) {
func (m *MockMetadataStore) FetchTemporalityAndTypeMulti(ctx context.Context, queryTimeRangeStartTs, queryTimeRangeEndTs uint64, metricNames ...string) (map[string]metrictypes.Temporality, map[string]metrictypes.Type, error) {
temporalities := make(map[string]metrictypes.Temporality)
types := make(map[string]metrictypes.Type)
reduced := make(map[string]bool)
for _, metricName := range metricNames {
if temporality, exists := m.TemporalityMap[metricName]; exists {
@@ -347,12 +344,9 @@ func (m *MockMetadataStore) FetchTemporalityAndTypeMulti(ctx context.Context, or
} else {
types[metricName] = metrictypes.UnspecifiedType
}
if m.ReducedMap[metricName] {
reduced[metricName] = true
}
}
return temporalities, types, reduced, nil
return temporalities, types, nil
}
// SetTemporality sets the temporality for a metric in the mock store.