Compare commits

..

54 Commits

Author SHA1 Message Date
Nikhil Soni
e4d11888d9 fix: make normalization declared fields aware 2026-08-20 17:44:13 +05:30
Nikhil Soni
01009eb727 chore: scope attribute named same as declared scope paths 2026-08-20 16:39:55 +05:30
Nikhil Soni
0d7350b2f1 test(queriertraces): cover scope-prefixed attribute resolution
Add a scope attribute whose own name carries a scope. prefix (scope.prefixed)
and a filter case for it. This exercises the targeted scope.<name> metadata
selector: without it the metadata fetch never surfaces a scope.-prefixed key
name and the filter resolves to nothing.

Assisted-by: Claude Opus 4.8
2026-08-19 22:41:58 +05:30
Nikhil Soni
79bfcc987b chore: regenerate openapi specs for scope field context
Add scope to the FieldContext enum in the generated OpenAPI spec and frontend
client (follows enabling FieldContextScope in the enum). Map the new scope value
to undefined in the alert-history suggestion Record so the exhaustive mapping
stays complete.

Assisted-by: Claude Opus 4.8
2026-08-19 19:28:52 +05:30
Nikhil Soni
8bad55bcb3 test(queriertraces): inline the query window in the scope-filter test
Replace the undefined _query_window(now) call with the inline start/end-ms
computation used by the other tests in the suite, fixing the ruff F821 lint
failure. Per the suite's pytest conventions, test modules avoid `_`-prefixed
helpers and inline such logic.

Assisted-by: Claude Opus 4.8
2026-08-19 19:25:27 +05:30
Nikhil Soni
cb05e09a85 refactor(traces): extract lookupIntrinsicOrCalculatedField as a package helper
Move the intrinsic/calculated field lookup out of adjustTraceKey into a
package-level function, consistent with the other helpers in this file.

Assisted-by: Claude Opus 4.8
2026-08-19 19:17:28 +05:30
Nikhil Soni
3bc493175e refactor(traces): resolve intrinsics by context-qualified name in adjustTraceKey
Look up the intrinsic/calculated field by the key's <context>.<name> first, then
its bare name. The qualified lookup lets a scope-context key match the declared
scope.name / scope.version intrinsic instead of the span field that shares its
short name (e.g. {name, scope} is the scope's name, not the span name column),
replacing the scope-specific guard with a generic name resolution.

Assisted-by: Claude Opus 4.8
2026-08-19 19:11:49 +05:30
Nikhil Soni
9f0bbf2209 refactor(traces): treat scope.name/version as intrinsics in adjustTraceKey
Resolve scope keys through the normal intrinsic path instead of skipping
adjustTraceKey. scope.name / scope.version are matched by their full name as
scope-context intrinsics (context preserved, no override), while a scope-context
key is guarded from binding to a span-context intrinsic that shares only its
short name (e.g. {name, scope} must not become the span name column). The
context-blind span<->attribute remapping of legacy fields is untouched.

Assisted-by: Claude Opus 4.8
2026-08-19 19:05:23 +05:30
Nikhil Soni
4c98b1e5b3 refactor(traces): synthesize scope attribute only under scope context
Drop the scope-attribute candidate from the bare (unspecified) and span/trace
synthesize paths in CandidateKeys; a scope attribute is synthesized only for an
explicit scope context, matching main's per-context synthesis. Unqualified keys
absent from metadata no longer probe scope.attributes. Revert the affected
condition/statement builder test expectations accordingly.

Assisted-by: Claude Opus 4.8
2026-08-19 19:00:51 +05:30
Nikhil Soni
1cb343dbd8 refactor(traces): simplify scope resolution to match main's field-mapper structure
Resolve scope keys through a dedicated ColumnExpressionFor branch
(MatchingLogicalFields + ResolveLogicalFields, CandidateKeys fallback) alongside
main's FieldFor switch for non-scope, dropping the resolveReferencedField
refactor. This restores main's non-scope behavior (accurateCastOrNull coercion,
single-candidate attribute resolution) that the refactor had unintentionally
changed, and keeps scope short-name/declared-path/attribute-union handling.

adjustTraceKey uses a single scope early-return; FieldContextScope is enabled in
the field-context enum. query_to_keys still emits a targeted scope.<name>
selector so scope.-prefixed attributes resolve reliably (test added).

Assisted-by: Claude Opus 4.8
2026-08-19 18:37:12 +05:30
Nikhil Soni
82b2331116 Merge remote-tracking branch 'origin/main' into ns/scope 2026-08-19 17:15:15 +05:30
Nikhil Soni
cb2e8f5d35 fix(traces): resolve short scope name to the declared scope path
A scope-context key whose short name matches a declared scope path (e.g.
{name, scope}) must resolve to that declared path (scope.name), not the span
name column and not an undeclared scope attribute. The adjustTraceKey guard
keeps such a key in scope context, and the field mapper's no-metadata synth now
maps the short name to the declared scope path. With a same-named scope
attribute also in metadata, the two union as before.

Assisted-by: Claude Opus 4.8
2026-08-19 11:52:22 +05:30
Nikhil Soni
d8d4c477c2 fix(traces): resolve scope-context intrinsic collision in adjustTraceKey
A scope-context key that shares its name with a span intrinsic (e.g. the
scope's name addressed as {name, scope}) was resolved against the span 'name'
column when no metadata matched. Guard the intrinsic/calculated lookup in
adjustTraceKey so a scope-context key never binds to a non-scope intrinsic;
the intentionally context-blind span<->attribute remapping of legacy fields is
left untouched. This supersedes the AdjustKey change in collision.go, which is
reverted to keep the shared query builder generic.

Assisted-by: Claude Opus 4.8
2026-08-18 20:08:16 +05:30
Nikhil Soni
54c31332bb test: cover scope select union of declared path and same-named attribute
Add ColumnExpressionFor cases for scope names that collide with a declared
scope path: a short name under scope context binds to the declared path, unions
a same-named scope attribute when present, and the full scope.<x> spelling under
explicit scope context addresses the declared path alone. Add the matching
statement-builder list-query case asserting the full union SELECT.

Assisted-by: Claude Opus 4.8
2026-08-18 19:49:05 +05:30
Nikhil Soni
050f5405e9 refactor: simplify the key resolution 2026-08-18 18:53:05 +05:30
Nikhil Soni
dd97cc3bf0 Merge remote-tracking branch 'origin/main' into ns/scope
# Conflicts:
#	pkg/telemetryschema/tracestelemetryschema/condition_builder.go
#	pkg/telemetryschema/tracestelemetryschema/field_mapper.go
#	pkg/telemetryschema/tracestelemetryschema/test_data.go
2026-08-18 17:31:54 +05:30
Nikhil Soni
e3d39386f9 chore: remove evolution logic from scope exists 2026-08-18 17:09:03 +05:30
Nikhil Soni
3205551b63 refactor: move scope exist expr to signal specific module 2026-08-18 14:55:38 +05:30
Nikhil Soni
7aafd63d11 fix: revert normz. fix due to large blast radius 2026-08-18 12:17:31 +05:30
Nikhil Soni
933c009093 fix: normalization consistant and wrong context override removed 2026-08-18 12:16:04 +05:30
Nikhil Soni
cff401ad79 fix: mark instrinsic only when given context doesn't exist 2026-08-17 19:35:54 +05:30
Nikhil Soni
abefa35fde fix: handle exists expr after recent refactor 2026-08-11 22:17:14 +05:30
Nikhil Soni
5c5a4a7a3f Merge remote-tracking branch 'origin/main' into ns/scope 2026-08-11 17:34:30 +05:30
Nikhil Soni
883e9492d6 Merge remote-tracking branch 'origin/main' into ns/scope 2026-08-10 15:01:20 +05:30
Nikhil Soni
ab3b88966e Merge remote-tracking branch 'origin/main' into ns/scope
# Conflicts:
#	pkg/telemetryschema/tracestelemetryschema/field_mapper.go
#	tests/conftest.py
#	tests/integration/tests/querier/04_traces.py
2026-08-05 16:44:41 +05:30
Nikhil Soni
08ebc37109 fix: handle fixed scope.name and scope.version fields to work without scope 2026-07-02 19:04:41 +05:30
Nikhil Soni
aa5a1c5e62 chore: update collector version 2026-07-02 14:46:07 +05:30
Nikhil Soni
7b34a47ac5 chore: run formatting on tests 2026-07-02 14:41:33 +05:30
Nikhil Soni
b4b2d7bb66 test: add test to show cross context matching 2026-06-24 18:34:38 +05:30
Nikhil Soni
e16416475b refactor: drop unused fields 2026-06-24 18:07:47 +05:30
Nikhil Soni
0ea7c1ae6e test: add more cases for scope name 2026-06-24 18:05:50 +05:30
Nikhil Soni
a023c8ed4a test: add integration test for scope fields 2026-06-24 15:25:17 +05:30
Nikhil Soni
a73ae62cd1 Merge remote-tracking branch 'origin/main' into ns/scope 2026-06-24 13:02:18 +05:30
Nikhil Soni
ec6fb58052 chore: add more tests 2026-06-24 12:37:42 +05:30
Nikhil Soni
d3d13eb7ff fix: remove handling of normalized properties for scope
Otherwise it will be impossible to query if scope attribute also
exists with same name - name and version
2026-05-21 11:19:22 +05:30
Nikhil Soni
782de2b210 fix: use correct error type for internal issues 2026-05-20 15:32:26 +05:30
Nikhil Soni
d3c38693f3 fix: allow 'scope.' prefix for keys with other context 2026-05-20 15:30:25 +05:30
Nikhil Soni
8791df3697 fix: avoid removing context prefix to support attr with prefix 2026-05-19 19:14:23 +05:30
Nikhil Soni
eb719c3d0d fix: use key selector with context prefix 2026-05-19 18:56:00 +05:30
Nikhil Soni
f10435c210 Merge remote-tracking branch 'origin' into ns/scope 2026-05-19 13:55:36 +05:30
Nikhil Soni
f3f1e9cb59 chore: add tests for denormalized field name as well 2026-05-19 13:55:25 +05:30
Nikhil Soni
d0370ce3ef fix: handle fields with included context for scope (select clause) 2026-05-14 17:02:56 +05:30
Nikhil Soni
d169761e65 Merge remote-tracking branch 'origin/main' into ns/scope 2026-05-14 11:50:33 +05:30
Nikhil Soni
87864ef5d4 chore: remove duplicates from .gitignore 2026-05-11 15:45:32 +05:30
Nikhil Soni
2e0bc8998e chore: use name as key name for scope instead of scope.name 2026-05-11 15:40:45 +05:30
Nikhil Soni
7e1f4aa50d Merge remote-tracking branch 'origin/main' into ns/scope 2026-05-11 14:27:19 +05:30
Nikhil Soni
35da39247c Merge branch 'main' into ns/scope 2026-05-07 17:41:11 +05:30
Nikhil Soni
ceccc47a34 fix: fix test for case without resource filter 2026-05-07 16:04:03 +05:30
Nikhil Soni
23da5e22ec Merge branch 'main' into ns/scope 2026-05-07 13:27:34 +05:30
Nikhil Soni
4c1b479149 chore: add tests for scope fields 2026-04-28 20:27:10 +05:30
Nikhil Soni
f72204a8b2 refactor: simplify field mapper for scope 2026-04-28 20:26:37 +05:30
Nikhil Soni
deb3f385fa chore: remove underscore version of scope fields 2026-04-23 10:26:55 +05:30
Nikhil Soni
77ce5f86b1 fix: use scope as json field instead with name and version 2026-04-23 01:15:02 +05:30
Nikhil Soni
ff211de441 feat: add support for scope fields in traces 2026-04-14 10:45:08 +05:30
60 changed files with 944 additions and 2079 deletions

View File

@@ -62,7 +62,6 @@ jobs:
- role
- rootuser
- savedview
- semconvfamilies
- serviceaccount
- spanmapper
- querier_json_body

View File

@@ -8807,6 +8807,7 @@ components:
- span
- trace
- resource
- scope
- attribute
- body
- ""

View File

@@ -3492,6 +3492,7 @@ export enum TelemetrytypesFieldContextDTO {
span = 'span',
trace = 'trace',
resource = 'resource',
scope = 'scope',
attribute = 'attribute',
body = 'body',
'' = '',

View File

@@ -1,3 +0,0 @@
<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path fill="#0078D4" d="M22.379 23.343a1.62 1.62 0 0 0 1.536-2.14v.002L17.35 1.76A1.62 1.62 0 0 0 15.816.657H8.184A1.62 1.62 0 0 0 6.65 1.76L.086 21.204a1.62 1.62 0 0 0 1.536 2.139h4.741a1.62 1.62 0 0 0 1.535-1.103l.977-2.892 4.947 3.675c.28.208.618.32.966.32m-3.084-12.531 3.624 10.739a.54.54 0 0 1-.51.713v-.001h-.03a.54.54 0 0 1-.322-.106l-9.287-6.9h4.853m6.313 7.006c.116-.326.13-.694.007-1.058L9.79 1.76a1.722 1.722 0 0 0-.007-.02h6.034a.54.54 0 0 1 .512.366l6.562 19.445a.54.54 0 0 1-.338.684"/>
</svg>

Before

Width:  |  Height:  |  Size: 583 B

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 7.3 KiB

View File

@@ -1,5 +0,0 @@
<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<title>Dify</title>
<path d="M7.043 6.487c1.635 0 2.241-1.003 2.241-2.243S8.681 2 7.044 2C5.405 2 4.801 3.003 4.801 4.244c0 1.24.604 2.243 2.241 2.243z" fill="#03F" />
<path d="M14.883 6.97v1.443h-3.679v3.203h3.68v8.012H8.801V8.41h-8v3.203h4.48v8.012H0v3.203h24v-3.203h-5.6v-8.012H24V8.41h-5.6V5.206H24V2.003h-4.161a4.97 4.97 0 00-4.961 4.967h.005z" fill="#03F" />
</svg>

Before

Width:  |  Height:  |  Size: 447 B

View File

@@ -1 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" viewBox="0 0 115.28 122.88"><path d="M25.38 57h64.88V37.34H69.59c-2.17 0-5.19-1.17-6.62-2.6s-2.3-4.01-2.3-6.17V7.64H8.15c-.18 0-.32.09-.41.18-.15.1-.19.23-.19.42v106.45c0 .14.09.32.18.41.09.14.28.18.41.18h81.51c.18 0 .17-.09.27-.18.14-.09.33-.28.33-.41v-11.16H25.38c-4.14 0-7.56-3.4-7.56-7.56V64.55c0-4.15 3.4-7.55 7.56-7.55m4.11 11.38h7.43v18.15h11.63v5.92H29.49zm20.4 12.05c0-3.93 1.09-6.99 3.28-9.17 2.19-2.19 5.24-3.28 9.15-3.28 4.01 0 7.09 1.08 9.26 3.22 2.17 2.15 3.25 5.16 3.25 9.04 0 2.81-.47 5.11-1.42 6.91q-1.425 2.7-4.11 4.2t-6.69 1.5c-2.71 0-4.96-.43-6.74-1.29-1.78-.87-3.22-2.23-4.32-4.11-1.11-1.87-1.66-4.21-1.66-7.02m7.42.01c0 2.43.45 4.17 1.36 5.23s2.14 1.59 3.7 1.59c1.6 0 2.84-.52 3.71-1.56.88-1.04 1.32-2.9 1.32-5.6 0-2.26-.46-3.92-1.37-4.96-.92-1.05-2.16-1.57-3.73-1.57-1.5 0-2.71.53-3.62 1.59-.91 1.08-1.37 2.83-1.37 5.28m33.11 3.3v-5.01h11.49v10.23c-2.2 1.5-4.15 2.53-5.83 3.07-1.69.54-3.7.81-6.02.81-2.86 0-5.19-.49-6.99-1.46s-3.19-2.42-4.18-4.35c-.99-1.92-1.48-4.13-1.48-6.63 0-2.63.54-4.91 1.62-6.85s2.67-3.41 4.76-4.42c1.63-.78 3.83-1.17 6.58-1.17 2.66 0 4.64.24 5.96.72s2.41 1.23 3.28 2.24 1.52 2.3 1.96 3.85l-7.16 1.29c-.3-.91-.8-1.61-1.5-2.09-.71-.49-1.6-.73-2.7-.73-1.62 0-2.92.57-3.89 1.7s-1.45 2.92-1.45 5.37c0 2.6.49 4.46 1.47 5.57.97 1.11 2.34 1.68 4.09 1.68q1.245 0 2.37-.36c.75-.24 1.61-.65 2.59-1.22v-2.25h-4.97zM97.79 57h9.93c4.16 0 7.56 3.41 7.56 7.56v31.42c0 4.15-3.41 7.56-7.56 7.56h-9.93v13.55c0 1.61-.65 3.04-1.7 4.1a5.74 5.74 0 0 1-4.1 1.7H5.81a5.74 5.74 0 0 1-4.1-1.7 5.74 5.74 0 0 1-1.7-4.1V5.85c0-1.61.65-3.04 1.7-4.1a5.8 5.8 0 0 1 4.1-1.7h58.72c.13-.05.27-.05.41-.05.64 0 1.29.28 1.75.69h.09c.09.05.14.09.23.18L97 31.23c.51.51.88 1.2.88 1.98 0 .23-.05.41-.09.65zM67.52 27.97V8.94l21.43 21.7H70.19c-.74 0-1.38-.32-1.89-.78-.46-.46-.78-1.15-.78-1.89" style="fill:#D1D5DB;fill-rule:evenodd;clip-rule:evenodd"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" viewBox="0 0 115.28 122.88"><path d="M25.38 57h64.88V37.34H69.59c-2.17 0-5.19-1.17-6.62-2.6s-2.3-4.01-2.3-6.17V7.64H8.15c-.18 0-.32.09-.41.18-.15.1-.19.23-.19.42v106.45c0 .14.09.32.18.41.09.14.28.18.41.18h81.51c.18 0 .17-.09.27-.18.14-.09.33-.28.33-.41v-11.16H25.38c-4.14 0-7.56-3.4-7.56-7.56V64.55c0-4.15 3.4-7.55 7.56-7.55m4.11 11.38h7.43v18.15h11.63v5.92H29.49zm20.4 12.05c0-3.93 1.09-6.99 3.28-9.17 2.19-2.19 5.24-3.28 9.15-3.28 4.01 0 7.09 1.08 9.26 3.22 2.17 2.15 3.25 5.16 3.25 9.04 0 2.81-.47 5.11-1.42 6.91q-1.425 2.7-4.11 4.2t-6.69 1.5c-2.71 0-4.96-.43-6.74-1.29-1.78-.87-3.22-2.23-4.32-4.11-1.11-1.87-1.66-4.21-1.66-7.02m7.42.01c0 2.43.45 4.17 1.36 5.23s2.14 1.59 3.7 1.59c1.6 0 2.84-.52 3.71-1.56.88-1.04 1.32-2.9 1.32-5.6 0-2.26-.46-3.92-1.37-4.96-.92-1.05-2.16-1.57-3.73-1.57-1.5 0-2.71.53-3.62 1.59-.91 1.08-1.37 2.83-1.37 5.28m33.11 3.3v-5.01h11.49v10.23c-2.2 1.5-4.15 2.53-5.83 3.07-1.69.54-3.7.81-6.02.81-2.86 0-5.19-.49-6.99-1.46s-3.19-2.42-4.18-4.35c-.99-1.92-1.48-4.13-1.48-6.63 0-2.63.54-4.91 1.62-6.85s2.67-3.41 4.76-4.42c1.63-.78 3.83-1.17 6.58-1.17 2.66 0 4.64.24 5.96.72s2.41 1.23 3.28 2.24 1.52 2.3 1.96 3.85l-7.16 1.29c-.3-.91-.8-1.61-1.5-2.09-.71-.49-1.6-.73-2.7-.73-1.62 0-2.92.57-3.89 1.7s-1.45 2.92-1.45 5.37c0 2.6.49 4.46 1.47 5.57.97 1.11 2.34 1.68 4.09 1.68q1.245 0 2.37-.36c.75-.24 1.61-.65 2.59-1.22v-2.25h-4.97zM97.79 57h9.93c4.16 0 7.56 3.41 7.56 7.56v31.42c0 4.15-3.41 7.56-7.56 7.56h-9.93v13.55c0 1.61-.65 3.04-1.7 4.1a5.74 5.74 0 0 1-4.1 1.7H5.81a5.74 5.74 0 0 1-4.1-1.7 5.74 5.74 0 0 1-1.7-4.1V5.85c0-1.61.65-3.04 1.7-4.1a5.8 5.8 0 0 1 4.1-1.7h58.72c.13-.05.27-.05.41-.05.64 0 1.29.28 1.75.69h.09c.09.05.14.09.23.18L97 31.23c.51.51.88 1.2.88 1.98 0 .23-.05.41-.09.65zM67.52 27.97V8.94l21.43 21.7H70.19c-.74 0-1.38-.32-1.89-.78-.46-.46-.78-1.15-.78-1.89" style="fill-rule:evenodd;clip-rule:evenodd"/></svg>

Before

Width:  |  Height:  |  Size: 1.9 KiB

After

Width:  |  Height:  |  Size: 1.9 KiB

View File

@@ -1,3 +0,0 @@
<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path fill="#9CA3AF" d="M23.922 16.997C23.061 18.492 18.063 22.02 12 22.02 5.937 22.02.939 18.492.078 16.997A.641.641 0 0 1 0 16.741v-2.869a.883.883 0 0 1 .053-.22c.372-.935 1.347-2.292 2.605-2.656.167-.429.414-1.055.644-1.517a10.098 10.098 0 0 1-.052-1.086c0-1.331.282-2.499 1.132-3.368.397-.406.89-.717 1.474-.952C7.255 2.937 9.248 1.98 11.978 1.98c2.731 0 4.767.957 6.166 2.093.584.235 1.077.546 1.474.952.85.869 1.132 2.037 1.132 3.368 0 .368-.014.733-.052 1.086.23.462.477 1.088.644 1.517 1.258.364 2.233 1.721 2.605 2.656a.841.841 0 0 1 .053.22v2.869a.641.641 0 0 1-.078.256Zm-11.75-5.992h-.344a4.359 4.359 0 0 1-.355.508c-.77.947-1.918 1.492-3.508 1.492-1.725 0-2.989-.359-3.782-1.259a2.137 2.137 0 0 1-.085-.104L4 11.746v6.585c1.435.779 4.514 2.179 8 2.179 3.486 0 6.565-1.4 8-2.179v-6.585l-.098-.104s-.033.045-.085.104c-.793.9-2.057 1.259-3.782 1.259-1.59 0-2.738-.545-3.508-1.492a4.359 4.359 0 0 1-.355-.508Zm2.328 3.25c.549 0 1 .451 1 1v2c0 .549-.451 1-1 1-.549 0-1-.451-1-1v-2c0-.549.451-1 1-1Zm-5 0c.549 0 1 .451 1 1v2c0 .549-.451 1-1 1-.549 0-1-.451-1-1v-2c0-.549.451-1 1-1Zm3.313-6.185c.136 1.057.403 1.913.878 2.497.442.544 1.134.938 2.344.938 1.573 0 2.292-.337 2.657-.751.384-.435.558-1.15.558-2.361 0-1.14-.243-1.847-.705-2.319-.477-.488-1.319-.862-2.824-1.025-1.487-.161-2.192.138-2.533.529-.269.307-.437.808-.438 1.578v.021c0 .265.021.562.063.893Zm-1.626 0c.042-.331.063-.628.063-.894v-.02c-.001-.77-.169-1.271-.438-1.578-.341-.391-1.046-.69-2.533-.529-1.505.163-2.347.537-2.824 1.025-.462.472-.705 1.179-.705 2.319 0 1.211.175 1.926.558 2.361.365.414 1.084.751 2.657.751 1.21 0 1.902-.394 2.344-.938.475-.584.742-1.44.878-2.497Z"/>
</svg>

Before

Width:  |  Height:  |  Size: 1.7 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 25 KiB

View File

@@ -1,24 +0,0 @@
<svg viewBox="0 0 44.8 40" xmlns="http://www.w3.org/2000/svg">
<defs>
<linearGradient id="kong-a" x1="38.204" x2="8.732" y1="18.417" y2="48.543" gradientUnits="userSpaceOnUse">
<stop stop-color="#11A06B" />
<stop offset="1" stop-color="#286FEB" />
</linearGradient>
<linearGradient id="kong-b" x1="38.107" x2="8.635" y1="18.322" y2="48.448" gradientUnits="userSpaceOnUse">
<stop stop-color="#11A06B" />
<stop offset="1" stop-color="#286FEB" />
</linearGradient>
<linearGradient id="kong-c" x1="29.439" x2="-0.033" y1="9.842" y2="39.968" gradientUnits="userSpaceOnUse">
<stop stop-color="#11A06B" />
<stop offset="1" stop-color="#286FEB" />
</linearGradient>
<linearGradient id="kong-d" x1="30.291" x2="0.819" y1="10.676" y2="40.801" gradientUnits="userSpaceOnUse">
<stop stop-color="#11A06B" />
<stop offset="1" stop-color="#286FEB" />
</linearGradient>
</defs>
<path d="m14.7 32.9-1.1 1.4 2.5 3.9-.3 1.8h10.6l.7-1.8-4.2-5.3z" fill="url(#kong-a)" />
<path d="M20.5 9.4 16.7 16l18.6 22-.5 2h8.5l1.5-7.1L24.9 9.4z" fill="url(#kong-b)" />
<path d="m23 4.4-1.8 3.3h4.5l7.7 9.2 4.6-3.8v-2.4l-1.6-2.2 1.2-1.2L28.4 0z" fill="url(#kong-c)" />
<path d="M9.1 22.9H6.6L0 31.3V40h7.1l1.3-1.6 5.5-7.1h7.9l2.4-3.7-8.6-10.2z" fill="url(#kong-d)" />
</svg>

Before

Width:  |  Height:  |  Size: 1.3 KiB

View File

@@ -1,3 +0,0 @@
<svg fill="none" viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg">
<path d="M63 0.018v63.535L38.418 42.25v21.303H0V0l63 .018ZM7.723 55.839h22.972V25.323l24.583 21.725V7.729L7.723 7.716v48.123Z" fill="#37C38F" />
</svg>

Before

Width:  |  Height:  |  Size: 226 B

View File

@@ -19,7 +19,6 @@ export type LogDetailProps = {
onScrollToLog?: (logId: string) => void;
handleOpenInExplorer?: MouseEventHandler;
getContainer?: DrawerProps['getContainer'];
onApplyLogFilter?: (expression: string) => void;
} & Pick<AddToQueryHOCProps, 'onAddToQuery'> &
Partial<Pick<ActionItemProps, 'onClickActionItem'>> &
Pick<DrawerProps, 'onClose'>;

View File

@@ -16,7 +16,6 @@ import {
Link,
} from '@signozhq/icons';
import { useTimezone } from 'providers/Timezone';
import { normalizeTimeToMs } from 'utils/timeUtils';
import { ILog } from 'types/api/logs/log';
import { MouseEvent, MouseEventHandler } from 'react';
import { useCopyToClipboard } from 'react-use';
@@ -68,11 +67,6 @@ function LogDetailsHeader({
},
];
const rawTimestamp = log.date ?? log.timestamp;
const displayTimestamp = Number.isNaN(Number(rawTimestamp))
? rawTimestamp
: normalizeTimeToMs(rawTimestamp);
return (
<div className={styles.header} data-log-detail-ignore="true">
<div className={styles.leftSection}>
@@ -82,7 +76,7 @@ function LogDetailsHeader({
data-testid="log-details-header-timestamp"
>
{formatTimezoneAdjustedTimestamp(
displayTimestamp,
log.date ?? log.timestamp,
DATE_TIME_FORMATS.DASH_DATETIME,
)}
</Typography.Text>

View File

@@ -91,24 +91,6 @@ describe('LogDetail drawer — header (isLogDetailsV2)', () => {
);
});
it('normalizes a nanosecond-epoch timestamp in the header', () => {
localStorage.setItem(LOCALSTORAGE.PREFERRED_TIMEZONE, 'UTC');
// Same instant as mockLog but as epoch nanoseconds (e.g. dashboard list panel).
// Must scale to ms, not render a wildly wrong date.
renderDrawer({
log: {
...mockLog,
date: '1705311930000000000',
timestamp: 1705311930000000000,
},
});
expect(screen.getByTestId('log-details-header-timestamp')).toHaveTextContent(
'Jan 15, 2024 ⎯ 09:45:30',
);
});
it('copies the log link from the ⋯ menu', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });

View File

@@ -75,7 +75,6 @@ function LogDetailInner({
onScrollToLog,
handleOpenInExplorer,
getContainer,
onApplyLogFilter,
}: LogDetailInnerProps): JSX.Element {
const initialContextQuery = useInitialQuery(log);
const [contextQuery, setContextQuery] = useState<Query | undefined>(
@@ -520,7 +519,6 @@ function LogDetailInner({
selectedOptions={options}
listViewPanelSelectedFields={listViewPanelSelectedFields}
handleChangeSelectedView={handleChangeSelectedView}
onApplyLogFilter={onApplyLogFilter}
/>
)}
{!isLogDetailsV2 && selectedView === VIEW_TYPES.JSON && (

View File

@@ -1,11 +1,9 @@
import ROUTES from 'constants/routes';
import { useLocation } from 'react-router-dom';
// v2 is rolled out only on the logs explorer route for now; every other surface
// (dashboards, infra monitoring, etc.) keeps the v1 log details view.
export function useIsLogDetailsV2(): boolean {
const { pathname } = useLocation();
return (
pathname === ROUTES.LOGS_EXPLORER ||
pathname.startsWith(ROUTES.INFRASTRUCTURE_MONITORING_BASE) ||
pathname.startsWith(`${ROUTES.ALL_DASHBOARD}/`)
);
return pathname === ROUTES.LOGS_EXPLORER;
}

View File

@@ -1,280 +0,0 @@
import { renderHook } from '@testing-library/react';
import {
useRecoverFromEmptyPage,
UseRecoverFromEmptyPageParams,
} from '../useRecoverFromEmptyPage';
const REPLACE = { history: 'replace' };
function renderRecovery(
overrides: Partial<UseRecoverFromEmptyPageParams> = {},
): { setPage: jest.Mock; rerender: (next?: unknown) => void } {
const setPage = jest.fn();
const props: UseRecoverFromEmptyPageParams = {
page: 1,
pageSize: 10,
rowCount: 10,
total: 100,
isFetching: false,
setPage,
...overrides,
};
const { rerender } = renderHook(
(next: UseRecoverFromEmptyPageParams) => useRecoverFromEmptyPage(next),
{ initialProps: props },
);
return {
setPage,
rerender: (next?: unknown): void =>
rerender({ ...props, ...(next as Partial<UseRecoverFromEmptyPageParams>) }),
};
}
describe('useRecoverFromEmptyPage', () => {
it('leaves the page alone while it still holds rows', () => {
const { setPage } = renderRecovery({ page: 3, rowCount: 10 });
expect(setPage).not.toHaveBeenCalled();
});
it('leaves the page alone on page 1 with no rows at all', () => {
const { setPage } = renderRecovery({ page: 1, rowCount: 0, total: 0 });
expect(setPage).not.toHaveBeenCalled();
});
it('jumps to the last page that holds data when the page is out of range', () => {
const { setPage } = renderRecovery({
page: 7,
pageSize: 10,
rowCount: 0,
total: 25,
});
expect(setPage).toHaveBeenCalledWith(3, REPLACE);
});
it('replaces the history entry so the back button does not return to the empty page', () => {
const { setPage } = renderRecovery({ page: 4, rowCount: 0, total: 10 });
expect(setPage).toHaveBeenCalledWith(1, REPLACE);
});
it('falls back to page 1 when the total is unknown', () => {
const { setPage } = renderRecovery({ page: 5, rowCount: 0, total: 0 });
expect(setPage).toHaveBeenCalledWith(1, REPLACE);
});
it('steps back one page when the total claims the page should have data', () => {
// total says 100 rows exist, yet page 5 came back empty — step back rather
// than stall on a page the query cannot actually serve.
const { setPage } = renderRecovery({
page: 5,
pageSize: 10,
rowCount: 0,
total: 100,
});
expect(setPage).toHaveBeenCalledWith(4, REPLACE);
});
it('clamps a page below the first one', () => {
const { setPage } = renderRecovery({ page: 0, rowCount: 10 });
expect(setPage).toHaveBeenCalledWith(1, REPLACE);
});
it('falls back to page 1 when pageSize is zero', () => {
const { setPage } = renderRecovery({
page: 5,
pageSize: 0,
rowCount: 0,
total: 100,
});
expect(setPage).toHaveBeenCalledWith(1, REPLACE);
});
it('waits for the request to settle before moving the user', () => {
const { setPage, rerender } = renderRecovery({
page: 3,
rowCount: 0,
total: 10,
isFetching: true,
});
expect(setPage).not.toHaveBeenCalled();
rerender({ page: 3, rowCount: 0, total: 10, isFetching: false });
expect(setPage).toHaveBeenCalledWith(1, REPLACE);
});
it('clamps a page below the first one even when the query failed', () => {
// A negative offset is what made the request fail (400 "offset cannot be
// negative"), so retrying the same page loops forever — clamp regardless.
const { setPage } = renderRecovery({
page: 0,
rowCount: 0,
total: 0,
isDisabled: true,
});
expect(setPage).toHaveBeenCalledWith(1, REPLACE);
});
it('clamps a page below the first one while the query is still in flight', () => {
const { setPage } = renderRecovery({
page: -2,
rowCount: 0,
isFetching: true,
});
expect(setPage).toHaveBeenCalledWith(1, REPLACE);
});
it('keeps the page when the query failed so a retry lands where the user was', () => {
const { setPage } = renderRecovery({
page: 3,
rowCount: 0,
total: 0,
isDisabled: true,
});
expect(setPage).not.toHaveBeenCalled();
});
it('clamps a page below the first one exactly once while the request settles', () => {
// The clamp runs ahead of both gates, so a request settling underneath an
// uncorrected page must not re-issue the same history rewrite.
const { setPage, rerender } = renderRecovery({
page: 0,
rowCount: 0,
total: 0,
isFetching: true,
});
expect(setPage).toHaveBeenCalledTimes(1);
rerender({ page: 0, rowCount: 0, total: 0, isFetching: false });
expect(setPage).toHaveBeenCalledTimes(1);
});
it('stops correcting once the corrected page comes back with rows', () => {
const { setPage, rerender } = renderRecovery({
page: 7,
pageSize: 10,
rowCount: 0,
total: 25,
});
expect(setPage).toHaveBeenCalledWith(3, REPLACE);
// The correction lands: the query refetches, then resolves with the rows page 3 holds.
rerender({ page: 3, pageSize: 10, rowCount: 0, total: 25, isFetching: true });
rerender({
page: 3,
pageSize: 10,
rowCount: 5,
total: 25,
isFetching: false,
});
expect(setPage).toHaveBeenCalledTimes(1);
});
it('does not correct again while the same page is still being observed', () => {
const { setPage, rerender } = renderRecovery({
page: 5,
pageSize: 10,
rowCount: 0,
total: 100,
});
expect(setPage).toHaveBeenCalledTimes(1);
// A refetch cycle that leaves the page untouched — the correction is already in flight.
rerender({
page: 5,
pageSize: 10,
rowCount: 0,
total: 100,
isFetching: true,
});
rerender({
page: 5,
pageSize: 10,
rowCount: 0,
total: 100,
isFetching: false,
});
expect(setPage).toHaveBeenCalledTimes(1);
});
it('gives up on step-backs and jumps to page 1 when the total keeps lying', () => {
// `total` claims 400 rows exist, but every page comes back empty. Walking back one
// page at a time would cost a request per hop, so bail out to page 1 instead.
const { setPage, rerender } = renderRecovery({
page: 40,
pageSize: 10,
rowCount: 0,
total: 400,
});
expect(setPage).toHaveBeenNthCalledWith(1, 39, REPLACE);
rerender({ page: 39, pageSize: 10, rowCount: 0, total: 400 });
expect(setPage).toHaveBeenNthCalledWith(2, 38, REPLACE);
rerender({ page: 38, pageSize: 10, rowCount: 0, total: 400 });
expect(setPage).toHaveBeenNthCalledWith(3, 1, REPLACE);
expect(setPage).toHaveBeenCalledTimes(3);
});
it('corrects again when the user returns to a page that is still empty', () => {
const { setPage, rerender } = renderRecovery({
page: 3,
pageSize: 10,
rowCount: 0,
total: 10,
});
expect(setPage).toHaveBeenNthCalledWith(1, 1, REPLACE);
rerender({ page: 1, pageSize: 10, rowCount: 10, total: 10 });
rerender({ page: 3, pageSize: 10, rowCount: 0, total: 10 });
expect(setPage).toHaveBeenNthCalledWith(2, 1, REPLACE);
});
it('does not re-run the correction when setPage is a fresh function each render', () => {
// The hook reads setPage through a ref, so an inline arrow must not turn the
// ungated `page < 1` clamp into a per-render history rewrite.
const setPage = jest.fn();
const { rerender } = renderHook(
() =>
useRecoverFromEmptyPage({
page: 0,
pageSize: 10,
rowCount: 0,
total: 0,
isFetching: false,
setPage: (nextPage, options): void => setPage(nextPage, options),
}),
{ initialProps: undefined },
);
rerender(undefined);
rerender(undefined);
expect(setPage).toHaveBeenCalledTimes(1);
});
});

View File

@@ -9,7 +9,6 @@ export * from './useCalculatedPageSize';
export * from './useColumnState';
export * from './useColumnStore';
export * from './usePreferredPageSize.store';
export * from './useRecoverFromEmptyPage';
export * from './useTableParams';
/**
@@ -286,51 +285,6 @@ export * from './useTableParams';
*
* **Pagination shows "Auto" option** when `calculatedPageSize` is passed, allowing users
* to reset to auto-calculated size.
*
* **`setPage` accepts history options**: `setPage(page, { history: 'replace' })` rewrites the
* current history entry instead of pushing a new one. Use `replace` for corrections the user
* did not ask for — otherwise the back button walks straight back into the state that was just
* corrected. Only applies when the page is synced to the URL; local (non-URL) pages ignore it.
*
* @example useRecoverFromEmptyPage — send the user back to a page that has data
*
* When rows disappear underneath the current page (filters narrowed, time range moved, items
* deleted), the user is stranded on an empty page they cannot leave by scrolling. This hook
* watches the fetched result and corrects the page with `history: 'replace'`, so the back
* button does not return to the empty page.
*
* Correction rules:
* - `page < 1` → jump to page 1, even while fetching or disabled. Such a page usually maps to a
* negative offset the API rejects (400 `offset cannot be negative`), so the response can never
* confirm the page is empty — deferring to it would strand the user on a permanent error.
* - Page is empty and not page 1 → go to `min(ceil(total / pageSize), page - 1)`. When `total`
* is trustworthy that lands on the last page holding data; when `total` is unknown or zero it
* lands on page 1; and when `total` claims this page should have had rows it steps back a
* single page. Repeated step-backs give up and jump to page 1 after the second one, so a
* badly inflated `total` cannot walk the user down one request at a time.
* - Page has rows, or the user is already on page 1 → do nothing (an empty page 1 means there
* is genuinely nothing to show).
*
* Pass `isFetching` so the hook waits for the request to settle, and `isDisabled` so a failed
* request is not mistaken for an empty page. Neither gate suppresses the `page < 1` clamp.
*
* ```tsx
* import { useRecoverFromEmptyPage, useTableParams } from 'components/TanStackTableView';
*
* const { page, limit, setPage } = useTableParams(QUERY_PARAMS, { page: 1, limit: 20 });
* const { data, isLoading, isFetching, isError } = useListQuery({ page, limit });
*
* useRecoverFromEmptyPage({
* page,
* pageSize: limit,
* rowCount: data?.rows.length ?? 0,
* total: data?.total ?? 0,
* isFetching: isLoading || isFetching,
* // Skip correction on errors — no rows there means "request failed", not "page is empty".
* isDisabled: isError,
* setPage,
* });
* ```
*/
const TanStackTable = Object.assign(TanStackTableBase, {
Text: TanStackTableText,

View File

@@ -1,107 +0,0 @@
import { useEffect, useRef } from 'react';
import { SetPageOptions } from './useTableParams';
const FIRST_PAGE = 1;
const REPLACE_HISTORY: SetPageOptions = { history: 'replace' };
/**
* How many single-page step-backs to attempt before giving up and going to page 1.
*
* A step-back only happens when `total` claims the current page should hold data but the
* response came back empty. Each hop costs a request, so an inflated `total` on a high page
* number would otherwise walk the user down one page at a time behind a spinner.
*/
const MAX_STEP_BACKS = 2;
type Correction = {
from: number;
to: number;
};
export type UseRecoverFromEmptyPageParams = {
page: number;
pageSize: number;
rowCount: number;
total: number;
isFetching: boolean;
isDisabled?: boolean;
setPage: (page: number, options?: SetPageOptions) => void;
};
export function useRecoverFromEmptyPage({
page,
pageSize,
rowCount,
total,
isFetching,
isDisabled = false,
setPage,
}: UseRecoverFromEmptyPageParams): void {
const setPageRef = useRef(setPage);
const lastCorrectionRef = useRef<Correction | null>(null);
const stepBacksRef = useRef(0);
useEffect(() => {
setPageRef.current = setPage;
});
useEffect(() => {
if (lastCorrectionRef.current && lastCorrectionRef.current.from !== page) {
lastCorrectionRef.current = null;
}
const correctTo = (nextPage: number): boolean => {
if (lastCorrectionRef.current?.to === nextPage) {
return false;
}
lastCorrectionRef.current = { from: page, to: nextPage };
setPageRef.current(nextPage, REPLACE_HISTORY);
return true;
};
// A page below the first one is invalid on its own terms — it usually maps to a
// negative offset the API rejects outright, so waiting for a response that will
// never arrive (or trusting a failed one) would strand the user for good.
if (page < FIRST_PAGE) {
stepBacksRef.current = 0;
void correctTo(FIRST_PAGE);
return;
}
if (isFetching || isDisabled) {
return;
}
// The page has data, or there is genuinely nothing to show anywhere.
if (rowCount > 0 || page === FIRST_PAGE) {
stepBacksRef.current = 0;
return;
}
const currentPage = Math.floor(page);
const lastPageWithData =
pageSize > 0 && total > 0 ? Math.ceil(total / pageSize) : FIRST_PAGE;
const nextPage = Math.max(
FIRST_PAGE,
Math.min(lastPageWithData, currentPage - 1),
);
// `total` disagrees with the response: it says this page should have rows, so the
// only safe move is one page back. Cap how often that repeats — every hop is a
// request, and a badly inflated `total` would otherwise crawl down from page 40.
const isStepBack = nextPage === currentPage - 1;
if (isStepBack && stepBacksRef.current >= MAX_STEP_BACKS) {
if (correctTo(FIRST_PAGE)) {
stepBacksRef.current = 0;
}
return;
}
if (correctTo(nextPage) && isStepBack) {
stepBacksRef.current += 1;
}
}, [isFetching, isDisabled, page, pageSize, rowCount, total]);
}

View File

@@ -29,16 +29,12 @@ type Defaults = {
cleanupOnUnmount?: boolean;
};
export type SetPageOptions = {
history?: 'push' | 'replace';
};
export type TableParamsResult = {
page: number;
limit: number;
orderBy: SortState | null;
expanded: ExpandedState;
setPage: (p: number, options?: SetPageOptions) => void;
setPage: (p: number) => void;
setLimit: (l: number) => void;
setOrderBy: (s: SortState | null) => void;
setExpanded: (updaterOrValue: Updater<ExpandedState>) => void;
@@ -253,17 +249,6 @@ export function useTableParams(
[],
);
const setUrlPageWithOptions = useCallback(
(page: number, options?: SetPageOptions): void => {
void setUrlPage(page, options);
},
[setUrlPage],
);
const setLocalPageValue = useCallback((page: number): void => {
setLocalPage(page);
}, []);
const orderByUrlMemoKey = `${urlOrderBy?.columnName}${urlOrderBy?.order}`;
const prevOrderByRef = useRef<string | null>(null);
@@ -318,7 +303,7 @@ export function useTableParams(
limit: useUrlForLimit ? urlLimit : localLimit,
orderBy: (useUrlForOrderBy ? urlOrderBy : localOrderBy) as SortState | null,
expanded: useUrlForExpanded ? urlExpanded : localExpanded,
setPage: useUrlForPage ? setUrlPageWithOptions : setLocalPageValue,
setPage: useUrlForPage ? setUrlPage : setLocalPage,
setLimit: useUrlForLimit ? setUrlLimit : setLocalLimitWithPersist,
setOrderBy: useUrlForOrderBy ? setUrlOrderBy : setLocalOrderBy,
setExpanded: useUrlForExpanded ? setUrlExpanded : handleSetLocalExpanded,

View File

@@ -10,6 +10,7 @@ const fieldContextToSuggestionMap: Record<
[TelemetrytypesFieldContextDTO.attribute]: 'attribute',
// no maps for the following values on suggestion context
[TelemetrytypesFieldContextDTO.trace]: undefined,
[TelemetrytypesFieldContextDTO.scope]: undefined,
[TelemetrytypesFieldContextDTO.body]: undefined,
[TelemetrytypesFieldContextDTO.metric]: undefined,
[TelemetrytypesFieldContextDTO.log]: undefined,

View File

@@ -7,7 +7,6 @@ import TanStackTable, {
TableColumnDef,
useCalculatedPageSize,
useHiddenColumnIds,
useRecoverFromEmptyPage,
useTableParams,
} from 'components/TanStackTableView';
import { InfraMonitoringEvents } from 'constants/events';
@@ -137,7 +136,6 @@ export function K8sBaseList<
page: currentPage,
limit: currentPageSize,
setLimit,
setPage,
} = useTableParams(
{
page: INFRA_MONITORING_K8S_PARAMS_KEYS.PAGE,
@@ -245,16 +243,6 @@ export function K8sBaseList<
const totalCount = data?.total || 0;
const hasFilters = !!expression?.trim();
useRecoverFromEmptyPage({
page: currentPage,
pageSize: currentPageSize,
rowCount: pageData.length,
total: totalCount,
isFetching: isLoading || isFetching,
isDisabled: isError || Boolean(data?.error),
setPage,
});
const getGroupKeyFn = useCallback(
(item: T) => getGroupedByMeta(item, groupBy),
[groupBy],

View File

@@ -591,14 +591,12 @@ describe('K8sBaseList', () => {
});
describe('with empty data', () => {
const onUrlUpdateMock = jest.fn<void, [UrlUpdateEvent]>();
const fetchListDataMock = jest.fn<
ReturnType<NonNullable<K8sBaseListProps<TestItem>['fetchListData']>>,
Parameters<NonNullable<K8sBaseListProps<TestItem>['fetchListData']>>
>();
beforeEach(() => {
onUrlUpdateMock.mockClear();
fetchListDataMock.mockClear();
fetchListDataMock.mockResolvedValue({
data: [],
@@ -607,7 +605,6 @@ describe('K8sBaseList', () => {
});
renderComponent<TestItem>({
onUrlUpdate: onUrlUpdateMock,
entity: InfraMonitoringEntity.PODS,
eventCategory: InfraMonitoringEvents.Pod,
fetchListData: fetchListDataMock,
@@ -628,177 +625,6 @@ describe('K8sBaseList', () => {
expect(fetchListDataMock).toHaveBeenCalled();
});
});
it('should not rewrite the page when already on the first page', async () => {
await waitFor(() => {
expect(fetchListDataMock).toHaveBeenCalled();
});
const pageUpdates = onUrlUpdateMock.mock.calls
.map((call) => call[0].searchParams.get('page'))
.filter(Boolean);
expect(pageUpdates).toHaveLength(0);
});
});
describe('with a page beyond the end of the list', () => {
const onUrlUpdateMock = jest.fn<void, [UrlUpdateEvent]>();
const fetchListDataMock = jest.fn<
ReturnType<NonNullable<K8sBaseListProps<TestItem>['fetchListData']>>,
Parameters<NonNullable<K8sBaseListProps<TestItem>['fetchListData']>>
>();
// 25 rows exist, so pages 1-3 serve data and page 7 of 10 comes back empty.
const rows: TestItem[] = Array.from({ length: 25 }, (_, index) => ({
id: `pod-${index + 1}`,
}));
beforeEach(() => {
onUrlUpdateMock.mockClear();
fetchListDataMock.mockClear();
// Offset-aware on purpose: a mock that answers empty for every offset would let
// the assertions pass against a page the recovery has already moved on from.
fetchListDataMock.mockImplementation(async ({ offset = 0, limit = 10 }) => ({
data: rows.slice(offset, offset + limit),
total: rows.length,
error: null,
}));
renderComponent<TestItem>({
onUrlUpdate: onUrlUpdateMock,
entity: InfraMonitoringEntity.PODS,
eventCategory: InfraMonitoringEvents.Pod,
fetchListData: fetchListDataMock,
queryParams: { page: '7', pageSize: '10' },
tableColumns: createTestColumns(),
getRowKey: (row): string => row.id,
getItemKey: (row): string => row.id,
});
});
it('should send the user back to the last page holding data', async () => {
// The rows of page 3 on screen are what proves the recovery settled there,
// rather than passing through on its way somewhere else.
await expect(screen.findByText('pod-21')).resolves.toBeInTheDocument();
const pageUpdates = onUrlUpdateMock.mock.calls
.map((call) => call[0].searchParams.get('page'))
.filter(Boolean);
expect(pageUpdates).toStrictEqual(['3']);
});
it('should correct the page in a single hop', async () => {
await expect(screen.findByText('pod-21')).resolves.toBeInTheDocument();
// Only the original out-of-range page and the corrected one are requested.
expect(
fetchListDataMock.mock.calls.map((call) => call[0].offset),
).toStrictEqual([60, 20]);
});
it('should replace the history entry instead of pushing the correction', async () => {
await expect(screen.findByText('pod-21')).resolves.toBeInTheDocument();
const pageCorrection = onUrlUpdateMock.mock.calls.find(
(call) => call[0].searchParams.get('page') === '3',
);
expect(pageCorrection?.[0].options.history).toBe('replace');
});
});
describe('with a page below the first one', () => {
const onUrlUpdateMock = jest.fn<void, [UrlUpdateEvent]>();
const fetchListDataMock = jest.fn<
ReturnType<NonNullable<K8sBaseListProps<TestItem>['fetchListData']>>,
Parameters<NonNullable<K8sBaseListProps<TestItem>['fetchListData']>>
>();
beforeEach(() => {
onUrlUpdateMock.mockClear();
fetchListDataMock.mockClear();
// page=0 turns into offset=-10, which the API rejects outright — the list
// can only recover by clamping the page, never by reading the response.
fetchListDataMock.mockImplementation(async ({ offset = 0 }) => {
if (offset < 0) {
throw new APIError({
httpStatusCode: 400,
error: {
code: 'invalid_input',
message: 'offset cannot be negative',
url: '',
errors: [],
},
});
}
return { data: [{ id: 'pod-1' }], total: 1, error: null };
});
renderComponent<TestItem>({
onUrlUpdate: onUrlUpdateMock,
entity: InfraMonitoringEntity.PODS,
eventCategory: InfraMonitoringEvents.Pod,
fetchListData: fetchListDataMock,
queryParams: { page: '0', pageSize: '10' },
tableColumns: createTestColumns(),
getRowKey: (row): string => row.id,
getItemKey: (row): string => row.id,
});
});
it('should reject the request that carried the negative offset', async () => {
await waitFor(() => {
expect(
fetchListDataMock.mock.calls.some((call) => call[0].offset === -10),
).toBe(true);
});
await expect(
fetchListDataMock.mock.results[0].value as Promise<unknown>,
).rejects.toThrow('offset cannot be negative');
});
it('should clamp the page to the first one even though the request failed', async () => {
await waitFor(() => {
expect(onUrlUpdateMock).toHaveBeenCalled();
});
// Page 1 is the default, so the correction drops the param rather than
// writing `page=1`.
const pageCorrection = onUrlUpdateMock.mock.calls.find(
(call) => call[0].searchParams.get('page') === null,
);
expect(pageCorrection).toBeDefined();
expect(pageCorrection?.[0].queryString).toBe('?pageSize=10');
});
it('should replace the history entry instead of pushing the correction', async () => {
await waitFor(() => {
expect(onUrlUpdateMock).toHaveBeenCalled();
});
const pageCorrection = onUrlUpdateMock.mock.calls.find(
(call) => call[0].searchParams.get('page') === null,
);
expect(pageCorrection?.[0].options.history).toBe('replace');
});
it('should refetch with a non-negative offset after clamping', async () => {
await waitFor(() => {
expect(
fetchListDataMock.mock.calls.some((call) => call[0].offset === 0),
).toBe(true);
});
await waitFor(() => {
expect(screen.getByText('pod-1')).toBeInTheDocument();
});
});
});
describe('with error response', () => {

View File

@@ -82,7 +82,6 @@ function EntityLogsContent({
const { activeLog, selectedTab, handleSetActiveLog, handleCloseLogDetail } =
useLogDetailHandlers();
// TODO: Move away from using onAddToQuery after old drawer cleanup
const onAddToQuery = useCallback(
(fieldKey: string, fieldValue: string, operator: string): void => {
handleCloseLogDetail();
@@ -105,21 +104,6 @@ function EntityLogsContent({
[userExpression, querySearchOnRun, handleCloseLogDetail, category],
);
const onApplyLogFilter = useCallback(
(expression: string): void => {
handleCloseLogDetail();
const newUser = userExpression.trim()
? `${userExpression} AND ${expression}`
: expression;
querySearchOnRun(newUser);
logInfraDrawerFilterCustomizedEvent(category, 'logs', newUser, 'logs');
},
[userExpression, querySearchOnRun, handleCloseLogDetail, category],
);
const {
logs,
loadMoreLogs,
@@ -344,7 +328,6 @@ function EntityLogsContent({
selectedTab={selectedTab}
onAddToQuery={onAddToQuery}
onClickActionItem={onAddToQuery}
onApplyLogFilter={onApplyLogFilter}
onScrollToLog={handleScrollToLog}
handleOpenInExplorer={(e) => handleOpenInExplorer(e, activeLog)}
getContainer={(): HTMLElement =>

View File

@@ -47,7 +47,6 @@ import {
useInfraMonitoringCategory,
useInfraMonitoringGroupBy,
useInfraMonitoringOrderBy,
useInfraMonitoringPageListing,
useInfraMonitoringSelectedItemParams,
} from './hooks';
@@ -68,7 +67,6 @@ export default function InfraMonitoringK8s(): JSX.Element {
const [, setGroupBy] = useInfraMonitoringGroupBy();
const [, setOrderBy] = useInfraMonitoringOrderBy();
const [, setSelectedItemParams] = useInfraMonitoringSelectedItemParams();
const [, setCurrentPage] = useInfraMonitoringPageListing();
const compositeQuery = useGetCompositeQueryParam();
const { currentQuery, redirectWithQueryBuilderData } = useQueryBuilder();
@@ -220,7 +218,6 @@ export default function InfraMonitoringK8s(): JSX.Element {
void setSelectedCategory(key as string);
void setOrderBy(null);
void setGroupBy(null);
void setCurrentPage(null);
setSelectedItemParams(null);
redirectWithQueryBuilderData({
...currentQuery,

View File

@@ -1,128 +0,0 @@
import React from 'react';
import { QueryClient, QueryClientProvider } from 'react-query';
// eslint-disable-next-line no-restricted-imports
import { Provider } from 'react-redux';
import { MemoryRouter } from 'react-router-dom';
import { MemoryRouter as MemoryRouterV5 } from 'react-router-dom-v5-compat';
import { TooltipProvider } from '@signozhq/ui/tooltip';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import { NuqsTestingAdapter, UrlUpdateEvent } from 'nuqs/adapters/testing';
import { AppProvider } from 'providers/App/App';
import { QueryBuilderProvider } from 'providers/QueryBuilder';
import TimezoneProvider from 'providers/Timezone';
import store from 'store';
import { K8sCategories } from '../constants';
import InfraMonitoringK8s from '../InfraMonitoringK8s';
// Quick filters fire their own field APIs and are irrelevant to pagination.
jest.mock('components/QuickFilters/QuickFilters', () => ({
__esModule: true,
default: (): JSX.Element => <div data-testid="quick-filters" />,
}));
// The list owns its own page recovery; stubbing it keeps the page param under the
// sole control of the category handler being tested here.
jest.mock('../Base/K8sDynamicList', () => ({
__esModule: true,
K8sDynamicList: (): JSX.Element => <div data-testid="k8s-dynamic-list" />,
default: (): JSX.Element => <div data-testid="k8s-dynamic-list" />,
}));
// Analytics only; jsdom lacks the Performance navigation entries it reads.
jest.mock('lib/navigation', () => ({
getNavigationReferrer: (): string => 'direct',
}));
function renderPage(
queryParams: Record<string, string>,
onUrlUpdate: jest.Mock<void, [UrlUpdateEvent]>,
): void {
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
render(
<MemoryRouter>
<MemoryRouterV5>
<TimezoneProvider>
<QueryClientProvider client={queryClient}>
<AppProvider>
<Provider store={store}>
<NuqsTestingAdapter
searchParams={queryParams}
onUrlUpdate={onUrlUpdate}
>
<TooltipProvider>
<QueryBuilderProvider>
<InfraMonitoringK8s />
</QueryBuilderProvider>
</TooltipProvider>
</NuqsTestingAdapter>
</Provider>
</AppProvider>
</QueryClientProvider>
</TimezoneProvider>
</MemoryRouterV5>
</MemoryRouter>,
);
}
describe('InfraMonitoringK8s', () => {
describe('when the category changes from a page other than the first', () => {
const onUrlUpdateMock = jest.fn<void, [UrlUpdateEvent]>();
beforeEach(async () => {
onUrlUpdateMock.mockClear();
renderPage(
{ category: K8sCategories.PODS, page: '3', pageSize: '10' },
onUrlUpdateMock,
);
await screen.findByTestId(`category-${K8sCategories.NODES}`);
});
it('should drop the page so the new category starts at the first one', async () => {
fireEvent.click(screen.getByTestId(`category-${K8sCategories.NODES}`));
// Page 3 of pods says nothing about nodes — keeping it asks the new entity
// for an offset it may not have. The param is cleared rather than set to 1,
// since an absent page already means the first one.
await waitFor(() => {
const categorySwitch = onUrlUpdateMock.mock.calls.find(
(call) => call[0].searchParams.get('category') === K8sCategories.NODES,
);
expect(categorySwitch).toBeDefined();
expect(categorySwitch?.[0].searchParams.get('page')).toBeNull();
});
});
it('should keep the page size, which is not category specific', async () => {
fireEvent.click(screen.getByTestId(`category-${K8sCategories.NODES}`));
await waitFor(() => {
const categorySwitch = onUrlUpdateMock.mock.calls.find(
(call) => call[0].searchParams.get('category') === K8sCategories.NODES,
);
expect(categorySwitch?.[0].searchParams.get('pageSize')).toBe('10');
});
});
it('should leave the page alone when the same category is clicked again', async () => {
fireEvent.click(screen.getByTestId(`category-${K8sCategories.PODS}`));
await waitFor(() => {
expect(screen.getByTestId('k8s-dynamic-list')).toBeInTheDocument();
});
const droppedPage = onUrlUpdateMock.mock.calls.some(
(call) => !call[0].searchParams.has('page'),
);
expect(droppedPage).toBe(false);
});
});
});

View File

@@ -34,9 +34,6 @@ export const useInfraMonitoringPageListing = (): UseQueryStateReturn<
> =>
useQueryState(
INFRA_MONITORING_K8S_PARAMS_KEYS.PAGE,
// do not use .withDefault here, this can cause bugs when
// two hooks of nuqs define default twice, this is also
// defined at useTableParams
parseAsInteger.withOptions(defaultNuqsOptions),
);

View File

@@ -41,7 +41,6 @@ interface OverviewProps {
selectedOptions: OptionsQuery;
listViewPanelSelectedFields?: IField[] | null;
handleChangeSelectedView?: ChangeViewFunctionType;
onApplyLogFilter?: (expression: string) => void;
}
type Props = OverviewProps &
@@ -56,7 +55,6 @@ function Overview({
selectedOptions,
listViewPanelSelectedFields,
handleChangeSelectedView,
onApplyLogFilter,
}: Props): JSX.Element {
const [isWrapWord, setIsWrapWord] = useState<boolean>(true);
const [isSearchVisible, setIsSearchVisible] = useState<boolean>(true);
@@ -69,7 +67,6 @@ function Overview({
const { actions, visibleActions } = useLogAttributeActions({
handleChangeSelectedView,
isListViewPanel,
onApplyLogFilter,
});
const isLogDetailsV2 = useIsLogDetailsV2();

View File

@@ -1,7 +1,6 @@
import { useCallback, useMemo } from 'react';
import { useLocation } from 'react-router-dom';
import { CircleMinus, CirclePlus, Layers, RefreshCw } from '@signozhq/icons';
import { convertFiltersToExpression } from 'components/QueryBuilderV2/utils';
import { FeatureKeys } from 'constants/features';
import { QueryParams } from 'constants/query';
import ROUTES from 'constants/routes';
@@ -16,7 +15,6 @@ import {
VisibleActionsConfig,
} from 'periscope/components/PrettyView/PrettyView';
import { useAppContext } from 'providers/App/App';
import { IBuilderQuery } from 'types/api/queryBuilder/queryBuilderData';
import { LogDetailsAction } from '../constants';
import {
@@ -29,7 +27,6 @@ import {
interface UseLogAttributeActionsParams {
handleChangeSelectedView?: ChangeViewFunctionType;
isListViewPanel?: boolean;
onApplyLogFilter?: (expression: string) => void;
}
interface UseLogAttributeActionsResult {
@@ -53,7 +50,6 @@ const ALL_LEAF_ACTIONS = [
export function useLogAttributeActions({
handleChangeSelectedView,
isListViewPanel = false,
onApplyLogFilter,
}: UseLogAttributeActionsParams): UseLogAttributeActionsResult {
const { pathname } = useLocation();
const { stagedQuery, updateQueriesData } = useQueryBuilder();
@@ -69,6 +65,9 @@ export function useLogAttributeActions({
const filterFor = useCallback(
(context: FieldContext, isFilterIn: boolean): void => {
if (!stagedQuery) {
return;
}
const target = buildLogFilterTarget(
context.fieldKeyPath,
context.fieldValue,
@@ -78,29 +77,6 @@ export function useLogAttributeActions({
? target.filterInOperator
: target.filterOutOperator;
// Non-explorer surfaces (infra monitoring, etc.) apply a ready v5
// expression fragment to their own query.
if (onApplyLogFilter) {
const base = {
filters: { items: [], op: 'AND' },
} as unknown as IBuilderQuery;
const nextFilters = getFilterQueryData(
base,
target,
context.fieldValue,
operator,
).filters ?? { items: [], op: 'AND' };
const { expression } = convertFiltersToExpression(nextFilters);
if (expression) {
onApplyLogFilter(expression);
}
return;
}
if (!stagedQuery) {
return;
}
const updatedQuery = updateQueriesData(
stagedQuery,
'queryData',
@@ -123,7 +99,6 @@ export function useLogAttributeActions({
updateQueriesData,
viewName,
handleChangeSelectedView,
onApplyLogFilter,
],
);
@@ -204,25 +179,20 @@ export function useLogAttributeActions({
buildLogFilterTarget(fieldKeyPath, undefined, isBodyJsonQueryEnabled)
.isRestricted;
// The using surface must provide an apply path.
const canApplyFilter = !!handleChangeSelectedView || !!onApplyLogFilter;
return [
{
key: LogDetailsAction.FILTER_IN,
label: 'Filter for value',
icon: <CirclePlus size={12} />,
onClick: (context): void => filterFor(context, true),
shouldHide: (_key, fieldKeyPath): boolean =>
!canApplyFilter || isRestricted(fieldKeyPath),
shouldHide: (_key, fieldKeyPath): boolean => isRestricted(fieldKeyPath),
},
{
key: LogDetailsAction.FILTER_OUT,
label: 'Filter out value',
icon: <CircleMinus size={12} />,
onClick: (context): void => filterFor(context, false),
shouldHide: (_key, fieldKeyPath): boolean =>
!canApplyFilter || isRestricted(fieldKeyPath),
shouldHide: (_key, fieldKeyPath): boolean => isRestricted(fieldKeyPath),
},
{
key: LogDetailsAction.GROUP_BY,
@@ -230,10 +200,8 @@ export function useLogAttributeActions({
icon: <Layers size={12} />,
onClick: groupBy,
shouldHide: (_key, fieldKeyPath): boolean =>
!handleChangeSelectedView ||
!buildLogFilterTarget(fieldKeyPath, undefined, isBodyJsonQueryEnabled)
.groupBySupported ||
isOldExplorerOrLive,
.groupBySupported || isOldExplorerOrLive,
},
{
key: LogDetailsAction.REPLACE_FILTER,
@@ -241,9 +209,7 @@ export function useLogAttributeActions({
icon: <RefreshCw size={12} />,
onClick: replaceFilter,
shouldHide: (_key, fieldKeyPath): boolean =>
!handleChangeSelectedView ||
isRestricted(fieldKeyPath) ||
isOldExplorerOrLive,
isRestricted(fieldKeyPath) || isOldExplorerOrLive,
},
];
}, [
@@ -252,8 +218,6 @@ export function useLogAttributeActions({
replaceFilter,
isBodyJsonQueryEnabled,
isOldExplorerOrLive,
handleChangeSelectedView,
onApplyLogFilter,
]);
const visibleActions = useMemo<VisibleActionsConfig>(

View File

@@ -21,7 +21,6 @@ import azureMysqlUrl from '@/assets/Logos/azure-mysql.svg';
import azureOpenaiUrl from '@/assets/Logos/azure-openai.svg';
import azureSqlDatabaseMetricsUrl from '@/assets/Logos/azure-sql-database-metrics.svg';
import azureVmUrl from '@/assets/Logos/azure-vm.svg';
import azureUrl from '@/assets/Logos/azure.svg';
import basetenUrl from '@/assets/Logos/baseten.svg';
import cassandraUrl from '@/assets/Logos/cassandra.svg';
import celeryUrl from '@/assets/Logos/celery.svg';
@@ -29,7 +28,6 @@ import certManagerUrl from '@/assets/Logos/cert-manager.svg';
import claudeCodeUrl from '@/assets/Logos/claude-code.svg';
import clickhouseUrl from '@/assets/Logos/clickhouse.svg';
import cloudflareUrl from '@/assets/Logos/cloudflare.svg';
import cloudnativePgUrl from '@/assets/Logos/cloudnative-pg.svg';
import cloudwatchLogsUrl from '@/assets/Logos/cloudwatch-logs.svg';
import cohereUrl from '@/assets/Logos/cohere.svg';
import confluentKafkaUrl from '@/assets/Logos/confluent-kafka.svg';
@@ -41,7 +39,6 @@ import datadogUrl from '@/assets/Logos/datadog.svg';
import dbosUrl from '@/assets/Logos/dbos.svg';
import deepseekUrl from '@/assets/Logos/deepseek.svg';
import denoUrl from '@/assets/Logos/deno.svg';
import difyUrl from '@/assets/Logos/dify.svg';
import dockerUrl from '@/assets/Logos/docker.svg';
import documentLoadUrl from '@/assets/Logos/document-load.svg';
import dotnetUrl from '@/assets/Logos/dotnet.svg';
@@ -73,8 +70,6 @@ import gcpCloudStorageUrl from '@/assets/Logos/gcp-cloud-storage.svg';
import gcpComputeEngineUrl from '@/assets/Logos/gcp-compute-engine.svg';
import gcpGkeUrl from '@/assets/Logos/gcp-gke.svg';
import gcpVpcUrl from '@/assets/Logos/gcp-vpc.svg';
import gcpUrl from '@/assets/Logos/gcp.svg';
import githubCopilotUrl from '@/assets/Logos/github-copilot.svg';
import githubUrl from '@/assets/Logos/github.svg';
import goUrl from '@/assets/Logos/go.svg';
import googleAdkUrl from '@/assets/Logos/google-adk.svg';
@@ -101,8 +96,6 @@ import javascriptUrl from '@/assets/Logos/javascript.svg';
import jbossUrl from '@/assets/Logos/jboss.svg';
import jenkinsUrl from '@/assets/Logos/jenkins.svg';
import kafkaUrl from '@/assets/Logos/kafka.svg';
import kedaUrl from '@/assets/Logos/keda.svg';
import kongUrl from '@/assets/Logos/kong.svg';
import kubernetesUrl from '@/assets/Logos/kubernetes.svg';
import lambdaUrl from '@/assets/Logos/lambda.svg';
import langchainUrl from '@/assets/Logos/langchain.svg';
@@ -121,7 +114,6 @@ import microsoftSqlServerUrl from '@/assets/Logos/microsoft-sql-server.svg';
import mistralUrl from '@/assets/Logos/mistral.svg';
import mongoUrl from '@/assets/Logos/mongo.svg';
import n8nUrl from '@/assets/Logos/n8n.svg';
import neonUrl from '@/assets/Logos/neon.svg';
import newrelicUrl from '@/assets/Logos/newrelic.svg';
import nextjsUrl from '@/assets/Logos/nextjs.svg';
import nginxUrl from '@/assets/Logos/nginx.svg';
@@ -2867,25 +2859,6 @@ const onboardingConfigWithLinks = [
label: 'Traces',
imgUrl: lambdaUrl,
link: '/docs/aws-monitoring/lambda/lambda-traces/',
question: {
desc: 'Which runtime does your Lambda function use?',
helpText:
'Python, Node.js, Java, and Ruby use the OpenTelemetry auto-instrumentation layer. Go has no layer, so you add the SDK to your code.',
options: [
{
key: 'aws-lambda-traces-auto',
label: 'Python, Node.js, Java, Ruby',
imgUrl: lambdaUrl,
link: '/docs/aws-monitoring/lambda/lambda-traces/',
},
{
key: 'aws-lambda-traces-golang',
label: 'Go',
imgUrl: goUrl,
link: '/docs/aws-monitoring/lambda/lambda-traces-golang/',
},
],
},
},
{
key: 'aws-lambda-metrics',
@@ -5537,10 +5510,8 @@ const onboardingConfigWithLinks = [
module: 'metrics',
relatedSearchKeywords: [
'integrations',
'logs',
'metrics',
'supabase',
'supabase logs',
'supabase metrics',
'supabase monitoring',
'supabase observability',
@@ -5574,23 +5545,43 @@ const onboardingConfigWithLinks = [
label: 'Traefik',
imgUrl: opentelemetryUrl,
tags: ['infrastructure monitoring'],
module: 'apm',
module: 'infrastructure',
relatedSearchKeywords: [
'infrastructure',
'traefik',
'traefik access logs',
'traefik logs',
'traefik metrics',
'traefik monitoring',
'traefik observability',
'traefik tracing',
],
link: '/docs/tutorial/traefik-observability/',
question: {
desc: 'Which Traefik signals do you want to send to SigNoz?',
type: 'select',
options: [
{
key: 'traefik-metrics-traces',
label: 'Metrics & Traces',
imgUrl: opentelemetryUrl,
link: '/docs/tutorial/traefik-observability/',
},
{
key: 'traefik-logs',
label: 'Access Logs',
imgUrl: opentelemetryUrl,
link: '/docs/integrations/opentelemetry-traefik/',
},
],
},
},
{
dataSource: 'mongodb-atlas',
label: 'MongoDB (Atlas)',
imgUrl: mongoUrl,
tags: ['database'],
module: 'metrics',
module: 'database',
relatedSearchKeywords: [
'atlas',
'atlas metrics',
@@ -5609,15 +5600,36 @@ const onboardingConfigWithLinks = [
label: 'MySQL',
imgUrl: opentelemetryUrl,
tags: ['database'],
module: 'metrics',
module: 'database',
relatedSearchKeywords: [
'database',
'mysql',
'mysql error log',
'mysql logs',
'mysql metrics',
'mysql monitoring',
'mysql observability',
'mysql slow query log',
],
link: '/docs/metrics-management/mysql-metrics/',
question: {
desc: 'Which MySQL signals do you want to send to SigNoz?',
type: 'select',
options: [
{
key: 'mysql-metrics',
label: 'Metrics',
imgUrl: opentelemetryUrl,
link: '/docs/metrics-management/mysql-metrics/',
},
{
key: 'mysql-logs',
label: 'Logs',
imgUrl: opentelemetryUrl,
link: '/docs/integrations/opentelemetry-mysql/',
},
],
},
},
{
dataSource: 'jmx',
@@ -6980,514 +6992,5 @@ const onboardingConfigWithLinks = [
id: 'dspy',
link: '/docs/dspy-observability/',
},
{
dataSource: 'grok-build',
label: 'Grok Build',
imgUrl: grokUrl,
tags: ['LLM Monitoring'],
module: 'metrics',
relatedSearchKeywords: [
'coding agent',
'grok build',
'grok build events',
'grok build logs',
'grok build metrics',
'grok build monitoring',
'grok build observability',
'llm',
'llm monitoring',
'metrics',
'monitoring',
'observability',
'otel grok build integration',
'terminal coding agent',
'token usage',
'xai',
],
id: 'grok-build',
link: '/docs/grok-build-observability/',
},
{
dataSource: 'neon',
label: 'Neon',
imgUrl: neonUrl,
tags: ['database'],
module: 'metrics',
relatedSearchKeywords: [
'database',
'neon',
'neon database',
'neon db',
'neon logs',
'neon metrics',
'neon monitoring',
'neon observability',
'neondb',
'opentelemetry neon',
'postgres',
'postgresql',
'serverless postgres',
],
id: 'neon',
link: '/docs/integrations/opentelemetry-neondb/',
},
{
dataSource: 'dify',
label: 'Dify',
imgUrl: difyUrl,
tags: ['LLM Monitoring'],
module: 'apm',
relatedSearchKeywords: [
'ai agent',
'dify',
'dify monitoring',
'dify observability',
'dify traces',
'llm',
'llm monitoring',
'metrics',
'no code ai',
'observability',
'opentelemetry dify',
'traces',
],
id: 'dify',
link: '/docs/dify-observability/',
},
{
dataSource: 'firecrawl',
label: 'Firecrawl',
imgUrl: llmMonitoringUrl,
tags: ['LLM Monitoring'],
module: 'apm',
relatedSearchKeywords: [
'crawl',
'firecrawl',
'firecrawl metrics',
'firecrawl monitoring',
'firecrawl observability',
'firecrawl traces',
'llm',
'llm monitoring',
'opentelemetry firecrawl',
'scrape',
'traces',
'web scraping',
],
id: 'firecrawl',
link: '/docs/firecrawl-monitoring/',
},
{
dataSource: 'keda',
label: 'KEDA',
imgUrl: kedaUrl,
tags: ['infrastructure monitoring', 'metrics'],
module: 'metrics',
relatedSearchKeywords: [
'autoscaling',
'event driven autoscaling',
'keda',
'keda metrics',
'keda monitoring',
'kubernetes',
'kubernetes autoscaling',
'metrics',
'opentelemetry keda',
'scaled jobs',
'scaled objects',
],
id: 'keda',
link: '/docs/metrics-management/keda-metrics/',
},
{
dataSource: 'opentelemetry-collector-metrics',
label: 'OpenTelemetry Collector Metrics',
imgUrl: opentelemetryUrl,
tags: ['infrastructure monitoring', 'metrics'],
module: 'metrics',
relatedSearchKeywords: [
'collector health',
'collector internal metrics',
'collector metrics',
'metrics',
'opentelemetry',
'opentelemetry collector',
'otel collector',
'otelcol metrics',
'pipeline health',
],
id: 'opentelemetry-collector-metrics',
link: '/docs/metrics-management/opentelemetry-collector-metrics/',
},
{
dataSource: 'cloudnative-pg',
label: 'CloudNativePG',
imgUrl: cloudnativePgUrl,
tags: ['database'],
module: 'metrics',
relatedSearchKeywords: [
'cloud native postgres',
'cloudnativepg',
'cnpg',
'cnpg metrics',
'database',
'kubernetes postgres',
'metrics',
'opentelemetry cloudnativepg',
'postgres',
'postgresql',
],
id: 'cloudnative-pg',
link: '/docs/metrics-management/opentelemetry-cloudnative-pg/',
},
{
dataSource: 'kong-gateway',
label: 'Kong Gateway',
imgUrl: kongUrl,
tags: ['infrastructure monitoring'],
module: 'apm',
relatedSearchKeywords: [
'api gateway',
'kong',
'kong gateway',
'kong logs',
'kong metrics',
'kong monitoring',
'kong observability',
'kong traces',
'opentelemetry kong',
'proxy',
'traces',
],
id: 'kong-gateway',
link: '/docs/integrations/kong-gateway/',
},
{
dataSource: 'github-copilot',
label: 'GitHub Copilot',
imgUrl: githubCopilotUrl,
tags: ['LLM Monitoring'],
module: 'apm',
relatedSearchKeywords: [
'coding agent',
'copilot',
'copilot chat',
'github copilot',
'github copilot metrics',
'github copilot monitoring',
'github copilot observability',
'github copilot traces',
'llm',
'llm monitoring',
'monitoring',
'observability',
'otel github copilot integration',
'traces',
'tracing',
'vs code',
],
id: 'github-copilot',
link: '/docs/github-copilot-monitoring/',
},
{
dataSource: 'serilog',
label: 'Serilog',
imgUrl: dotnetUrl,
tags: ['logs'],
module: 'logs',
relatedSearchKeywords: [
'.net',
'asp.net',
'c#',
'csharp',
'dotnet',
'dotnet logs',
'logging',
'logs',
'monitoring',
'observability',
'otel serilog integration',
'serilog',
'serilog logs',
'serilog sink',
'structured logging',
],
id: 'serilog',
link: '/docs/logs-management/send-logs/serilog-to-signoz/',
},
{
dataSource: 'gcp-integration',
label: 'GCP Integration',
imgUrl: gcpUrl,
tags: ['GCP'],
module: 'metrics',
relatedSearchKeywords: [
'cloud monitoring metrics',
'connect gcp account',
'gcp',
'gcp integration',
'gcp manual setup',
'gcp metrics',
'gcp monitoring',
'gcp observability',
'gcp service account',
'google cloud',
'google cloud integration',
'metrics',
'monitoring',
'observability',
'opentelemetry collector gcp',
],
id: 'gcp-integration',
link: '/docs/integrations/gcp/gcp-integration/',
},
{
dataSource: 'azure-cosmos-db',
label: 'Azure Cosmos DB',
imgUrl: azureUrl,
tags: ['Azure'],
module: 'dashboards',
relatedSearchKeywords: [
'azure',
'azure cosmos db',
'azure integration',
'cosmos db',
'cosmosdb',
'database',
'metrics',
'monitoring',
'nosql',
'observability',
'one click azure',
'request units',
],
id: 'azure-cosmos-db',
link: '/integrations/azure?service=cosmosdb',
internalRedirect: true,
},
{
dataSource: 'azure-mongodb',
label: 'Azure MongoDB vCore',
imgUrl: mongoUrl,
tags: ['Azure'],
module: 'dashboards',
relatedSearchKeywords: [
'azure',
'azure integration',
'azure mongodb',
'azure mongodb vcore',
'database',
'metrics',
'mongodb',
'mongodb vcore',
'monitoring',
'nosql',
'observability',
'one click azure',
],
id: 'azure-mongodb',
link: '/integrations/azure?service=mongodb',
internalRedirect: true,
},
{
dataSource: 'azure-postgresql-flexible-server',
label: 'Azure PostgreSQL Flexible Server',
imgUrl: postgresqlUrl,
tags: ['Azure'],
module: 'dashboards',
relatedSearchKeywords: [
'azure',
'azure integration',
'azure postgresql',
'azure postgresql flexible server',
'database',
'metrics',
'monitoring',
'observability',
'one click azure',
'postgres',
'postgresql',
'postgresql flexible server',
],
id: 'azure-postgresql-flexible-server',
link: '/integrations/azure?service=postgresqlflexibleserver',
internalRedirect: true,
},
{
dataSource: 'azure-cache-redis',
label: 'Azure Cache for Redis',
imgUrl: redisUrl,
tags: ['Azure'],
module: 'dashboards',
relatedSearchKeywords: [
'azure',
'azure cache for redis',
'azure integration',
'azure redis',
'cache',
'database',
'metrics',
'monitoring',
'observability',
'one click azure',
'redis',
],
id: 'azure-cache-redis',
link: '/integrations/azure?service=redis',
internalRedirect: true,
},
{
dataSource: 'azure-sql-managed-instance',
label: 'Azure SQL Managed Instance',
imgUrl: azureSqlDatabaseMetricsUrl,
tags: ['Azure'],
module: 'dashboards',
relatedSearchKeywords: [
'azure',
'azure integration',
'azure sql',
'azure sql managed instance',
'database',
'managed instance',
'metrics',
'monitoring',
'observability',
'one click azure',
'sql server',
],
id: 'azure-sql-managed-instance',
link: '/integrations/azure?service=sqldatabasemi',
internalRedirect: true,
},
{
dataSource: 'azure-cassandra-db',
label: 'Azure Managed Instance for Apache Cassandra',
imgUrl: cassandraUrl,
tags: ['Azure'],
module: 'dashboards',
relatedSearchKeywords: [
'apache cassandra',
'azure',
'azure cassandra',
'azure integration',
'azure managed instance for apache cassandra',
'cassandra',
'database',
'metrics',
'monitoring',
'observability',
'one click azure',
],
id: 'azure-cassandra-db',
link: '/integrations/azure?service=cassandradb',
internalRedirect: true,
},
{
dataSource: 'gcp-cloud-sql-postgresql',
label: 'GCP Cloud SQL for PostgreSQL',
imgUrl: gcpCloudSqlUrl,
tags: ['GCP'],
module: 'dashboards',
relatedSearchKeywords: [
'cloud sql',
'cloud sql for postgresql',
'database',
'gcp',
'gcp integration',
'google cloud',
'metrics',
'monitoring',
'observability',
'postgres',
'postgresql',
],
id: 'gcp-cloud-sql-postgresql',
link: '/integrations/gcp?service=cloudsql_postgres',
internalRedirect: true,
},
{
dataSource: 'gcp-memorystore-redis',
label: 'GCP Memorystore for Redis',
imgUrl: redisUrl,
tags: ['GCP'],
module: 'dashboards',
relatedSearchKeywords: [
'cache',
'database',
'gcp',
'gcp integration',
'google cloud',
'memorystore',
'memorystore for redis',
'metrics',
'monitoring',
'observability',
'redis',
],
id: 'gcp-memorystore-redis',
link: '/integrations/gcp?service=memorystore_redis',
internalRedirect: true,
},
{
dataSource: 'supabase-logs',
label: 'Supabase Logs',
imgUrl: supabaseUrl,
tags: ['logs'],
module: 'logs',
relatedSearchKeywords: [
'database',
'logging',
'logs',
'postgres',
'postgresql',
'send supabase logs',
'supabase',
'supabase log drains',
'supabase logs',
'supabase observability',
],
id: 'supabase-logs',
link: '/docs/logs-management/send-logs/supabase-logs/',
},
{
dataSource: 'traefik-logs',
label: 'Traefik Access Logs',
imgUrl: opentelemetryUrl,
tags: ['logs'],
module: 'logs',
relatedSearchKeywords: [
'access logs',
'logging',
'logs',
'opentelemetry traefik',
'proxy',
'reverse proxy',
'traefik',
'traefik access logs',
'traefik logs',
],
id: 'traefik-logs',
link: '/docs/integrations/opentelemetry-traefik/',
},
{
dataSource: 'mysql-logs',
label: 'MySQL Logs',
imgUrl: opentelemetryUrl,
tags: ['logs'],
module: 'logs',
relatedSearchKeywords: [
'database',
'logging',
'logs',
'mysql',
'mysql error log',
'mysql general query log',
'mysql logs',
'mysql slow query log',
'opentelemetry mysql',
],
id: 'mysql-logs',
link: '/docs/integrations/opentelemetry-mysql/',
},
];
export default onboardingConfigWithLinks;

View File

@@ -336,7 +336,7 @@ func (r *ClickHouseReader) GetTopLevelOperations(ctx context.Context, start, end
return &operations, nil
}
func (r *ClickHouseReader) buildResourceSubQuery(ctx context.Context, orgID valuer.UUID, tags []model.TagQueryParam, svc string, start, end time.Time) (string, error) {
func (r *ClickHouseReader) buildResourceSubQuery(tags []model.TagQueryParam, svc string, start, end time.Time) (string, error) {
// assuming all will be resource attributes.
// and resource attributes are string for traces
filterSet := v3.FilterSet{}
@@ -387,8 +387,7 @@ func (r *ClickHouseReader) buildResourceSubQuery(ctx context.Context, orgID valu
&filterSet,
[]v3.AttributeKey{},
v3.AttributeKey{},
false,
r.fl.BooleanOrEmpty(ctx, flagger.FeatureResolveSemconvFamilies, featuretypes.NewFlaggerEvaluationContext(orgID)))
false)
if err != nil {
r.logger.Error("Error in processing sql query", errorsV2.Attr(err))
return "", err
@@ -396,7 +395,7 @@ func (r *ClickHouseReader) buildResourceSubQuery(ctx context.Context, orgID valu
return resourceSubQuery, nil
}
func (r *ClickHouseReader) GetServices(ctx context.Context, orgID valuer.UUID, queryParams *model.GetServicesParams) (*[]model.ServiceItem, *model.ApiError) {
func (r *ClickHouseReader) GetServices(ctx context.Context, queryParams *model.GetServicesParams) (*[]model.ServiceItem, *model.ApiError) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalTraces.StringValue(),
@@ -468,7 +467,7 @@ func (r *ClickHouseReader) GetServices(ctx context.Context, orgID valuer.UUID, q
clickhouse.Named("names", ops),
)
resourceSubQuery, err := r.buildResourceSubQuery(ctx, orgID, queryParams.Tags, svc, *queryParams.Start, *queryParams.End)
resourceSubQuery, err := r.buildResourceSubQuery(queryParams.Tags, svc, *queryParams.Start, *queryParams.End)
if err != nil {
r.logger.Error("Error in processing sql query", errorsV2.Attr(err))
return
@@ -704,9 +703,9 @@ func addExistsOperator(item model.TagQuery, tagMapType string, not bool) (string
return fmt.Sprintf(" AND %s (%s)", notStr, strings.Join(tagOperatorPair, " OR ")), args
}
func (r *ClickHouseReader) GetEntryPointOperations(ctx context.Context, orgID valuer.UUID, queryParams *model.GetTopOperationsParams) (*[]model.TopOperationsItem, error) {
func (r *ClickHouseReader) GetEntryPointOperations(ctx context.Context, queryParams *model.GetTopOperationsParams) (*[]model.TopOperationsItem, error) {
// Step 1: Get top operations for the given service
topOps, err := r.GetTopOperations(ctx, orgID, queryParams)
topOps, err := r.GetTopOperations(ctx, queryParams)
if err != nil {
return nil, errorsV2.Wrapf(err, errorsV2.TypeInternal, errorsV2.CodeInternal, "Error in getting Top Operations")
}
@@ -758,7 +757,7 @@ func (r *ClickHouseReader) GetEntryPointOperations(ctx context.Context, orgID va
return &filtered, nil
}
func (r *ClickHouseReader) GetTopOperations(ctx context.Context, orgID valuer.UUID, queryParams *model.GetTopOperationsParams) (*[]model.TopOperationsItem, *model.ApiError) {
func (r *ClickHouseReader) GetTopOperations(ctx context.Context, queryParams *model.GetTopOperationsParams) (*[]model.TopOperationsItem, *model.ApiError) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalTraces.StringValue(),
@@ -788,7 +787,7 @@ func (r *ClickHouseReader) GetTopOperations(ctx context.Context, orgID valuer.UU
r.TraceDB, r.traceTableName,
)
resourceSubQuery, err := r.buildResourceSubQuery(ctx, orgID, queryParams.Tags, queryParams.ServiceName, *queryParams.Start, *queryParams.End)
resourceSubQuery, err := r.buildResourceSubQuery(queryParams.Tags, queryParams.ServiceName, *queryParams.Start, *queryParams.End)
if err != nil {
r.logger.Error("Error in processing sql query", errorsV2.Attr(err))
return nil, &model.ApiError{Typ: model.ErrorExec, Err: fmt.Errorf("error in processing sql query")}
@@ -859,7 +858,7 @@ func (r *ClickHouseReader) GetUsage(ctx context.Context, queryParams *model.GetU
return &usageItems, nil
}
func (r *ClickHouseReader) GetDependencyGraph(ctx context.Context, orgID valuer.UUID, queryParams *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error) {
func (r *ClickHouseReader) GetDependencyGraph(ctx context.Context, queryParams *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalTraces.StringValue(),
@@ -896,7 +895,7 @@ func (r *ClickHouseReader) GetDependencyGraph(ctx context.Context, orgID valuer.
)
tags := createTagQueryFromTagQueryParams(queryParams.Tags)
filterQuery, filterArgs := services.BuildServiceMapQuery(tags, r.fl.BooleanOrEmpty(ctx, flagger.FeatureResolveSemconvFamilies, featuretypes.NewFlaggerEvaluationContext(orgID)))
filterQuery, filterArgs := services.BuildServiceMapQuery(tags)
query += filterQuery + " GROUP BY src, dest;"
args = append(args, filterArgs...)

View File

@@ -1128,19 +1128,13 @@ func (aH *APIHandler) registerEvent(w http.ResponseWriter, r *http.Request) {
}
func (aH *APIHandler) getTopOperations(w http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
query, err := parseGetTopOperationsRequest(r)
if aH.HandleError(w, err, http.StatusBadRequest) {
return
}
result, apiErr := aH.reader.GetTopOperations(r.Context(), orgID, query)
result, apiErr := aH.reader.GetTopOperations(r.Context(), query)
if apiErr != nil && aH.HandleError(w, apiErr.Err, http.StatusInternalServerError) {
return
@@ -1151,20 +1145,13 @@ func (aH *APIHandler) getTopOperations(w http.ResponseWriter, r *http.Request) {
}
func (aH *APIHandler) getEntryPointOps(w http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
query, err := parseGetTopOperationsRequest(r)
if err != nil {
render.Error(w, err)
return
}
result, apiErr := aH.reader.GetEntryPointOperations(r.Context(), orgID, query)
result, apiErr := aH.reader.GetEntryPointOperations(r.Context(), query)
if apiErr != nil {
render.Error(w, apiErr)
return
@@ -1239,19 +1226,12 @@ func (aH *APIHandler) getServicesTopLevelOps(w http.ResponseWriter, r *http.Requ
}
func (aH *APIHandler) getServices(w http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
query, err := parseGetServicesRequest(r)
if aH.HandleError(w, err, http.StatusBadRequest) {
return
}
result, apiErr := aH.reader.GetServices(r.Context(), orgID, query)
result, apiErr := aH.reader.GetServices(r.Context(), query)
if apiErr != nil && aH.HandleError(w, apiErr.Err, http.StatusInternalServerError) {
return
}
@@ -1260,19 +1240,13 @@ func (aH *APIHandler) getServices(w http.ResponseWriter, r *http.Request) {
}
func (aH *APIHandler) dependencyGraph(w http.ResponseWriter, r *http.Request) {
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
query, err := parseGetServicesRequest(r)
if aH.HandleError(w, err, http.StatusBadRequest) {
return
}
result, err := aH.reader.GetDependencyGraph(r.Context(), orgID, query)
result, err := aH.reader.GetDependencyGraph(r.Context(), query)
if aH.HandleError(w, err, http.StatusBadRequest) {
return
}

View File

@@ -383,7 +383,7 @@ func buildLogsQuery(panelType v3.PanelType, start, end, step int64, mq *v3.Build
}
// build the where clause for resource table
resourceSubQuery, err := resource.BuildResourceSubQuery(DB_NAME, DISTRIBUTED_LOGS_V2_RESOURCE, bucketStart, bucketEnd, mq.Filters, mq.GroupBy, mq.AggregateAttribute, false, false)
resourceSubQuery, err := resource.BuildResourceSubQuery(DB_NAME, DISTRIBUTED_LOGS_V2_RESOURCE, bucketStart, bucketEnd, mq.Filters, mq.GroupBy, mq.AggregateAttribute, false)
if err != nil {
return "", err
}
@@ -475,7 +475,7 @@ func buildLogsLiveTailQuery(mq *v3.BuilderQuery) (string, error) {
}
// no values for bucket start and end
resourceSubQuery, err := resource.BuildResourceSubQuery(DB_NAME, DISTRIBUTED_LOGS_V2_RESOURCE, 0, 0, mq.Filters, mq.GroupBy, mq.AggregateAttribute, true, false)
resourceSubQuery, err := resource.BuildResourceSubQuery(DB_NAME, DISTRIBUTED_LOGS_V2_RESOURCE, 0, 0, mq.Filters, mq.GroupBy, mq.AggregateAttribute, true)
if err != nil {
return "", err
}

View File

@@ -6,9 +6,6 @@ import (
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/utils"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/semconv"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
var resourceLogOperators = map[v3.FilterOperator]string{
@@ -33,49 +30,22 @@ var resourceLogOperators = map[v3.FilterOperator]string{
}
// buildResourceFilter builds a clickhouse filter string for resource labels
func buildResourceFilter(logsOp string, key string, op v3.FilterOperator, value interface{}, members []string) string {
func buildResourceFilter(logsOp string, key string, op v3.FilterOperator, value interface{}) string {
// for all operators except contains and like
searchKey := fmt.Sprintf("simpleJSONExtractString(labels, %s)", querybuilder.ClickHouseStringLiteral(key))
if len(members) > 1 {
values := make([]string, 0, len(members))
for _, member := range members {
values = append(values, fmt.Sprintf("NULLIF(simpleJSONExtractString(labels, %s), '')", querybuilder.ClickHouseStringLiteral(member)))
}
searchKey = "COALESCE(" + strings.Join(values, ", ") + ", '')"
}
searchKey := fmt.Sprintf("simpleJSONExtractString(labels, '%s')", key)
// for contains and like it will be case insensitive
lowerSearchKey := fmt.Sprintf("simpleJSONExtractString(lower(labels), %s)", querybuilder.ClickHouseStringLiteral(key))
if len(members) > 1 {
lowerSearchKey = "lower(" + searchKey + ")"
}
lowerSearchKey := fmt.Sprintf("simpleJSONExtractString(lower(labels), '%s')", key)
chFmtVal := utils.ClickHouseFormattedValue(value)
lowerValue := strings.ToLower(fmt.Sprintf("%s", value))
switch op {
case v3.FilterOperatorExists, v3.FilterOperatorNotExists:
exists := op == v3.FilterOperatorExists
if len(members) == 1 {
if exists {
return fmt.Sprintf("simpleJSONHas(labels, %s)", querybuilder.ClickHouseStringLiteral(key))
}
return fmt.Sprintf("not simpleJSONHas(labels, %s)", querybuilder.ClickHouseStringLiteral(key))
}
presence := make([]string, 0, len(members))
for _, member := range members {
if exists {
presence = append(presence, fmt.Sprintf("simpleJSONHas(labels, %s)", querybuilder.ClickHouseStringLiteral(member)))
} else {
presence = append(presence, fmt.Sprintf("not simpleJSONHas(labels, %s)", querybuilder.ClickHouseStringLiteral(member)))
}
}
separator := " OR "
if !exists {
separator = " AND "
}
return "(" + strings.Join(presence, separator) + ")"
case v3.FilterOperatorExists:
return fmt.Sprintf("simpleJSONHas(labels, '%s')", key)
case v3.FilterOperatorNotExists:
return fmt.Sprintf("not simpleJSONHas(labels, '%s')", key)
case v3.FilterOperatorRegex, v3.FilterOperatorNotRegex:
return fmt.Sprintf(logsOp, searchKey, chFmtVal)
case v3.FilterOperatorContains, v3.FilterOperatorNotContains:
@@ -123,10 +93,9 @@ func buildIndexFilterForInOperator(key string, op v3.FilterOperator, value inter
// if there are no values to filter on, return an empty string
if len(values) > 0 {
escapedKey := utils.QuoteEscapedStringForContains(key, true)
for _, v := range values {
value := utils.QuoteEscapedStringForContains(v, true)
conditions = append(conditions, fmt.Sprintf("labels %s '%%\"%s\":\"%s\"%%'", sqlOp, escapedKey, value))
conditions = append(conditions, fmt.Sprintf("labels %s '%%\"%s\":\"%s\"%%'", sqlOp, key, value))
}
return "(" + strings.Join(conditions, separator) + ")"
}
@@ -140,34 +109,8 @@ func buildIndexFilterForInOperator(key string, op v3.FilterOperator, value inter
// for like/contains we will use lower index
// we can use lower index for =, in etc but it's difficult to do it for !=, NIN etc
// if as x != "ABC" we cannot predict something like "not lower(labels) like '%%x%%abc%%'". It has it be "not lower(labels) like '%%x%%ABC%%'"
func buildResourceIndexFilter(key string, op v3.FilterOperator, value interface{}, members []string) string {
if len(members) > 1 {
// A negated hint would drop rows where another member holds the value.
switch op {
case v3.FilterOperatorNotEqual,
v3.FilterOperatorNotLike,
v3.FilterOperatorNotILike,
v3.FilterOperatorNotContains,
v3.FilterOperatorNotExists,
v3.FilterOperatorNotRegex,
v3.FilterOperatorNotIn:
return ""
}
conditions := make([]string, 0, len(members))
for _, member := range members {
if condition := buildResourceIndexFilter(member, op, value, []string{member}); condition != "" {
conditions = append(conditions, condition)
}
}
if len(conditions) == 0 {
return ""
}
return "(" + strings.Join(conditions, " OR ") + ")"
}
func buildResourceIndexFilter(key string, op v3.FilterOperator, value interface{}) string {
// not using clickhouseFormattedValue as we don't wan't the quotes
escapedKey := utils.QuoteEscapedStringForContains(key, true)
strVal := fmt.Sprintf("%s", value)
fmtValEscapedForContains := utils.QuoteEscapedStringForContains(strVal, true)
fmtValEscapedForContainsLower := strings.ToLower(fmtValEscapedForContains)
@@ -176,36 +119,36 @@ func buildResourceIndexFilter(key string, op v3.FilterOperator, value interface{
// add index filters
switch op {
case v3.FilterOperatorEqual:
return fmt.Sprintf("labels like '%%%s\":\"%s%%'", escapedKey, fmtValEscapedForContains)
return fmt.Sprintf("labels like '%%%s\":\"%s%%'", key, fmtValEscapedForContains)
case v3.FilterOperatorNotEqual:
return fmt.Sprintf("labels not like '%%%s\":\"%s%%'", escapedKey, fmtValEscapedForContains)
return fmt.Sprintf("labels not like '%%%s\":\"%s%%'", key, fmtValEscapedForContains)
case v3.FilterOperatorLike, v3.FilterOperatorILike:
return fmt.Sprintf("lower(labels) like '%%%s%%%s%%'", escapedKey, fmtValEscapedLower)
return fmt.Sprintf("lower(labels) like '%%%s%%%s%%'", key, fmtValEscapedLower)
case v3.FilterOperatorNotLike, v3.FilterOperatorNotILike:
// cannot apply not contains x%y as y can be somewhere else
return ""
case v3.FilterOperatorContains:
return fmt.Sprintf("lower(labels) like '%%%s%%%s%%'", escapedKey, fmtValEscapedForContainsLower)
return fmt.Sprintf("lower(labels) like '%%%s%%%s%%'", key, fmtValEscapedForContainsLower)
case v3.FilterOperatorNotContains:
// cannot apply not contains x%y as y can be somewhere else
return ""
case v3.FilterOperatorExists:
return fmt.Sprintf("lower(labels) like '%%%s%%'", escapedKey)
return fmt.Sprintf("lower(labels) like '%%%s%%'", key)
case v3.FilterOperatorNotExists:
return fmt.Sprintf("lower(labels) not like '%%%s%%'", escapedKey)
return fmt.Sprintf("lower(labels) not like '%%%s%%'", key)
case v3.FilterOperatorRegex, v3.FilterOperatorNotRegex:
// don't try to do anything for regex.
return ""
case v3.FilterOperatorIn, v3.FilterOperatorNotIn:
return buildIndexFilterForInOperator(key, op, value)
default:
return fmt.Sprintf("labels like '%%%s%%'", escapedKey)
return fmt.Sprintf("labels like '%%%s%%'", key)
}
}
// buildResourceFiltersFromFilterItems builds a list of clickhouse filter strings for resource labels from a FilterSet.
// It skips any filter items that are not resource attributes and checks that the operator is supported and the data type is correct.
func buildResourceFiltersFromFilterItems(fs *v3.FilterSet, resolveSemconvFamilies bool) ([]string, error) {
func buildResourceFiltersFromFilterItems(fs *v3.FilterSet) ([]string, error) {
var conditions []string
if fs == nil || len(fs.Items) == 0 {
return nil, nil
@@ -239,20 +182,12 @@ func buildResourceFiltersFromFilterItems(fs *v3.FilterSet, resolveSemconvFamilie
}
if logsOp, ok := resourceLogOperators[op]; ok {
members := []string{keyName}
if resolveSemconvFamilies {
members = semconv.Members(semconv.KindAttribute, telemetrytypes.FieldKeySelector{
Name: keyName,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
})
}
// the filter
if resourceFilter := buildResourceFilter(logsOp, keyName, op, value, members); resourceFilter != "" {
if resourceFilter := buildResourceFilter(logsOp, keyName, op, value); resourceFilter != "" {
conditions = append(conditions, resourceFilter)
}
// the additional filter for better usage of the index
if resourceIndexFilter := buildResourceIndexFilter(keyName, op, value, members); resourceIndexFilter != "" {
if resourceIndexFilter := buildResourceIndexFilter(keyName, op, value); resourceIndexFilter != "" {
conditions = append(conditions, resourceIndexFilter)
}
} else {
@@ -284,12 +219,12 @@ func buildResourceFiltersFromAggregateAttribute(aggregateAttribute v3.AttributeK
return ""
}
func BuildResourceSubQuery(dbName, tableName string, bucketStart, bucketEnd int64, fs *v3.FilterSet, groupBy []v3.AttributeKey, aggregateAttribute v3.AttributeKey, isLiveTail bool, resolveSemconvFamilies bool) (string, error) {
func BuildResourceSubQuery(dbName, tableName string, bucketStart, bucketEnd int64, fs *v3.FilterSet, groupBy []v3.AttributeKey, aggregateAttribute v3.AttributeKey, isLiveTail bool) (string, error) {
// BUILD THE WHERE CLAUSE
var conditions []string
// only add the resource attributes to the filters here
rs, err := buildResourceFiltersFromFilterItems(fs, resolveSemconvFamilies)
rs, err := buildResourceFiltersFromFilterItems(fs)
if err != nil {
return "", err
}

View File

@@ -5,7 +5,6 @@ import (
"testing"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/stretchr/testify/require"
)
func Test_buildResourceFilter(t *testing.T) {
@@ -89,7 +88,7 @@ func Test_buildResourceFilter(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := buildResourceFilter(tt.args.logsOp, tt.args.key, tt.args.op, tt.args.value, []string{tt.args.key}); got != tt.want {
if got := buildResourceFilter(tt.args.logsOp, tt.args.key, tt.args.op, tt.args.value); got != tt.want {
t.Errorf("buildResourceFilter() = %v, want %v", got, tt.want)
}
})
@@ -283,7 +282,7 @@ func Test_buildResourceIndexFilter(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := buildResourceIndexFilter(tt.args.key, tt.args.op, tt.args.value, []string{tt.args.key}); got != tt.want {
if got := buildResourceIndexFilter(tt.args.key, tt.args.op, tt.args.value); got != tt.want {
t.Errorf("buildResourceIndexFilter() = %v, want %v", got, tt.want)
}
})
@@ -380,7 +379,7 @@ func Test_buildResourceFiltersFromFilterItems(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := buildResourceFiltersFromFilterItems(tt.args.fs, false)
got, err := buildResourceFiltersFromFilterItems(tt.args.fs)
if (err != nil) != tt.wantErr {
t.Errorf("buildResourceFiltersFromFilterItems() error = %v, wantErr %v", err, tt.wantErr)
return
@@ -542,7 +541,7 @@ func Test_buildResourceSubQuery(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := BuildResourceSubQuery("signoz_logs", "distributed_logs_v2_resource", tt.args.bucketStart, tt.args.bucketEnd, tt.args.fs, tt.args.groupBy, tt.args.aggregateAttribute, false, false)
got, err := BuildResourceSubQuery("signoz_logs", "distributed_logs_v2_resource", tt.args.bucketStart, tt.args.bucketEnd, tt.args.fs, tt.args.groupBy, tt.args.aggregateAttribute, false)
if (err != nil) != tt.wantErr {
t.Errorf("buildResourceSubQuery() error = %v, wantErr %v", err, tt.wantErr)
return
@@ -553,58 +552,3 @@ func Test_buildResourceSubQuery(t *testing.T) {
})
}
}
func Test_buildResourceFilterFamily(t *testing.T) {
members := []string{"deployment.environment.name", "deployment.environment"}
require.Equal(t,
"COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') = 'production'",
buildResourceFilter("=", "deployment.environment.name", v3.FilterOperatorEqual, "production", members))
require.Equal(t,
"COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') != 'production'",
buildResourceFilter("!=", "deployment.environment.name", v3.FilterOperatorNotEqual, "production", members))
require.Equal(t,
"(simpleJSONHas(labels, 'deployment.environment.name') OR simpleJSONHas(labels, 'deployment.environment'))",
buildResourceFilter("", "deployment.environment.name", v3.FilterOperatorExists, nil, members))
require.Equal(t,
"(not simpleJSONHas(labels, 'deployment.environment.name') AND not simpleJSONHas(labels, 'deployment.environment'))",
buildResourceFilter("", "deployment.environment.name", v3.FilterOperatorNotExists, nil, members))
}
func Test_buildResourceIndexFilterFamily(t *testing.T) {
members := []string{"deployment.environment.name", "deployment.environment"}
require.Equal(t,
`(labels like '%deployment.environment.name":"production%' OR labels like '%deployment.environment":"production%')`,
buildResourceIndexFilter("deployment.environment.name", v3.FilterOperatorEqual, "production", members))
require.Equal(t, "",
buildResourceIndexFilter("deployment.environment.name", v3.FilterOperatorNotEqual, "production", members))
require.Equal(t, "",
buildResourceIndexFilter("deployment.environment.name", v3.FilterOperatorNotIn, []interface{}{"production"}, members))
}
func TestBuildResourceSubQueryFamily(t *testing.T) {
fs := &v3.FilterSet{Items: []v3.FilterItem{{
Key: v3.AttributeKey{
Key: "deployment.environment.name",
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorEqual,
Value: "production",
}}}
familyOn, err := BuildResourceSubQuery("signoz_traces", "distributed_traces_v3_resource", 1, 2, fs, nil, v3.AttributeKey{}, false, true)
require.NoError(t, err)
require.Contains(t, familyOn, "COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') = 'production'")
require.Contains(t, familyOn, `(labels like '%deployment.environment.name":"production%' OR labels like '%deployment.environment":"production%')`)
familyOff, err := BuildResourceSubQuery("signoz_traces", "distributed_traces_v3_resource", 1, 2, fs, nil, v3.AttributeKey{}, false, false)
require.NoError(t, err)
require.Contains(t, familyOff, "simpleJSONExtractString(labels, 'deployment.environment.name') = 'production'")
require.NotContains(t, familyOff, "COALESCE")
}

View File

@@ -6,25 +6,17 @@ import (
"github.com/ClickHouse/clickhouse-go/v2"
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/semconv"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
func BuildServiceMapQuery(tags []model.TagQuery, resolveSemconvFamilies bool) (string, []interface{}) {
columns := map[string]string{
"deployment_environment": "deployment_environment",
"k8s_cluster_name": "k8s_cluster_name",
"k8s_namespace_name": "k8s_namespace_name",
}
if resolveSemconvFamilies {
for _, member := range semconv.Members(semconv.KindAttribute, telemetrytypes.FieldKeySelector{
Name: "deployment.environment.name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
}) {
columns[strings.ReplaceAll(member, ".", "_")] = "deployment_environment"
}
var (
columns = map[string]struct{}{
"deployment_environment": {},
"k8s_cluster_name": {},
"k8s_namespace_name": {},
}
)
func BuildServiceMapQuery(tags []model.TagQuery) (string, []interface{}) {
var filterQuery string
var namedArgs []interface{}
for _, tag := range tags {
@@ -32,40 +24,39 @@ func BuildServiceMapQuery(tags []model.TagQuery, resolveSemconvFamilies bool) (s
operator := tag.GetOperator()
value := tag.GetValues()
column, ok := columns[key]
if !ok {
if _, ok := columns[key]; !ok {
continue
}
switch operator {
case model.InOperator:
filterQuery += fmt.Sprintf(" AND %s IN @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s IN @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, value))
case model.NotInOperator:
filterQuery += fmt.Sprintf(" AND %s NOT IN @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s NOT IN @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, value))
case model.EqualOperator:
filterQuery += fmt.Sprintf(" AND %s = @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s = @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, value))
case model.NotEqualOperator:
filterQuery += fmt.Sprintf(" AND %s != @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s != @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, value))
case model.ContainsOperator:
filterQuery += fmt.Sprintf(" AND %s LIKE @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s LIKE @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, fmt.Sprintf("%%%s%%", value)))
case model.NotContainsOperator:
filterQuery += fmt.Sprintf(" AND %s NOT LIKE @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s NOT LIKE @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, fmt.Sprintf("%%%s%%", value)))
case model.StartsWithOperator:
filterQuery += fmt.Sprintf(" AND %s LIKE @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s LIKE @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, fmt.Sprintf("%s%%", value)))
case model.NotStartsWithOperator:
filterQuery += fmt.Sprintf(" AND %s NOT LIKE @%s", column, key)
filterQuery += fmt.Sprintf(" AND %s NOT LIKE @%s", key, key)
namedArgs = append(namedArgs, clickhouse.Named(key, fmt.Sprintf("%s%%", value)))
case model.ExistsOperator:
filterQuery += fmt.Sprintf(" AND %s IS NOT NULL", column)
filterQuery += fmt.Sprintf(" AND %s IS NOT NULL", key)
case model.NotExistsOperator:
filterQuery += fmt.Sprintf(" AND %s IS NULL", column)
filterQuery += fmt.Sprintf(" AND %s IS NULL", key)
}
}
return filterQuery, namedArgs

View File

@@ -1,37 +0,0 @@
package services
import (
"testing"
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/stretchr/testify/require"
)
func TestBuildServiceMapQueryFamily(t *testing.T) {
newSpelling := []model.TagQuery{model.NewTagQueryString(model.TagQueryParam{
Key: "deployment.environment.name",
StringValues: []string{"production"},
Operator: model.EqualOperator,
})}
oldSpelling := []model.TagQuery{model.NewTagQueryString(model.TagQueryParam{
Key: "deployment.environment",
StringValues: []string{"production"},
Operator: model.EqualOperator,
})}
query, args := BuildServiceMapQuery(newSpelling, true)
require.Equal(t, " AND deployment_environment = @deployment_environment_name", query)
require.Len(t, args, 1)
query, args = BuildServiceMapQuery(oldSpelling, true)
require.Equal(t, " AND deployment_environment = @deployment_environment", query)
require.Len(t, args, 1)
query, args = BuildServiceMapQuery(newSpelling, false)
require.Equal(t, "", query)
require.Empty(t, args)
query, args = BuildServiceMapQuery(oldSpelling, false)
require.Equal(t, " AND deployment_environment = @deployment_environment", query)
require.Len(t, args, 1)
}

View File

@@ -282,7 +282,7 @@ func buildTracesQuery(start, end, step int64, mq *v3.BuilderQuery, panelType v3.
filterSubQuery = filterSubQuery + " AND " + emptyValuesInGroupByFilter
}
resourceSubQuery, err := resource.BuildResourceSubQuery("signoz_traces", "distributed_traces_v3_resource", bucketStart, bucketEnd, mq.Filters, mq.GroupBy, mq.AggregateAttribute, false, false)
resourceSubQuery, err := resource.BuildResourceSubQuery("signoz_traces", "distributed_traces_v3_resource", bucketStart, bucketEnd, mq.Filters, mq.GroupBy, mq.AggregateAttribute, false)
if err != nil {
return "", err
}

View File

@@ -17,12 +17,12 @@ type Reader interface {
GetInstantQueryMetricsResult(ctx context.Context, query *model.InstantQueryMetricsParams) (*promql.Result, *stats.QueryStats, *model.ApiError)
GetQueryRangeResult(ctx context.Context, query *model.QueryRangeParams) (*promql.Result, *stats.QueryStats, *model.ApiError)
GetTopLevelOperations(ctx context.Context, start, end time.Time, services []string) (*map[string][]string, *model.ApiError)
GetEntryPointOperations(ctx context.Context, orgID valuer.UUID, query *model.GetTopOperationsParams) (*[]model.TopOperationsItem, error)
GetServices(ctx context.Context, orgID valuer.UUID, query *model.GetServicesParams) (*[]model.ServiceItem, *model.ApiError)
GetTopOperations(ctx context.Context, orgID valuer.UUID, query *model.GetTopOperationsParams) (*[]model.TopOperationsItem, *model.ApiError)
GetEntryPointOperations(ctx context.Context, query *model.GetTopOperationsParams) (*[]model.TopOperationsItem, error)
GetServices(ctx context.Context, query *model.GetServicesParams) (*[]model.ServiceItem, *model.ApiError)
GetTopOperations(ctx context.Context, query *model.GetTopOperationsParams) (*[]model.TopOperationsItem, *model.ApiError)
GetUsage(ctx context.Context, query *model.GetUsageParams) (*[]model.UsageItem, error)
GetServicesList(ctx context.Context) (*[]string, error)
GetDependencyGraph(ctx context.Context, orgID valuer.UUID, query *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error)
GetDependencyGraph(ctx context.Context, query *model.GetServicesParams) (*[]model.ServiceMapDependencyResponseItem, error)
GetTTL(ctx context.Context, orgID string, ttlParams *retentiontypes.GetTTLParams) (*retentiontypes.GetTTLResponseItem, *model.ApiError)
GetCustomRetentionTTL(ctx context.Context, orgID string) (*retentiontypes.GetCustomRetentionTTLResponse, error)

View File

@@ -1,6 +1,8 @@
package querybuilder
import (
"strings"
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/antlr4-go/antlr/v4"
@@ -56,6 +58,22 @@ func QueryStringToKeysSelectors(query string) []*telemetrytypes.FieldKeySelector
FieldDataType: key.FieldDataType,
})
}
// todo(tushar): consider reverting changes done to this method in below PR to avoid scope specific checks
// https://github.com/SigNoz/signoz/issues/11374
//
// A scope attribute lets its `scope.`-prefixed name resolve under other contexts too.
// Declared paths (scope.name/scope.version) keep their compound name after
// normalization and address the scope field only, so they get no such selector.
scopePrefix := telemetrytypes.FieldContextScope.StringValue() + "."
if key.FieldContext == telemetrytypes.FieldContextScope && !strings.HasPrefix(key.Name, scopePrefix) {
keys = append(keys, &telemetrytypes.FieldKeySelector{
Name: scopePrefix + key.Name,
Signal: key.Signal,
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: key.FieldDataType,
})
}
}
}

View File

@@ -72,6 +72,41 @@ func TestQueryToKeys(t *testing.T) {
},
},
},
{
// A declared scope path keeps its compound name and addresses the scope field
// only, so it yields a single scope-context selector (no `scope.`-prefixed
// cross-context companion).
query: `scope.version = '1.0.0'`,
expectedKeys: []telemetrytypes.FieldKeySelector{
{
Name: "scope.version",
Signal: telemetrytypes.SignalUnspecified,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
},
},
{
// A scope attribute whose own name carries a `scope.` prefix. `scope.prefixed`
// normalizes to {prefixed, scope}; the second selector re-adds the prefix so the
// metadata fetch can target the attribute's exact key `scope.prefixed` rather than
// relying on the broad `%prefixed%` match.
query: `scope.prefixed = 'x'`,
expectedKeys: []telemetrytypes.FieldKeySelector{
{
Name: "prefixed",
Signal: telemetrytypes.SignalUnspecified,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
{
Name: "scope.prefixed",
Signal: telemetrytypes.SignalUnspecified,
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
},
},
}
for _, testCase := range testCases {

View File

@@ -269,20 +269,14 @@ func adjustTraceKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*te
For example: trace_id (intrinsic), response_status_code (calculated).
*/
// Resolve against the context-qualified name first, then the bare name since that can be instrinsic field e.g. scope.name.
var isIntrinsicOrCalculatedField bool
var intrinsicOrCalculatedField telemetrytypes.TelemetryFieldKey
if _, ok := tracestelemetryschema.IntrinsicFields[key.Name]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.IntrinsicFields[key.Name]
} else if _, ok := tracestelemetryschema.CalculatedFields[key.Name]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.CalculatedFields[key.Name]
} else if _, ok := tracestelemetryschema.IntrinsicFieldsDeprecated[key.Name]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.IntrinsicFieldsDeprecated[key.Name]
} else if _, ok := tracestelemetryschema.CalculatedFieldsDeprecated[key.Name]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.CalculatedFieldsDeprecated[key.Name]
if key.FieldContext != telemetrytypes.FieldContextUnspecified {
intrinsicOrCalculatedField, isIntrinsicOrCalculatedField = lookupIntrinsicOrCalculatedField(key.FieldContext.StringValue() + "." + key.Name)
}
if !isIntrinsicOrCalculatedField {
intrinsicOrCalculatedField, isIntrinsicOrCalculatedField = lookupIntrinsicOrCalculatedField(key.Name)
}
if isIntrinsicOrCalculatedField {
@@ -294,6 +288,24 @@ func adjustTraceKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*te
return actions
}
// lookupIntrinsicOrCalculatedField returns the intrinsic or calculated field registered under
// name, across the current and deprecated tables.
func lookupIntrinsicOrCalculatedField(name string) (telemetrytypes.TelemetryFieldKey, bool) {
if f, ok := tracestelemetryschema.IntrinsicFields[name]; ok {
return f, true
}
if f, ok := tracestelemetryschema.CalculatedFields[name]; ok {
return f, true
}
if f, ok := tracestelemetryschema.IntrinsicFieldsDeprecated[name]; ok {
return f, true
}
if f, ok := tracestelemetryschema.CalculatedFieldsDeprecated[name]; ok {
return f, true
}
return telemetrytypes.TelemetryFieldKey{}, false
}
// buildListQuery builds a query for list panel type.
func (b *traceQueryStatementBuilder) buildListQuery(
ctx context.Context,

View File

@@ -374,6 +374,94 @@ func TestStatementBuilder(t *testing.T) {
},
expectedErr: nil,
},
{
name: "scope.name filter and group by",
requestType: qbtypes.RequestTypeTimeSeries,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Aggregations: []qbtypes.TraceAggregation{
{
Expression: "count()",
},
},
Filter: &qbtypes.Filter{
Expression: "scope.name = 'opentelemetry-io'",
},
Limit: 10,
GroupBy: []qbtypes.GroupByKey{
{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "scope.name",
FieldContext: telemetrytypes.FieldContextScope,
},
},
},
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(scope.name::String <> '', scope.name::String, NULL)) AS `__GROUP_BY_KEY_0_scope.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.name::String = ? AND scope.name::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_scope.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(scope.name::String <> '', scope.name::String, NULL)) AS `__GROUP_BY_KEY_0_scope.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.name::String = ? AND scope.name::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_scope.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_scope.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_scope.name`",
Args: []any{"opentelemetry-io", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "opentelemetry-io", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
},
{
name: "scope.version filter with scope.name group by",
requestType: qbtypes.RequestTypeTimeSeries,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Aggregations: []qbtypes.TraceAggregation{
{
Expression: "count()",
},
},
Filter: &qbtypes.Filter{
Expression: "scope.version = '1.0.0'",
},
Limit: 10,
GroupBy: []qbtypes.GroupByKey{
{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "scope.name",
FieldContext: telemetrytypes.FieldContextScope,
},
},
},
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(scope.name::String <> '', scope.name::String, NULL)) AS `__GROUP_BY_KEY_0_scope.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.version::String = ? AND scope.version::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_scope.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(scope.name::String <> '', scope.name::String, NULL)) AS `__GROUP_BY_KEY_0_scope.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.version::String = ? AND scope.version::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_scope.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_scope.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_scope.name`",
Args: []any{"1.0.0", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1.0.0", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
},
{
name: "scope.version filter only (no scope field in group by)",
requestType: qbtypes.RequestTypeTimeSeries,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Aggregations: []qbtypes.TraceAggregation{
{
Expression: "count()",
},
},
Filter: &qbtypes.Filter{
Expression: "scope.version = '1.0.0'",
},
Limit: 10,
GroupBy: []qbtypes.GroupByKey{
{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "service.name",
},
},
},
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.version::String = ? AND scope.version::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) IS NOT NULL, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL), NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.version::String = ? AND scope.version::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"1.0.0", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1.0.0", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
},
}
fl := flaggertest.New(t)
@@ -800,6 +888,112 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
},
expectedErr: nil,
},
{
name: "List query with scope filter only (no scope in select or group by)",
requestType: qbtypes.RequestTypeRaw,
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.version": {
{
Name: "scope.version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
},
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Filter: &qbtypes.Filter{
Expression: "scope.version = '1.0.0'",
},
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, trace_state AS `__SELECT_KEY_3_trace_state`, parent_span_id AS `__SELECT_KEY_4_parent_span_id`, flags AS `__SELECT_KEY_5_flags`, name AS `__SELECT_KEY_6_name`, kind AS `__SELECT_KEY_7_kind`, kind_string AS `__SELECT_KEY_8_kind_string`, duration_nano AS `__SELECT_KEY_9_duration_nano`, status_code AS `__SELECT_KEY_10_status_code`, status_message AS `__SELECT_KEY_11_status_message`, status_code_string AS `__SELECT_KEY_12_status_code_string`, events AS `__SELECT_KEY_13_events`, links AS `__SELECT_KEY_14_links`, response_status_code AS `__SELECT_KEY_15_response_status_code`, external_http_url AS `__SELECT_KEY_16_external_http_url`, http_url AS `__SELECT_KEY_17_http_url`, external_http_method AS `__SELECT_KEY_18_external_http_method`, http_method AS `__SELECT_KEY_19_http_method`, http_host AS `__SELECT_KEY_20_http_host`, db_name AS `__SELECT_KEY_21_db_name`, db_operation AS `__SELECT_KEY_22_db_operation`, has_error AS `__SELECT_KEY_23_has_error`, is_remote AS `__SELECT_KEY_24_is_remote`, attributes_string, attributes_number, attributes_bool, resources_string FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.version::String = ? AND scope.version::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1.0.0", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
},
{
// Regression test: scope.version in selectFields with no metadata (isColumn=true filters it out)
// must still produce scope.version::String, not scope.attributes.version::String
name: "scope.version in selectFields only, no metadata (intrinsic field fallback)",
requestType: qbtypes.RequestTypeRaw,
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{},
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Filter: &qbtypes.Filter{},
SelectFields: []telemetrytypes.TelemetryFieldKey{
{Name: "scope.version", FieldContext: telemetrytypes.FieldContextUnspecified},
},
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.version::String <> '', scope.version::String, NULL) AS `__SELECT_KEY_3_scope.version` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
},
{
// A scope-context key whose name matches a declared scope path resolves to that
// declared path (scope.name), not the span `name` column and not an undeclared
// scope attribute, even with no metadata.
name: "scope-context name with no metadata resolves to the declared scope path",
requestType: qbtypes.RequestTypeRaw,
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{},
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Filter: &qbtypes.Filter{},
SelectFields: []telemetrytypes.TelemetryFieldKey{
{Name: "name", FieldContext: telemetrytypes.FieldContextScope},
},
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.name::String <> '', scope.name::String, NULL) AS `__SELECT_KEY_3_name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
},
{
// A scope name that collides with a declared path: even with a scope attribute
// literally named `version` in metadata alongside the declared scope.version, a
// select on `{version, scope}` binds to the declared path only. The reserved-name
// attribute is addressed separately as scope.attribute.version.
name: "scope select field binds to the declared path, ignoring a same-named scope attribute",
requestType: qbtypes.RequestTypeRaw,
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.version": {
{
Name: "scope.version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
"version": {
{
Name: "version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
},
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Filter: &qbtypes.Filter{},
SelectFields: []telemetrytypes.TelemetryFieldKey{
{Name: "version", FieldContext: telemetrytypes.FieldContextScope},
},
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.version::String <> '', scope.version::String, NULL) AS `__SELECT_KEY_3_version` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
},
}
for _, c := range cases {

View File

@@ -180,7 +180,7 @@ func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelector
`CASE
// WHEN tagType = 'spanfield' THEN 1
WHEN tagType = 'resource' THEN 2
// WHEN tagType = 'scope' THEN 3
WHEN tagType = 'scope' THEN 3
WHEN tagType = 'tag' THEN 4
ELSE 5
END as priority`,

View File

@@ -391,6 +391,83 @@ func TestConditionForResourceWithEvolution(t *testing.T) {
}
}
// TestConditionForScopeIntrinsicFields covers the scope.name/scope.version intrinsic
// fields against the "scope" JSON column. These are *declared* String paths on that
// column, so a row without a scope reads as ” and never NULL: presence must be an
// empty-string check, since "IS NOT NULL" would hold for every row. That also rules
// out treating them as nested attribute keys under scope.attributes, which are
// undeclared (Dynamic) paths and genuinely NULL when absent.
func TestConditionForScopeIntrinsicFields(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
conditionBuilder := NewConditionBuilder(fm, flaggertest.New(t))
testCases := []struct {
name string
key telemetrytypes.TelemetryFieldKey
operator qbtypes.FilterOperator
value any
expectedSQL string
}{
{
name: "Equal - scope.name",
key: telemetrytypes.TelemetryFieldKey{
Name: "scope.name",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
operator: qbtypes.FilterOperatorEqual,
value: "io.signoz.payment",
expectedSQL: "(scope.name::String = ? AND scope.name::String <> '')",
},
{
name: "Equal - scope.version",
key: telemetrytypes.TelemetryFieldKey{
Name: "scope.version",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
operator: qbtypes.FilterOperatorEqual,
value: "2.3.1",
expectedSQL: "(scope.version::String = ? AND scope.version::String <> '')",
},
{
name: "Exists - scope.name",
key: telemetrytypes.TelemetryFieldKey{
Name: "scope.name",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
operator: qbtypes.FilterOperatorExists,
value: nil,
expectedSQL: "scope.name::String <> ''",
},
{
name: "NotExists - scope.version",
key: telemetrytypes.TelemetryFieldKey{
Name: "scope.version",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
operator: qbtypes.FilterOperatorNotExists,
value: nil,
expectedSQL: "scope.version::String = ''",
},
}
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
conds, _, err := conditionBuilder.ConditionFor(ctx, valuer.UUID{}, 0, 0, &tc.key, map[string][]*telemetrytypes.TelemetryFieldKey{tc.key.Name: {&tc.key}}, qbtypes.ConditionBuilderOptions{}, tc.operator, tc.value, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, tc.expectedSQL)
assert.NotContains(t, sql, "scope.`scope.", "must not double-prefix the scope JSON path")
})
}
}
// TestConditionForSynthesizedKeys covers the KeyNotFound fallback: when a
// referenced attribute key has no metadata match, the builder synthesizes key(s) from
// user input and queries anyway, emitting a warning instead of failing.

View File

@@ -121,6 +121,20 @@ var (
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"scope.name": {
Name: "scope.name",
Description: "Instrumentation scope name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"scope.version": {
Name: "scope.version",
Description: "Instrumentation scope version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
}
IntrinsicFieldsDeprecated = map[string]telemetrytypes.TelemetryFieldKey{
"traceID": {

View File

@@ -53,6 +53,7 @@ var (
ValueType: schema.ColumnTypeString,
}},
"resource": {Name: "resource", Type: schema.JSONColumnType{}},
"scope": {Name: "scope", Type: schema.JSONColumnType{}},
"events": {Name: "events", Type: schema.ArrayColumnType{
ElementType: schema.ColumnTypeString,
@@ -181,7 +182,7 @@ func (m *fieldMapper) getColumn(
case telemetrytypes.FieldContextResource:
return []*schema.Column{indexV3Columns["resource"], indexV3Columns["resources_string"]}, nil
case telemetrytypes.FieldContextScope:
return []*schema.Column{}, qbtypes.ErrColumnNotFound
return []*schema.Column{indexV3Columns["scope"]}, nil
case telemetrytypes.FieldContextAttribute:
switch key.FieldDataType {
case telemetrytypes.FieldDataTypeString:
@@ -292,14 +293,24 @@ func (m *fieldMapper) resolveColumnExprs(
switch column.Type.GetType() {
case schema.ColumnTypeEnumJSON:
// json is only supported for resource context as of now
if key.FieldContext != telemetrytypes.FieldContextResource {
return nil, nil, nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource context fields are supported for json columns, got %s", key.FieldContext.String)
}
// have to add ::string as clickHouse throws an error :- data types Variant/Dynamic are not allowed in GROUP BY
// once clickHouse dependency is updated, we need to check if we can remove it.
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
switch key.FieldContext {
case telemetrytypes.FieldContextResource:
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
case telemetrytypes.FieldContextScope:
if path, ok := declaredScopePath(key); ok {
// declared String paths on the scope column read '' for the missing case
exprs = append(exprs, fmt.Sprintf("%s::String", path))
existExprs = append(existExprs, fmt.Sprintf("%s <> ''", path))
} else {
exprs = append(exprs, fmt.Sprintf("%s.attributes.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.attributes.`%s` IS NOT NULL", columnName, key.Name))
}
default:
return nil, nil, nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource and scope context fields are supported for json columns, got %s", key.FieldContext.String)
}
case schema.ColumnTypeEnumString,
schema.ColumnTypeEnumUInt64,
schema.ColumnTypeEnumUInt32,
@@ -341,9 +352,9 @@ func (m *fieldMapper) resolveColumnExprs(
return exprs, existExprs, columns, nil
}
// logicalForResolvedColumn upgrades a directly-resolvable key (the FieldFor
// probe succeeded) to its family when the metadata map proves membership;
// otherwise the key stays a single-member logical field.
// logicalForResolvedColumn returns the logical field for a directly-resolvable key: its
// semantic-convention family when the metadata map proves membership, otherwise the
// single-member field for the key as given.
func (m *fieldMapper) logicalForResolvedColumn(ctx context.Context, orgID valuer.UUID, field *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) *telemetrytypes.LogicalField {
for _, logical := range querybuilder.MatchingLogicalFields(ctx, orgID, m.fl, field, keys) {
if logical.IsFamily() &&
@@ -423,10 +434,9 @@ func (m *fieldMapper) ColumnExpressionFor(
// map proves membership; otherwise it stays single-member.
candidates = []*telemetrytypes.LogicalField{m.logicalForResolvedColumn(ctx, orgID, field, keys)}
case errors.Is(err, qbtypes.ErrColumnNotFound):
// The legacy candidate flow, unchanged: column (when the bare name is
// one) plus metadata matches, else synthesized type-variant keys. The
// family step below only swaps candidates for their family; it never
// changes candidate order or non-family behavior.
// The legacy candidate flow: column (when the bare name is one) plus metadata
// matches, else synthesized type-variant keys. The family step only swaps candidates
// for their family; it never changes candidate order or non-family behavior.
raw := m.CandidateKeys(ctx, orgID, field, nil, keys)
if len(raw) == 0 {
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys))...)
@@ -599,11 +609,66 @@ func (m *fieldMapper) CandidateKeys(ctx context.Context, _ valuer.UUID, field *t
// strict context honored as-is: stripped interpretation first, literal spelling second
literal := telemetrytypes.NewTelemetryFieldKey(field.FieldContext.StringValue()+"."+field.Name, field.FieldContext, field.FieldDataType)
return append(querybuilder.SynthesizeKeys(field, value), querybuilder.SynthesizeKeys(literal, value)...)
case telemetrytypes.FieldContextScope:
// A short scope name that names a declared scope path (e.g. {name, scope} -> scope.name)
// resolves to that declared path, not an undeclared scope attribute.
if compound := field.FieldContext.StringValue() + "." + field.Name; isDeclaredScopePath(compound) {
return []*telemetrytypes.TelemetryFieldKey{telemetrytypes.NewTelemetryFieldKey(compound, telemetrytypes.FieldContextScope, telemetrytypes.FieldDataTypeString)}
}
return []*telemetrytypes.TelemetryFieldKey{synthScopeAttributeKey(field)}
}
// contexts that don't exist on spans (log, body, scope, …) have nothing to synthesize
// contexts that don't exist on spans (log, body, …) have nothing to synthesize
return nil
}
// synthScopeAttributeKey guesses a scope attribute (scope.attributes.<name>) for a name absent
// from metadata — the scope analog of querybuilder.SynthesizeKeys.
func synthScopeAttributeKey(field *telemetrytypes.TelemetryFieldKey) *telemetrytypes.TelemetryFieldKey {
return telemetrytypes.NewTelemetryFieldKey(field.Name, telemetrytypes.FieldContextScope, telemetrytypes.FieldDataTypeString)
}
func isDeclaredScopePath(name string) bool {
f, ok := IntrinsicFields[name]
return ok && f.FieldContext == telemetrytypes.FieldContextScope
}
// declaredScopePath returns the compound declared scope path (e.g. `scope.name`) for a scope
// key given in either its short ({name, scope}) or already-compound ({scope.name, scope})
// form, and whether it names a declared path at all. Normalization strips the `scope.` prefix,
// so the declared paths reach the renderers in short form; IntrinsicFields keys them compound.
func declaredScopePath(key *telemetrytypes.TelemetryFieldKey) (string, bool) {
if isDeclaredScopePath(key.Name) {
return key.Name, true
}
compound := telemetrytypes.FieldContextScope.StringValue() + "." + key.Name
if isDeclaredScopePath(compound) {
return compound, true
}
return "", false
}
// scopeJSONExistsExpression renders the existence predicate for the scope JSON column, the one
// signal-specific case the generic querybuilder.ExistsExpression must not carry.
func scopeJSONExistsExpression(key *telemetrytypes.TelemetryFieldKey, fieldExpression string, exists bool) (string, bool) {
if key.FieldContext != telemetrytypes.FieldContextScope {
return "", false
}
// Declared String paths are non-Nullable (absent reads '' not NULL).
if _, ok := declaredScopePath(key); ok {
if exists {
return fieldExpression + " <> ''", true
}
return fieldExpression + " = ''", true
}
// Scope attribute: the value expression casts the JSON path to String, which folds a missing
// key's NULL to '', so presence must test the raw path — drop the ::String cast.
path := strings.TrimSuffix(fieldExpression, "::String")
if exists {
return path + " IS NOT NULL", true
}
return path + " IS NULL", true
}
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
func (m *fieldMapper) ExistsFor(
ctx context.Context,
@@ -620,5 +685,8 @@ func (m *fieldMapper) ExistsFor(
if err != nil {
return "", err
}
if expr, ok := scopeJSONExistsExpression(key, fieldExpression, exists); ok {
return expr, nil
}
return querybuilder.ExistsExpression(columns, key, tsStart, tsEnd, fieldExpression, exists)
}

View File

@@ -84,6 +84,33 @@ func TestGetFieldKeyName(t *testing.T) {
expectedResult: "multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, `resource_string_deployment$$environment_exists`, `resource_string_deployment$$environment`, NULL)",
expectedError: nil,
},
{
name: "Scope field - scope.name",
key: telemetrytypes.TelemetryFieldKey{
Name: "scope.name",
FieldContext: telemetrytypes.FieldContextScope,
},
expectedResult: "scope.name::String",
expectedError: nil,
},
{
name: "Scope field - scope.version",
key: telemetrytypes.TelemetryFieldKey{
Name: "scope.version",
FieldContext: telemetrytypes.FieldContextScope,
},
expectedResult: "scope.version::String",
expectedError: nil,
},
{
name: "Scope field - custom attribute",
key: telemetrytypes.TelemetryFieldKey{
Name: "custom.attr",
FieldContext: telemetrytypes.FieldContextScope,
},
expectedResult: "scope.attributes.`custom.attr`::String",
expectedError: nil,
},
{
// Query like `attribute.attribute_string:string` should resolve to `attributes_string['attribute_string']`.
name: "Attribute key whose name collides with contextual map column resolves as a map lookup",
@@ -304,3 +331,78 @@ func TestColumnExpressionForTimestampAttributeCollision(t *testing.T) {
assert.Contains(t, result, "attributes_number['timestamp']")
})
}
// TestColumnExpressionForScopeUnion covers select-side resolution of scope names that
// collide with a declared scope path. A short name under scope context (or the bare
// `scope.<x>` spelling that normalizes to it) binds to the declared path, and unions a
// same-named scope attribute when one is also in metadata. The full `scope.<x>` name under
// explicit scope context addresses the declared path alone.
func TestColumnExpressionForScopeUnion(t *testing.T) {
ctx := context.Background()
scopeKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
}
declaredOnly := map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.name": {scopeKey("scope.name")},
"scope.version": {scopeKey("scope.version")},
}
withAttr := map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.name": {scopeKey("scope.name")},
"scope.version": {scopeKey("scope.version")},
"name": {scopeKey("name")},
"version": {scopeKey("version")},
}
testCases := []struct {
name string
key telemetrytypes.TelemetryFieldKey
keys map[string][]*telemetrytypes.TelemetryFieldKey
expectedResult string
}{
{
name: "short name under scope context binds to the declared path",
key: telemetrytypes.TelemetryFieldKey{Name: "version", FieldContext: telemetrytypes.FieldContextScope},
keys: declaredOnly,
expectedResult: "multiIf(scope.version::String <> '', scope.version::String, NULL)",
},
{
name: "short name binds to the declared path only, ignoring a same-named scope attribute",
key: telemetrytypes.TelemetryFieldKey{Name: "version", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.version::String <> '', scope.version::String, NULL)",
},
{
name: "full scope.version name under scope context addresses the declared path alone",
key: telemetrytypes.TelemetryFieldKey{Name: "scope.version", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.version::String <> '', scope.version::String, NULL)",
},
{
name: "short scope name binds to the declared scope.name only, ignoring a same-named attribute",
key: telemetrytypes.TelemetryFieldKey{Name: "name", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.name::String <> '', scope.name::String, NULL)",
},
{
name: "full scope.name name under scope context addresses the declared path alone",
key: telemetrytypes.TelemetryFieldKey{Name: "scope.name", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.name::String <> '', scope.name::String, NULL)",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fm := NewFieldMapper(flaggertest.New(t))
result, err := fm.ColumnExpressionFor(ctx, valuer.UUID{}, 0, 0, &tc.key, telemetrytypes.FieldDataTypeUnspecified, tc.keys)
require.NoError(t, err)
assert.Equal(t, tc.expectedResult, result)
})
}
}

View File

@@ -113,6 +113,20 @@ func BuildCompleteFieldKeyMap(releaseTime time.Time) map[string][]*telemetrytype
FieldDataType: telemetrytypes.FieldDataTypeBool,
},
},
"scope.name": {
{
Name: "scope.name",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
"scope.version": {
{
Name: "scope.version",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
// both spellings of an enabled semantic-convention family
"deployment.environment.name": {
{

View File

@@ -145,6 +145,19 @@ func (f *TelemetryFieldKey) Equal(key *TelemetryFieldKey) bool {
// key := &TelemetryFieldKey{Name: "resource.service.name:string"}
// key.Normalize()
// // Result: Name: "service.name", FieldContext: FieldContextResource, FieldDataType: FieldDataTypeString
// declaredScopePathSuffixes are the OTel InstrumentationScope fields that address a
// declared top-level path on the scope column rather than a scope attribute. They keep
// their compound `scope.<suffix>` name through normalization.
var declaredScopePathSuffixes = map[string]struct{}{
"name": {},
"version": {},
}
func isDeclaredScopePathSuffix(name string) bool {
_, ok := declaredScopePathSuffixes[name]
return ok
}
func (f *TelemetryFieldKey) Normalize() {
// Step 1: Parse data type from the right (after the last ":") if not already specified
@@ -163,9 +176,17 @@ func (f *TelemetryFieldKey) Normalize() {
if dotIdx := strings.Index(f.Name, "."); dotIdx != -1 {
potentialContext := f.Name[:dotIdx]
if fc, ok := fieldContexts[potentialContext]; ok && fc != FieldContextUnspecified {
f.Name = f.Name[dotIdx+1:]
remainder := f.Name[dotIdx+1:]
f.FieldContext = fc
// The declared scope paths (scope.name / scope.version) keep their compound
// name so they stay distinct from a scope attribute of the same short name.
if fc == FieldContextScope && isDeclaredScopePathSuffix(remainder) {
// f.Name stays as the compound `scope.<suffix>`
} else {
f.Name = remainder
}
// Step 2a: Handle special case for log.body.* fields
if f.FieldContext == FieldContextLog && strings.HasPrefix(f.Name, BodyJSONStringSearchPrefix) {
f.FieldContext = FieldContextBody

View File

@@ -18,7 +18,7 @@ import (
// - Use `scope.` prefix to explicitly indicate and enforce scope context. Example
// - `scope.name`
// - `scope.version`
// - `scope.my.custom.attribute` and `scope.attribute.my.custom.attribute` resolve to same attribute
// - `scope.my.custom.attribute` resolves to the `my.custom.attribute` scope attribute
//
// - Use `attribute.` to explicitly indicate and enforce attribute context. Example
// - `attribute.http.method`
@@ -190,7 +190,7 @@ func (FieldContext) Enum() []any {
FieldContextSpan,
FieldContextTrace,
FieldContextResource,
// FieldContextScope,
FieldContextScope,
FieldContextAttribute,
// FieldContextEvent,
FieldContextBody,

View File

@@ -22,7 +22,7 @@ func TestGetFieldKeyFromKeyText(t *testing.T) {
{
keyText: "scope.name",
expected: TelemetryFieldKey{
Name: "name",
Name: "scope.name",
FieldContext: FieldContextScope,
FieldDataType: FieldDataTypeUnspecified,
},
@@ -30,7 +30,7 @@ func TestGetFieldKeyFromKeyText(t *testing.T) {
{
keyText: "scope.version",
expected: TelemetryFieldKey{
Name: "version",
Name: "scope.version",
FieldContext: FieldContextScope,
FieldDataType: FieldDataTypeUnspecified,
},
@@ -294,6 +294,17 @@ func TestNormalize(t *testing.T) {
FieldDataType: FieldDataTypeString,
},
},
{
name: "Normalize keeps a prefix that does not match the set context",
input: TelemetryFieldKey{
Name: "scope.name",
FieldContext: FieldContextAttribute,
},
expected: TelemetryFieldKey{
Name: "scope.name",
FieldContext: FieldContextAttribute,
},
},
{
name: "Normalize body field",
input: TelemetryFieldKey{

View File

@@ -19,7 +19,6 @@ pytest_plugins = [
"fixtures.traces",
"fixtures.metrics",
"fixtures.queriercommon",
"fixtures.semconvfamilies",
"fixtures.metadata",
"fixtures.meter",
"fixtures.browser",

View File

@@ -999,6 +999,8 @@ def generate_traces_with_corrupt_metadata() -> list[Traces]:
"cloud.provider": "integration",
"cloud.account.id": "000",
"trace_id": "corrupt_data",
"scope_name": "corrupt_data",
"scope.scope.name": "corrupt_data",
},
attributes={
"net.transport": "IP.TCP",
@@ -1007,7 +1009,10 @@ def generate_traces_with_corrupt_metadata() -> list[Traces]:
"http.request.method": "POST",
"http.response.status_code": "200",
"timestamp": "corrupt_data",
"version": "1.0.0",
"scope.scope.version": "1.0.0",
},
scope={"name": "io.signoz.http.server", "version": "2.0.0"},
),
Traces(
timestamp=now - timedelta(seconds=3.5),
@@ -1027,12 +1032,24 @@ def generate_traces_with_corrupt_metadata() -> list[Traces]:
"cloud.provider": "integration",
"cloud.account.id": "000",
"timestamp": "corrupt_data",
"scope.attributes.name": "corrupt_data",
},
attributes={
"db.name": "integration",
"db.operation": "SELECT",
"db.statement": "SELECT * FROM integration",
"trace_d": "corrupt_data",
"scope.attributes.version": "corrupt_data",
},
scope={
"name": "io.opentelemetry.contrib.http",
"version": "1.0.0",
"attributes": {
"telemetry.sdk.language": "cpp",
"name": "not-the-real-name",
"version": "not-the-real-version",
"attributes": "literally-a-key-named-attributes",
},
},
),
Traces(
@@ -1053,12 +1070,15 @@ def generate_traces_with_corrupt_metadata() -> list[Traces]:
"cloud.provider": "integration",
"cloud.account.id": "000",
"duration_nano": "corrupt_data",
"scope.scope.attributes.version": "corrupt_data",
},
attributes={
"http.request.method": "PATCH",
"http.status_code": "404",
"id": "1",
"scope.scope.version": "corrupt_data",
},
scope={"name": "io.signoz.http.client", "version": "2.0.0"},
),
Traces(
timestamp=now - timedelta(seconds=1),
@@ -1077,6 +1097,7 @@ def generate_traces_with_corrupt_metadata() -> list[Traces]:
"host.name": "linux-001",
"cloud.provider": "integration",
"cloud.account.id": "001",
"scope.scope.version": "corrupt_data",
},
attributes={
"message.type": "SENT",
@@ -1084,7 +1105,10 @@ def generate_traces_with_corrupt_metadata() -> list[Traces]:
"messaging.message.id": "001",
"duration_nano": "corrupt_data",
"id": 1,
"scope": "corrupt_data",
"scope.attributes.name": "corrupt_data",
},
scope={"name": "io.signoz.messaging", "version": "3.0.0"},
),
]

View File

@@ -1,64 +0,0 @@
from collections.abc import Callable, Generator
from datetime import UTC, datetime, timedelta
import pytest
from fixtures.logs import Logs
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
PREFIX = "semconv-fam"
CURRENT_KEY = "deployment.environment.name"
OLD_KEY = "deployment.environment"
# Row identities. The span name, the log body, and service.name are the identity.
# Tests compare identity sets filtered by PREFIX, so reruns on a reused stack
# with leftover rows stay stable.
OLD = f"{PREFIX}-old" # only the old spelling, value "production"
NEW = f"{PREFIX}-new" # only the current spelling, value "production"
BOTH = f"{PREFIX}-both" # current "staging" and old "production" - the conflict row
NEITHER = f"{PREFIX}-neither" # no member at all
_ROWS = [
(OLD, {OLD_KEY: "production"}, timedelta(seconds=4)),
(NEW, {CURRENT_KEY: "production"}, timedelta(seconds=3)),
(BOTH, {CURRENT_KEY: "staging", OLD_KEY: "production"}, timedelta(seconds=2)),
(NEITHER, {}, timedelta(seconds=1)),
]
@pytest.fixture(name="family_fleet", scope="function")
def family_fleet(
insert_logs: Callable[[list[Logs]], None],
insert_traces: Callable[[list[Traces]], None],
) -> Generator[datetime]:
"""Yields the base timestamp of the inserted rows."""
now = datetime.now(tz=UTC).replace(microsecond=0) - timedelta(minutes=1)
insert_traces(
[
Traces(
timestamp=now - offset,
duration=timedelta(milliseconds=10),
trace_id=TraceIdGenerator.trace_id(),
span_id=TraceIdGenerator.span_id(),
name=identity,
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources={"service.name": identity, **family},
attributes=dict(family),
)
for identity, family, offset in _ROWS
]
)
insert_logs(
[
Logs(
timestamp=now - offset,
body=identity,
resources={"service.name": identity, **family},
attributes=dict(family),
)
for identity, family, offset in _ROWS
]
)
yield now

View File

@@ -302,6 +302,7 @@ class Traces(ABC):
db_operation: str
has_error: bool
is_remote: str
scope_json: dict[str, Any]
resource: list[TracesResource]
tag_attributes: list[TracesTagAttributes]
@@ -327,6 +328,7 @@ class Traces(ABC):
links: list[TracesLink] = [],
trace_state: str = "",
flags: np.uint32 = 0,
scope: dict[str, Any] = {},
resource_write_mode: Literal["legacy_only", "dual_write"] = "dual_write",
) -> None:
if timestamp is None:
@@ -408,6 +410,33 @@ class Traces(ABC):
# Calculate resource fingerprint
self.resource_fingerprint = LogsOrTracesFingerprint(self.resources_string).calculate()
# Process scope mirroring the InstrumentationScope on the OTLP span.
scope_name = scope.get("name", "")
scope_version = scope.get("version", "")
scope_string = {k: str(v) for k, v in scope.get("attributes", {}).items()}
self.scope_json = {
"name": scope_name,
"version": scope_version,
"attributes": scope_string,
}
scope_keys = {"scope.name": scope_name, "scope.version": scope_version}
scope_keys.update(scope_string)
for k, v in scope_keys.items():
if v == "":
continue
self.tag_attributes.append(
TracesTagAttributes(
timestamp=timestamp,
tag_key=k,
tag_type="scope",
tag_data_type="string",
string_value=v,
number_value=None,
)
)
self.attribute_keys.append(TracesResourceOrAttributeKeys(name=k, datatype="string", tag_type="scope"))
# Process attributes by type and populate custom fields
self.attribute_string = {}
self.attributes_number = {}
@@ -659,6 +688,7 @@ class Traces(ABC):
self.has_error,
self.is_remote,
self.resource_json,
self.scope_json,
],
dtype=object,
)
@@ -689,6 +719,7 @@ class Traces(ABC):
attributes=data.get("attributes", {}),
trace_state=data.get("trace_state", ""),
flags=data.get("flags", 0),
scope=data.get("scope", {}),
)
@classmethod
@@ -828,6 +859,7 @@ def insert_traces_to_clickhouse(conn, traces: list[Traces]) -> None:
"has_error",
"is_remote",
"resource",
"scope",
],
data=[trace.np_arr() for trace in traces],
)

View File

@@ -1240,6 +1240,13 @@ def test_traces_list_span_scope(
lambda x: {"duration_nano": int(x[1].duration_nano), "span_id": x[1].span_id, "timestamp": format_timestamp(x[1].timestamp), "trace_id": x[1].trace_id},
id="select_attribute_duration_order_intrinsic",
),
# Case 9: filter on the intrinsic scope.version. Only x[1] should match.
pytest.param(
BuilderQuery(signal="traces", name="A", select_fields=[TelemetryFieldKey("timestamp")], filter_expression="scope.version = '1.0.0'", limit=1),
HTTPStatus.OK,
lambda x: {"span_id": x[1].span_id, "timestamp": format_timestamp(x[1].timestamp), "trace_id": x[1].trace_id},
id="filter_scope_version",
),
],
)
def test_traces_list_with_corrupt_data(
@@ -1283,6 +1290,162 @@ def test_traces_list_with_corrupt_data(
assert get_rows(response)[0]["data"] == expected(traces)
@pytest.mark.parametrize(
"filter_expression,expected_indices",
[
# Intrinsic scope.name / scope.version resolve to the JSON sub-columns.
pytest.param("scope.name = 'io.signoz.payment'", [1], id="intrinsic_scope_name"),
pytest.param("scope.version = '2.3.1'", [0], id="intrinsic_scope_version"),
# A scope attribute resolves against the scope JSON column's attributes.
pytest.param("scope.telemetry.sdk.language = 'python'", [1], id="scope_attribute"),
# A scope attribute whose own name carries a `scope.` prefix. `scope.prefixed`
# normalizes to {prefixed, scope} and must still resolve to the attribute.
pytest.param("scope.prefixed = 'prefixed-val'", [0], id="scope_prefixed_attribute"),
# `env.tier` is a span attribute on span 0 and a scope attribute on
# span 1. Unprefixed -> no explicit context, so it is checked in every
# applicable context (attribute OR scope) and both spans match.
pytest.param("env.tier = 'gold'", [0, 1], id="bare_cross_context"),
# The explicit `scope.` prefix forces scope context only, so span 0's
# span attribute is ignored — only span 1 matches.
pytest.param("scope.env.tier = 'gold'", [1], id="scope_prefixed_cross_context"),
# `scope.name` binds to the declared scope.name field only (span 0). A scope
# attribute literally named `name` (span 1) is reserved-shadowed and is addressed
# separately as scope.attribute.name, so it does not match here.
pytest.param("scope.name = 'io.signoz.checkout'", [0], id="scope_name_reserved_declared_only"),
# A span attribute literally named `scope.name` is addressed with an explicit
# attribute context; the scope-prefixed spelling binds to the declared field only.
# Span 2 carries attribute scope.name='attr-scope-name'.
pytest.param("attribute.scope.name = 'attr-scope-name'", [2], id="scope_name_attribute_explicit_context"),
# An unprefixed `name` resolves to the intrinsic span `name` column and a
# `name` scope attribute, but NOT the scope.name field. Span 2's span
# name and span 1's scope attribute `name` both equal 'io.signoz.checkout';
# span 0's scope.name field equals it too but is NOT matched.
pytest.param("name = 'io.signoz.checkout'", [1, 2], id="bare_name_excludes_scope_name_field"),
# A value that no resolvable key holds (scope.name/scope.version field,
# a `name`/`version` scope attribute, or a same-named attribute/resource)
# returns nothing.
pytest.param("scope.version = 'corrupt_data'", [], id="scope_version_no_match"),
pytest.param("scope.name = 'corrupt_data'", [], id="scope_name_no_match"),
],
)
def test_traces_list_with_scope_filter(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_traces: Callable[[list[Traces]], None],
filter_expression: str,
expected_indices: list[int],
) -> None:
"""
Setup three spans with different scope key resolution:
- x[0]: scope.name/version 'io.signoz.checkout'/'2.3.1'; span attribute
env.tier='gold'.
- x[1]: scope.name/version 'io.signoz.payment'/'4.5.6'; scope attributes
telemetry.sdk.language='python', env.tier='gold', and a `name` scope
attribute colliding with x[0]'s scope.name value.
- x[2]: span name 'io.signoz.checkout' (colliding with x[0]'s scope.name
value) and a span attribute literally named `scope.name`.
Tests:
- Filtering on scope.name / scope.version / a scope attribute.
- An unprefixed key is resolved across contexts (scope checked alongside
attribute / intrinsic), while a `scope.`-prefixed key is scope-only.
- `scope.name` hits the declared scope.name field only, not a `name` scope
attribute (reserved-shadowed) nor a span attribute literally named
`scope.name` (addressed as `attribute.scope.name`); a bare `name` hits the
span name column and a `name` scope attribute but never the scope.name field.
"""
now = datetime.now(tz=UTC).replace(microsecond=0)
trace_id = TraceIdGenerator.trace_id()
span_ids = [TraceIdGenerator.span_id() for _ in range(3)]
traces = [
Traces(
timestamp=now - timedelta(seconds=4),
duration=timedelta(seconds=2),
trace_id=trace_id,
span_id=span_ids[0],
parent_span_id="",
name="GET /checkout",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources={"service.name": "checkout"},
attributes={"http.request.method": "GET", "env.tier": "gold"},
scope={
"name": "io.signoz.checkout",
"version": "2.3.1",
# a scope attribute whose own name carries a `scope.` prefix
"attributes": {"telemetry.sdk.language": "go", "scope.prefixed": "prefixed-val"},
},
),
Traces(
timestamp=now - timedelta(seconds=2),
duration=timedelta(seconds=1),
trace_id=trace_id,
span_id=span_ids[1],
parent_span_id="",
name="POST /pay",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources={"service.name": "payment"},
attributes={"http.request.method": "POST"},
# env.tier is a scope attribute here (cross-context with span 0);
# `name` is a scope attribute colliding with span 0's scope.name.
scope={
"name": "io.signoz.payment",
"version": "4.5.6",
"attributes": {
"telemetry.sdk.language": "python",
"env.tier": "gold",
"name": "io.signoz.checkout",
},
},
),
Traces(
timestamp=now - timedelta(seconds=1),
duration=timedelta(seconds=1),
trace_id=trace_id,
span_id=span_ids[2],
parent_span_id="",
# span name collides with span 0's scope.name value
name="io.signoz.checkout",
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources={"service.name": "probe"},
# a span attribute named `scope.name`
attributes={"scope.name": "attr-scope-name"},
scope={"name": "span-gamma", "version": "9.9.9"},
),
]
insert_traces(traces)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
start_ms = int((now - timedelta(minutes=1)).timestamp() * 1000)
end_ms = int((now + timedelta(seconds=1)).timestamp() * 1000)
response = make_query_request(
signoz,
token,
start_ms=start_ms,
end_ms=end_ms,
request_type=RequestType.RAW,
queries=[
BuilderQuery(
signal="traces",
name="A",
select_fields=[TelemetryFieldKey("timestamp")],
filter_expression=filter_expression,
limit=10,
).to_dict()
],
)
assert response.status_code == HTTPStatus.OK, response.text
got_span_ids = {row["data"]["span_id"] for row in get_rows(response)}
expected_span_ids = {traces[i].span_id for i in expected_indices}
assert got_span_ids == expected_span_ids
@pytest.mark.parametrize("surface", ["filter", "select", "order"])
def test_traces_list_unknown_span_context_synthesizes(
signoz: types.SigNoz,

View File

@@ -1,218 +0,0 @@
from collections.abc import Callable
from datetime import datetime, timedelta
from http import HTTPStatus
import pytest
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.querier import (
RequestType,
build_aggregation,
build_group_by_field,
build_order_by,
build_raw_query,
build_traces_scalar_query,
get_column_data_from_response,
make_query_request,
)
from fixtures.semconvfamilies import (
BOTH,
CURRENT_KEY,
NEITHER,
NEW,
OLD,
OLD_KEY,
PREFIX,
)
FILTER_MATRIX = [
pytest.param("{key} = 'production'", {OLD, NEW}, id="eq_matches_either_spelling"),
pytest.param("{key} = 'staging'", {BOTH}, id="eq_current_wins_on_conflict"),
pytest.param("{key} != 'production'", {BOTH, NEITHER}, id="neq_keeps_keyless_and_conflict"),
pytest.param("{key} IN ['production', 'staging']", {OLD, NEW, BOTH}, id="in_matches_merged_value"),
pytest.param("{key} NOT IN ['production']", {BOTH, NEITHER}, id="not_in_keeps_keyless"),
pytest.param("{key} LIKE '%prod%'", {OLD, NEW}, id="like_matches_merged_value"),
pytest.param("{key} EXISTS", {OLD, NEW, BOTH}, id="exists_is_any_member"),
pytest.param("{key} NOT EXISTS", {NEITHER}, id="not_exists_is_no_member"),
pytest.param("{key} != 'production' AND {key} EXISTS", {BOTH}, id="neq_composed_with_exists"),
]
LITERAL_MATRIX = [
pytest.param("{key} = 'production'", {NEW}, id="literal_eq_reads_one_spelling"),
pytest.param("{key} != 'production'", {OLD, BOTH, NEITHER}, id="literal_neq_reads_one_spelling"),
]
@pytest.mark.parametrize("expression_template,expected", FILTER_MATRIX)
@pytest.mark.parametrize("requested_key", [CURRENT_KEY, OLD_KEY], ids=["current", "old"])
@pytest.mark.parametrize("context", ["resource", "attribute"])
def test_family_filters(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
family_fleet: datetime,
context: str,
requested_key: str,
expression_template: str,
expected: set[str],
) -> None:
"""The result set is a property of the family, not of the requested spelling."""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
expression = expression_template.format(key=f"{context}.{requested_key}")
response = make_query_request(
signoz,
token,
start_ms=int((family_fleet - timedelta(minutes=2)).timestamp() * 1000),
end_ms=int((family_fleet + timedelta(minutes=1)).timestamp() * 1000),
request_type=RequestType.RAW,
queries=[
build_raw_query(
"A",
"traces",
limit=100,
filter_expression=expression,
order=[build_order_by("timestamp", "asc")],
select_fields=[{"name": "span.name"}],
)
],
)
assert response.status_code == HTTPStatus.OK, response.text
matched = {name for name in get_column_data_from_response(response.json(), "name") if name.startswith(PREFIX)}
assert matched == expected, expression
@pytest.mark.parametrize("expression_template,expected", LITERAL_MATRIX)
def test_flag_off_stays_literal(
signoz_families_off: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
family_fleet: datetime,
expression_template: str,
expected: set[str],
) -> None:
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
expression = expression_template.format(key=f"resource.{CURRENT_KEY}")
response = make_query_request(
signoz_families_off,
token,
start_ms=int((family_fleet - timedelta(minutes=2)).timestamp() * 1000),
end_ms=int((family_fleet + timedelta(minutes=1)).timestamp() * 1000),
request_type=RequestType.RAW,
queries=[
build_raw_query(
"A",
"traces",
limit=100,
filter_expression=expression,
order=[build_order_by("timestamp", "asc")],
select_fields=[{"name": "span.name"}],
)
],
)
assert response.status_code == HTTPStatus.OK, response.text
matched = {name for name in get_column_data_from_response(response.json(), "name") if name.startswith(PREFIX)}
assert matched == expected, expression
@pytest.mark.parametrize("expression_template,expected", LITERAL_MATRIX)
def test_logs_stay_literal_with_flag_on(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
family_fleet: datetime,
expression_template: str,
expected: set[str],
) -> None:
"""Only traces have family support today."""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
expression = expression_template.format(key=f"resource.{CURRENT_KEY}")
response = make_query_request(
signoz,
token,
start_ms=int((family_fleet - timedelta(minutes=2)).timestamp() * 1000),
end_ms=int((family_fleet + timedelta(minutes=1)).timestamp() * 1000),
request_type=RequestType.RAW,
queries=[
build_raw_query(
"A",
"logs",
limit=100,
filter_expression=expression,
order=[build_order_by("timestamp", "asc")],
select_fields=[{"name": "body"}],
)
],
)
assert response.status_code == HTTPStatus.OK, response.text
matched = {body for body in get_column_data_from_response(response.json(), "body") if body.startswith(PREFIX)}
assert matched == expected, expression
def test_group_by_merges_and_echoes_requested_spelling(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
family_fleet: datetime,
) -> None:
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = make_query_request(
signoz,
token,
start_ms=int((family_fleet - timedelta(minutes=2)).timestamp() * 1000),
end_ms=int((family_fleet + timedelta(minutes=1)).timestamp() * 1000),
request_type=RequestType.SCALAR,
queries=[
build_traces_scalar_query(
[build_aggregation("count()")],
filter_expression=f"service.name LIKE '{PREFIX}%'",
group_by=[build_group_by_field(CURRENT_KEY, "string", "resource")],
)
],
)
assert response.status_code == HTTPStatus.OK, response.text
result = response.json()["data"]["data"]["results"][0]
group_column = result["columns"][0]
assert group_column["name"] == CURRENT_KEY, group_column
assert group_column["columnType"] == "group", group_column
groups = {row[0] for row in result["data"]}
assert {"production", "staging"}.issubset(groups), groups
assert None in groups, groups
def test_bare_name_prefers_resource_and_warns(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
family_fleet: datetime,
) -> None:
"""Both contexts carry the family, so a bare name is ambiguous: resolution
warns and keeps the resource side."""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = make_query_request(
signoz,
token,
start_ms=int((family_fleet - timedelta(minutes=2)).timestamp() * 1000),
end_ms=int((family_fleet + timedelta(minutes=1)).timestamp() * 1000),
request_type=RequestType.RAW,
queries=[
build_raw_query(
"A",
"traces",
limit=100,
filter_expression=f"{CURRENT_KEY} = 'production'",
order=[build_order_by("timestamp", "asc")],
select_fields=[{"name": "span.name"}],
)
],
)
assert response.status_code == HTTPStatus.OK, response.text
matched = {name for name in get_column_data_from_response(response.json(), "name") if name.startswith(PREFIX)}
assert matched == {OLD, NEW}
warning = response.json()["data"].get("warning") or {}
messages = " ".join(entry.get("message", "") for entry in warning.get("warnings", []))
assert "ambiguous" in messages.lower(), messages

View File

@@ -1,55 +0,0 @@
import pytest
from testcontainers.core.container import Network
from fixtures import types
from fixtures.signoz import create_signoz
@pytest.fixture(name="signoz", scope="package")
def signoz_semconv_families(
network: Network,
zeus: types.TestContainerDocker,
gateway: types.TestContainerDocker,
sqlstore: types.TestContainerSQL,
clickhouse: types.TestContainerClickhouse,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.SigNoz:
return create_signoz(
network=network,
zeus=zeus,
gateway=gateway,
sqlstore=sqlstore,
clickhouse=clickhouse,
request=request,
pytestconfig=pytestconfig,
cache_key="signoz-semconv-families",
env_overrides={
"SIGNOZ_FLAGGER_CONFIG_BOOLEAN_RESOLVE__SEMCONV__FAMILIES": True,
},
)
@pytest.fixture(name="signoz_families_off", scope="package")
def signoz_families_off(
network: Network,
zeus: types.TestContainerDocker,
gateway: types.TestContainerDocker,
sqlstore: types.TestContainerSQL,
clickhouse: types.TestContainerClickhouse,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.SigNoz:
"""Shares the sqlstore and clickhouse with the flag-on instance, so the
same admin token and seeded rows work."""
return create_signoz(
network=network,
zeus=zeus,
gateway=gateway,
sqlstore=sqlstore,
clickhouse=clickhouse,
request=request,
pytestconfig=pytestconfig,
cache_key="signoz-semconv-families-off",
env_overrides={},
)