Compare commits

..

11 Commits

Author SHA1 Message Date
Nikhil Soni
7f5bc3a4f3 refactor(traces-qb): drop logicalForResolvedColumn
Its family branch is unreachable now that select asks metadata first. Reaching
it requires FieldFor to succeed, which requires a context on the key (getColumn
declines an unspecified one); a key carrying a context has already been put to
MatchingLogicalFields, and only lands here when that came back empty. Calling
the same function again with the same arguments therefore cannot match, and the
method reduces to the single-member field it falls back to.

Instrumenting the branch and running the traces suites confirms it: no call
reaches it with a non-empty match set.

Assisted-by: Claude Opus 5
2026-08-25 16:27:43 +05:30
Nikhil Soni
3425776ca6 test(traces-qb): cover semconv families for context-carrying select keys
TestColumnExpressionForFamilyGroupBy only exercised a bare key, which resolves
through the candidate path. A key carrying a context now resolves through
metadata instead, and nothing covered that the family is still reached there.

Run the existing assertion over all three key shapes. The expression is
identical for each: MatchingLogicalFields returns the grouped family directly,
which is what logicalForResolvedColumn was selecting from the same call.

Assisted-by: Claude Opus 5
2026-08-25 16:17:57 +05:30
Nikhil Soni
39b2fc5cef refactor(traces-qb): restore CandidateKeys metadata match to main's form
The context-filtered bare-name match and the appended `{context}.{name}`
spelling existed to let a scope key find its declared path once getColumn
declined scope `name`/`version`. Resolving select through metadata first moved
that job to MatchingLogicalFields, so the extra matching is dead weight: the
suite is green with main's two straight lookups restored.

Keep the scope synthesize case -- it is not redundant. A scope attribute
metadata does not know still reaches CandidateKeys from the filter path, which
passes nil keys for strict contexts, and without the case it resolves to
nothing and the filter fails with "key not found". No test covered that, so
add one.

Assisted-by: Claude Opus 5
2026-08-25 12:44:58 +05:30
Nikhil Soni
bae9a15f2a fix(traces-qb): ask metadata before the column probe when selecting
ColumnExpressionFor resolved a key by probing FieldFor first, while the filter
path asks metadata first. The probe only answers whether a key resolves to a
column, and was standing in for whether it names one field. For a JSON column
that resolves for either of two homes -- a declared scope path or a same-named
scope attribute -- it reported a single resolved field for an ambiguous name,
so getColumn had to decline scope `name` and `version` to force the key back
onto the candidate path. That made a scope attribute so named unselectable:
it failed with "field not found".

Ask metadata first for a key that carries a context, the way the filter path
does. A bare key is left exactly as it was -- it cannot resolve through the
probe at all, since getColumn needs a context, so it already reaches
CandidateKeys, which consults metadata itself.

The filter path is untouched: its rule that one home wins an ambiguous name is
a deliberate choice, and selecting still coalesces the homes instead.

Assisted-by: Claude Opus 5
2026-08-24 12:40:01 +05:30
Nikhil Soni
3871692b3d fix: resolve declared scope paths and dual-context keys in CandidateKeys
The metadata match in CandidateKeys context-filtered every match, dropping a
bare key's cross-context metadata (a name present in both attribute and
resource resolved to attributes only) and, for a specified context, discarding
the literal {context}.{name} spelling (e.g. an attribute named span.test was
unreachable under span context). Filter only the bare-name matches by context,
take the compound-name match as-is, and keep the unfiltered fallback.

The scope synth branch no longer special-cases the declared scope paths:
getTracesKeys already surfaces scope.name / scope.version as intrinsic keys, so
they resolve through the metadata match; synth now only handles undeclared scope
attributes.

Assisted-by: Claude Opus 4.8
2026-08-21 20:41:07 +05:30
Nikhil Soni
5a7a36b83f fix: fix context check for matched keys 2026-08-21 19:20:12 +05:30
Nikhil Soni
c5264c617f refactor: check for context in key match 2026-08-21 18:55:58 +05:30
Nikhil Soni
2a88baae4f fix: prefer context prefixed keys over bare names if available 2026-08-21 15:03:11 +05:30
Nikhil Soni
e3f9628c5a chore: remove unnecessary redirection with small methods 2026-08-21 12:48:47 +05:30
Nikhil Soni
545ba1d64d fix: add context.name in the key selectors for select clause 2026-08-20 21:18:51 +05:30
Nikhil Soni
f5b6ec6f28 fix: restrict direct query on attributes with same name as declared path 2026-08-20 20:08:01 +05:30
11 changed files with 242 additions and 192 deletions

View File

@@ -1,8 +1,6 @@
package querybuilder
import (
"strings"
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/antlr4-go/antlr/v4"
@@ -61,16 +59,11 @@ func QueryStringToKeysSelectors(query string) []*telemetrytypes.FieldKeySelector
// todo(tushar): consider reverting changes done to this method in below PR to avoid scope specific checks
// https://github.com/SigNoz/signoz/issues/11374
//
// A scope attribute lets its `scope.`-prefixed name resolve under other contexts too.
// Declared paths (scope.name/scope.version) keep their compound name after
// normalization and address the scope field only, so they get no such selector.
scopePrefix := telemetrytypes.FieldContextScope.StringValue() + "."
if key.FieldContext == telemetrytypes.FieldContextScope && !strings.HasPrefix(key.Name, scopePrefix) {
if key.FieldContext == telemetrytypes.FieldContextScope {
keys = append(keys, &telemetrytypes.FieldKeySelector{
Name: scopePrefix + key.Name,
Name: key.FieldContext.StringValue() + "." + key.Name,
Signal: key.Signal,
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldContext: telemetrytypes.FieldContextUnspecified, // this allows 'scope.' prefix for keys with other context as well
FieldDataType: key.FieldDataType,
})
}

View File

@@ -73,17 +73,20 @@ func TestQueryToKeys(t *testing.T) {
},
},
{
// A declared scope path keeps its compound name and addresses the scope field
// only, so it yields a single scope-context selector (no `scope.`-prefixed
// cross-context companion).
query: `scope.version = '1.0.0'`,
expectedKeys: []telemetrytypes.FieldKeySelector{
{
Name: "scope.version",
Name: "version",
Signal: telemetrytypes.SignalUnspecified,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
{
Name: "scope.version",
Signal: telemetrytypes.SignalUnspecified,
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
},
},
{

View File

@@ -163,31 +163,15 @@ func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation])
}
for idx := range query.GroupBy {
groupBy := query.GroupBy[idx]
keySelectors = append(keySelectors, &telemetrytypes.FieldKeySelector{
Name: groupBy.Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: groupBy.FieldContext,
FieldDataType: groupBy.FieldDataType,
})
keySelectors = append(keySelectors, keySelectorsForField(query.GroupBy[idx].TelemetryFieldKey)...)
}
for idx := range query.SelectFields {
keySelectors = append(keySelectors, &telemetrytypes.FieldKeySelector{
Name: query.SelectFields[idx].Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: query.SelectFields[idx].FieldContext,
FieldDataType: query.SelectFields[idx].FieldDataType,
})
keySelectors = append(keySelectors, keySelectorsForField(query.SelectFields[idx])...)
}
for idx := range query.Order {
keySelectors = append(keySelectors, &telemetrytypes.FieldKeySelector{
Name: query.Order[idx].Key.Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: query.Order[idx].Key.FieldContext,
FieldDataType: query.Order[idx].Key.FieldDataType,
})
keySelectors = append(keySelectors, keySelectorsForField(query.Order[idx].Key.TelemetryFieldKey)...)
}
for idx := range keySelectors {
@@ -198,6 +182,26 @@ func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation])
return keySelectors
}
func keySelectorsForField(key telemetrytypes.TelemetryFieldKey) []*telemetrytypes.FieldKeySelector {
selectors := []*telemetrytypes.FieldKeySelector{
{
Name: key.Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: key.FieldContext,
FieldDataType: key.FieldDataType,
},
}
if key.FieldContext != telemetrytypes.FieldContextUnspecified {
selectors = append(selectors, &telemetrytypes.FieldKeySelector{
Name: key.FieldContext.StringValue() + "." + key.Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: key.FieldDataType,
})
}
return selectors
}
// mergeDeprecatedTraceKeys prepends deprecated intrinsic/calculated trace field
// definitions to the keys map. We do this during statement building, not at
// metadata fetch time, because:

View File

@@ -934,13 +934,63 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
},
{
// A scope attribute whose own name literally carries a `scope.` prefix (`scope.prefixed`,
// normalized to {prefixed, scope}) resolves to that attribute in a SELECT even without a
// filter: getKeySelectors emits the reconstructed `scope.prefixed` selector so the metadata
// fetch surfaces it and AdjustKey recovers the full name. Without it the `scope.` prefix is
// lost and it wrongly reads `scope.attributes.prefixed`.
name: "scope-prefixed attribute in selectFields resolves without a filter",
requestType: qbtypes.RequestTypeRaw,
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.prefixed": {
{
Name: "scope.prefixed",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
},
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Filter: &qbtypes.Filter{},
SelectFields: []telemetrytypes.TelemetryFieldKey{
{Name: "prefixed", FieldContext: telemetrytypes.FieldContextScope},
},
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.attributes.`scope.prefixed` IS NOT NULL, scope.attributes.`scope.prefixed`::String, NULL) AS `__SELECT_KEY_3_scope.prefixed` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
},
{
// A scope-context key whose name matches a declared scope path resolves to that
// declared path (scope.name), not the span `name` column and not an undeclared
// scope attribute, even with no metadata.
name: "scope-context name with no metadata resolves to the declared scope path",
// scope attribute. getTracesKeys surfaces the declared path as an intrinsic key
// (metadata.go), which shadows the same-named span intrinsic.
name: "scope-context name resolves to the declared scope path",
requestType: qbtypes.RequestTypeRaw,
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{},
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.name": {
{
Name: "scope.name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
"name": {
{
Name: "name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
},
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
@@ -956,41 +1006,22 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
},
},
{
// A scope name that collides with a declared path: even with a scope attribute
// literally named `version` in metadata alongside the declared scope.version, a
// select on `{version, scope}` binds to the declared path only. The reserved-name
// attribute is addressed separately as scope.attribute.version.
name: "scope select field binds to the declared path, ignoring a same-named scope attribute",
// span.scope.name (span context, name "scope.name") resolves to the declared
// scope path scope.name, not a span attribute literally named scope.name.
name: "span-context scope.name in selectFields resolves to the declared scope path",
requestType: qbtypes.RequestTypeRaw,
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.version": {
{
Name: "scope.version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
"version": {
{
Name: "version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
},
keysMap: map[string][]*telemetrytypes.TelemetryFieldKey{},
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Filter: &qbtypes.Filter{},
SelectFields: []telemetrytypes.TelemetryFieldKey{
{Name: "version", FieldContext: telemetrytypes.FieldContextScope},
{Name: "scope.name", FieldContext: telemetrytypes.FieldContextSpan},
},
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.version::String <> '', scope.version::String, NULL) AS `__SELECT_KEY_3_version` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.name::String <> '', scope.name::String, NULL) AS `__SELECT_KEY_3_scope.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
},

View File

@@ -453,6 +453,19 @@ func TestConditionForScopeIntrinsicFields(t *testing.T) {
value: nil,
expectedSQL: "scope.version::String = ''",
},
{
// `scope.attribute.name` (normalized to {attribute.name, scope}) addresses the scope
// attribute named `name` — the declared `scope.name` path is never reached this way.
name: "Equal - scope.attribute.name reaches the named scope attribute",
key: telemetrytypes.TelemetryFieldKey{
Name: "attribute.name",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
operator: qbtypes.FilterOperatorEqual,
value: "io.signoz.checkout",
expectedSQL: "(scope.attributes.`name`::String = ? AND scope.attributes.`name` IS NOT NULL)",
},
}
for _, tc := range testCases {
@@ -491,6 +504,17 @@ func TestConditionForSynthesizedKeys(t *testing.T) {
assert.Contains(t, args, "timeout")
})
t.Run("scope context with no metadata -> scope attribute", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "custom.attr", FieldContext: telemetrytypes.FieldContextScope}
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, noMatches, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "v", sb)
assert.NoError(t, err, "an undeclared scope attribute must still be filterable")
assert.NotEmpty(t, warnings)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "scope.attributes.`custom.attr`")
})
t.Run("bare key with number operand -> attribute number", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
key := telemetrytypes.TelemetryFieldKey{Name: "http.status"}

View File

@@ -236,8 +236,30 @@ func TestColumnExpressionForFamilyGroupBy(t *testing.T) {
}
fm := NewFieldMapper(familyFlagOn(t))
expr, err := fm.ColumnExpressionFor(context.Background(), valuer.UUID{}, startNs, endNs,
&telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}, telemetrytypes.FieldDataTypeString, fieldKeys)
require.NoError(t, err)
require.Equal(t, "multiIf((multiIf(resource.`deployment.environment.name` IS NOT NULL, resource.`deployment.environment.name`::String, mapContains(resources_string, 'deployment.environment.name'), resources_string['deployment.environment.name'], NULL) IS NOT NULL OR multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, mapContains(resources_string, 'deployment.environment'), resources_string['deployment.environment'], NULL) IS NOT NULL), COALESCE(NULLIF(multiIf(resource.`deployment.environment.name` IS NOT NULL, resource.`deployment.environment.name`::String, mapContains(resources_string, 'deployment.environment.name'), resources_string['deployment.environment.name'], NULL), ''), NULLIF(multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, mapContains(resources_string, 'deployment.environment'), resources_string['deployment.environment'], NULL), ''), ''), NULL)", expr)
// The family must be reached whether or not the key carries a context: a bare key
// resolves through the candidate path, a context-carrying one through metadata.
keys := []struct {
name string
key telemetrytypes.TelemetryFieldKey
}{
{"bare", telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}},
{"with context", telemetrytypes.TelemetryFieldKey{
Name: "deployment.environment.name",
FieldContext: telemetrytypes.FieldContextResource,
}},
{"with context and data type", telemetrytypes.TelemetryFieldKey{
Name: "deployment.environment.name",
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
}},
}
for _, tc := range keys {
t.Run(tc.name, func(t *testing.T) {
key := tc.key
expr, err := fm.ColumnExpressionFor(context.Background(), valuer.UUID{}, startNs, endNs,
&key, telemetrytypes.FieldDataTypeString, fieldKeys)
require.NoError(t, err)
require.Equal(t, "multiIf((multiIf(resource.`deployment.environment.name` IS NOT NULL, resource.`deployment.environment.name`::String, mapContains(resources_string, 'deployment.environment.name'), resources_string['deployment.environment.name'], NULL) IS NOT NULL OR multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, mapContains(resources_string, 'deployment.environment'), resources_string['deployment.environment'], NULL) IS NOT NULL), COALESCE(NULLIF(multiIf(resource.`deployment.environment.name` IS NOT NULL, resource.`deployment.environment.name`::String, mapContains(resources_string, 'deployment.environment.name'), resources_string['deployment.environment.name'], NULL), ''), NULLIF(multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, mapContains(resources_string, 'deployment.environment'), resources_string['deployment.environment'], NULL), ''), ''), NULL)", expr)
})
}
}

View File

@@ -300,16 +300,17 @@ func (m *fieldMapper) resolveColumnExprs(
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
case telemetrytypes.FieldContextScope:
if path, ok := declaredScopePath(key); ok {
if f, ok := IntrinsicFields[key.Name]; ok && f.FieldContext == telemetrytypes.FieldContextScope {
// declared String paths on the scope column read '' for the missing case
exprs = append(exprs, fmt.Sprintf("%s::String", path))
existExprs = append(existExprs, fmt.Sprintf("%s <> ''", path))
exprs = append(exprs, fmt.Sprintf("%s::String", key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s <> ''", key.Name))
} else {
exprs = append(exprs, fmt.Sprintf("%s.attributes.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.attributes.`%s` IS NOT NULL", columnName, key.Name))
attributeName := strings.TrimPrefix(key.Name, "attribute.") // literal "attribute" prefix in attribute keys needs double prefix
exprs = append(exprs, fmt.Sprintf("%s.attributes.%s::String", columnName, querybuilder.ClickHouseIdentifier(attributeName)))
existExprs = append(existExprs, fmt.Sprintf("%s.attributes.%s IS NOT NULL", columnName, querybuilder.ClickHouseIdentifier(attributeName)))
}
default:
return nil, nil, nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource and scope context fields are supported for json columns, got %s", key.FieldContext.String)
return nil, nil, nil, errors.NewInternalf(errors.CodeInternal, "only resource and scope context fields are supported for json columns, got %s", key.FieldContext.String)
}
case schema.ColumnTypeEnumString,
schema.ColumnTypeEnumUInt64,
@@ -352,20 +353,6 @@ func (m *fieldMapper) resolveColumnExprs(
return exprs, existExprs, columns, nil
}
// logicalForResolvedColumn returns the logical field for a directly-resolvable key: its
// semantic-convention family when the metadata map proves membership, otherwise the
// single-member field for the key as given.
func (m *fieldMapper) logicalForResolvedColumn(ctx context.Context, orgID valuer.UUID, field *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) *telemetrytypes.LogicalField {
for _, logical := range querybuilder.MatchingLogicalFields(ctx, orgID, m.fl, field, keys) {
if logical.IsFamily() &&
logical.FieldContext == field.FieldContext &&
(field.FieldDataType == telemetrytypes.FieldDataTypeUnspecified || logical.FieldDataType == field.FieldDataType) {
return logical
}
}
return telemetrytypes.SingleLogicalField(field.Name, field)
}
// upgradeToFamilies swaps single-member candidates for their family when the
// metadata map proves membership. Candidate order and every non-family
// candidate stay exactly as the legacy flow produced them; sibling candidates
@@ -426,24 +413,32 @@ func (m *fieldMapper) ColumnExpressionFor(
keys map[string][]*telemetrytypes.TelemetryFieldKey,
) (string, error) {
// Resolve the candidate logical field(s).
// Resolve the candidate logical field(s). A key carrying a context is asked of metadata
// first, the way the filter path asks: the probe below only answers whether a key
// resolves to a column and stands in for whether it names one field, so a column that
// resolves for either of two homes -- a declared scope path or a same-named scope
// attribute -- would be reported as resolved while still being ambiguous. A bare key
// cannot resolve through the probe at all (getColumn needs a context), so it already
// reaches CandidateKeys, which consults metadata itself, and is left alone here.
var candidates []*telemetrytypes.LogicalField
switch _, err := m.FieldFor(ctx, orgID, startNs, endNs, field); {
case err == nil:
// A directly-resolvable key upgrades to its family when the metadata
// map proves membership; otherwise it stays single-member.
candidates = []*telemetrytypes.LogicalField{m.logicalForResolvedColumn(ctx, orgID, field, keys)}
case errors.Is(err, qbtypes.ErrColumnNotFound):
// The legacy candidate flow: column (when the bare name is one) plus metadata
// matches, else synthesized type-variant keys. The family step only swaps candidates
// for their family; it never changes candidate order or non-family behavior.
raw := m.CandidateKeys(ctx, orgID, field, nil, keys)
if len(raw) == 0 {
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys))...)
if field.FieldContext != telemetrytypes.FieldContextUnspecified {
candidates = querybuilder.MatchingLogicalFields(ctx, orgID, m.fl, field, keys)
}
if len(candidates) == 0 {
switch _, err := m.FieldFor(ctx, orgID, startNs, endNs, field); {
case err == nil:
// Metadata knows nothing about this name (it was asked above, or the key is
// bare and cannot resolve here at all), so the column stands alone.
candidates = []*telemetrytypes.LogicalField{telemetrytypes.SingleLogicalField(field.Name, field)}
case errors.Is(err, qbtypes.ErrColumnNotFound):
raw := m.CandidateKeys(ctx, orgID, field, nil, keys)
if len(raw) == 0 {
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys))...)
}
candidates = m.upgradeToFamilies(ctx, orgID, field, querybuilder.WrapAsLogicalFields(field.Name, raw), keys)
default:
return "", err
}
candidates = m.upgradeToFamilies(ctx, orgID, field, querybuilder.WrapAsLogicalFields(field.Name, raw), keys)
default:
return "", err
}
// Group-by/order (String) and aggregation (String/Float64): every candidate is
@@ -610,43 +605,14 @@ func (m *fieldMapper) CandidateKeys(ctx context.Context, _ valuer.UUID, field *t
literal := telemetrytypes.NewTelemetryFieldKey(field.FieldContext.StringValue()+"."+field.Name, field.FieldContext, field.FieldDataType)
return append(querybuilder.SynthesizeKeys(field, value), querybuilder.SynthesizeKeys(literal, value)...)
case telemetrytypes.FieldContextScope:
// A short scope name that names a declared scope path (e.g. {name, scope} -> scope.name)
// resolves to that declared path, not an undeclared scope attribute.
if compound := field.FieldContext.StringValue() + "." + field.Name; isDeclaredScopePath(compound) {
return []*telemetrytypes.TelemetryFieldKey{telemetrytypes.NewTelemetryFieldKey(compound, telemetrytypes.FieldContextScope, telemetrytypes.FieldDataTypeString)}
}
return []*telemetrytypes.TelemetryFieldKey{synthScopeAttributeKey(field)}
// Declared scope paths (scope.name / scope.version) arrive via metadata as intrinsics;
// anything reaching synth is an undeclared scope attribute.
return []*telemetrytypes.TelemetryFieldKey{telemetrytypes.NewTelemetryFieldKey(field.Name, telemetrytypes.FieldContextScope, telemetrytypes.FieldDataTypeString)}
}
// contexts that don't exist on spans (log, body, …) have nothing to synthesize
return nil
}
// synthScopeAttributeKey guesses a scope attribute (scope.attributes.<name>) for a name absent
// from metadata — the scope analog of querybuilder.SynthesizeKeys.
func synthScopeAttributeKey(field *telemetrytypes.TelemetryFieldKey) *telemetrytypes.TelemetryFieldKey {
return telemetrytypes.NewTelemetryFieldKey(field.Name, telemetrytypes.FieldContextScope, telemetrytypes.FieldDataTypeString)
}
func isDeclaredScopePath(name string) bool {
f, ok := IntrinsicFields[name]
return ok && f.FieldContext == telemetrytypes.FieldContextScope
}
// declaredScopePath returns the compound declared scope path (e.g. `scope.name`) for a scope
// key given in either its short ({name, scope}) or already-compound ({scope.name, scope})
// form, and whether it names a declared path at all. Normalization strips the `scope.` prefix,
// so the declared paths reach the renderers in short form; IntrinsicFields keys them compound.
func declaredScopePath(key *telemetrytypes.TelemetryFieldKey) (string, bool) {
if isDeclaredScopePath(key.Name) {
return key.Name, true
}
compound := telemetrytypes.FieldContextScope.StringValue() + "." + key.Name
if isDeclaredScopePath(compound) {
return compound, true
}
return "", false
}
// scopeJSONExistsExpression renders the existence predicate for the scope JSON column, the one
// signal-specific case the generic querybuilder.ExistsExpression must not carry.
func scopeJSONExistsExpression(key *telemetrytypes.TelemetryFieldKey, fieldExpression string, exists bool) (string, bool) {
@@ -654,7 +620,7 @@ func scopeJSONExistsExpression(key *telemetrytypes.TelemetryFieldKey, fieldExpre
return "", false
}
// Declared String paths are non-Nullable (absent reads '' not NULL).
if _, ok := declaredScopePath(key); ok {
if f, ok := IntrinsicFields[key.Name]; ok && f.FieldContext == telemetrytypes.FieldContextScope {
if exists {
return fieldExpression + " <> ''", true
}

View File

@@ -111,6 +111,18 @@ func TestGetFieldKeyName(t *testing.T) {
expectedResult: "scope.attributes.`custom.attr`::String",
expectedError: nil,
},
{
// `scope.attribute.name` normalizes to {attribute.name, scope}; the literal
// `attribute.` prefix is dropped so it addresses the scope attribute named `name`
// (which the declared `scope.name` path deliberately does not).
name: "Scope field - attribute prefix addresses the named scope attribute",
key: telemetrytypes.TelemetryFieldKey{
Name: "attribute.name",
FieldContext: telemetrytypes.FieldContextScope,
},
expectedResult: "scope.attributes.`name`::String",
expectedError: nil,
},
{
// Query like `attribute.attribute_string:string` should resolve to `attributes_string['attribute_string']`.
name: "Attribute key whose name collides with contextual map column resolves as a map lookup",
@@ -332,12 +344,13 @@ func TestColumnExpressionForTimestampAttributeCollision(t *testing.T) {
})
}
// TestColumnExpressionForScopeUnion covers select-side resolution of scope names that
// TestColumnExpressionForScopeDeclaredPath covers select-side resolution of scope names that
// collide with a declared scope path. A short name under scope context (or the bare
// `scope.<x>` spelling that normalizes to it) binds to the declared path, and unions a
// same-named scope attribute when one is also in metadata. The full `scope.<x>` name under
// explicit scope context addresses the declared path alone.
func TestColumnExpressionForScopeUnion(t *testing.T) {
// `scope.<x>` spelling that normalizes to it) binds to the declared path — a same-named
// scope attribute never shadows it. The full `scope.<x>` name under explicit scope context
// likewise addresses the declared path alone. A `name`/`version` scope attribute is reachable
// only via the explicit `scope.attribute.` prefix.
func TestColumnExpressionForScopeDeclaredPath(t *testing.T) {
ctx := context.Background()
scopeKey := func(name string) *telemetrytypes.TelemetryFieldKey {
@@ -371,30 +384,41 @@ func TestColumnExpressionForScopeUnion(t *testing.T) {
keys: declaredOnly,
expectedResult: "multiIf(scope.version::String <> '', scope.version::String, NULL)",
},
{
name: "short name binds to the declared path only, ignoring a same-named scope attribute",
key: telemetrytypes.TelemetryFieldKey{Name: "version", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.version::String <> '', scope.version::String, NULL)",
},
{
name: "full scope.version name under scope context addresses the declared path alone",
key: telemetrytypes.TelemetryFieldKey{Name: "scope.version", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.version::String <> '', scope.version::String, NULL)",
},
{
name: "short scope name binds to the declared scope.name only, ignoring a same-named attribute",
key: telemetrytypes.TelemetryFieldKey{Name: "name", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.name::String <> '', scope.name::String, NULL)",
},
{
name: "full scope.name name under scope context addresses the declared path alone",
key: telemetrytypes.TelemetryFieldKey{Name: "scope.name", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.name::String <> '', scope.name::String, NULL)",
},
{
// `scope.attribute.name` normalizes to {attribute.name, scope}; the `attribute.`
// prefix is dropped so it addresses the scope attribute named `name` — the only
// way to reach it, since `scope.name` is reserved for the declared path.
name: "attribute prefix reaches the named scope attribute",
key: telemetrytypes.TelemetryFieldKey{Name: "attribute.name", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.attributes.`name` IS NOT NULL, scope.attributes.`name`::String, NULL)",
},
{
// metadata knows both homes under this name, so the short spelling coalesces
// them instead of being rejected as ambiguous
name: "short name coalesces a known scope attribute with the declared path",
key: telemetrytypes.TelemetryFieldKey{Name: "name", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.attributes.`name` IS NOT NULL, toString(scope.attributes.`name`::String), scope.name::String <> '', toString(scope.name::String), NULL)",
},
{
name: "short version coalesces a known scope attribute with the declared path",
key: telemetrytypes.TelemetryFieldKey{Name: "version", FieldContext: telemetrytypes.FieldContextScope},
keys: withAttr,
expectedResult: "multiIf(scope.attributes.`version` IS NOT NULL, toString(scope.attributes.`version`::String), scope.version::String <> '', toString(scope.version::String), NULL)",
},
}
for _, tc := range testCases {

View File

@@ -145,19 +145,6 @@ func (f *TelemetryFieldKey) Equal(key *TelemetryFieldKey) bool {
// key := &TelemetryFieldKey{Name: "resource.service.name:string"}
// key.Normalize()
// // Result: Name: "service.name", FieldContext: FieldContextResource, FieldDataType: FieldDataTypeString
// declaredScopePathSuffixes are the OTel InstrumentationScope fields that address a
// declared top-level path on the scope column rather than a scope attribute. They keep
// their compound `scope.<suffix>` name through normalization.
var declaredScopePathSuffixes = map[string]struct{}{
"name": {},
"version": {},
}
func isDeclaredScopePathSuffix(name string) bool {
_, ok := declaredScopePathSuffixes[name]
return ok
}
func (f *TelemetryFieldKey) Normalize() {
// Step 1: Parse data type from the right (after the last ":") if not already specified
@@ -176,17 +163,9 @@ func (f *TelemetryFieldKey) Normalize() {
if dotIdx := strings.Index(f.Name, "."); dotIdx != -1 {
potentialContext := f.Name[:dotIdx]
if fc, ok := fieldContexts[potentialContext]; ok && fc != FieldContextUnspecified {
remainder := f.Name[dotIdx+1:]
f.Name = f.Name[dotIdx+1:]
f.FieldContext = fc
// The declared scope paths (scope.name / scope.version) keep their compound
// name so they stay distinct from a scope attribute of the same short name.
if fc == FieldContextScope && isDeclaredScopePathSuffix(remainder) {
// f.Name stays as the compound `scope.<suffix>`
} else {
f.Name = remainder
}
// Step 2a: Handle special case for log.body.* fields
if f.FieldContext == FieldContextLog && strings.HasPrefix(f.Name, BodyJSONStringSearchPrefix) {
f.FieldContext = FieldContextBody

View File

@@ -22,7 +22,7 @@ func TestGetFieldKeyFromKeyText(t *testing.T) {
{
keyText: "scope.name",
expected: TelemetryFieldKey{
Name: "scope.name",
Name: "name",
FieldContext: FieldContextScope,
FieldDataType: FieldDataTypeUnspecified,
},
@@ -30,7 +30,7 @@ func TestGetFieldKeyFromKeyText(t *testing.T) {
{
keyText: "scope.version",
expected: TelemetryFieldKey{
Name: "scope.version",
Name: "version",
FieldContext: FieldContextScope,
FieldDataType: FieldDataTypeUnspecified,
},

View File

@@ -1308,19 +1308,23 @@ def test_traces_list_with_corrupt_data(
# The explicit `scope.` prefix forces scope context only, so span 0's
# span attribute is ignored — only span 1 matches.
pytest.param("scope.env.tier = 'gold'", [1], id="scope_prefixed_cross_context"),
# `scope.name` binds to the declared scope.name field only (span 0). A scope
# attribute literally named `name` (span 1) is reserved-shadowed and is addressed
# separately as scope.attribute.name, so it does not match here.
pytest.param("scope.name = 'io.signoz.checkout'", [0], id="scope_name_reserved_declared_only"),
# A span attribute literally named `scope.name` is addressed with an explicit
# attribute context; the scope-prefixed spelling binds to the declared field only.
# Span 2 carries attribute scope.name='attr-scope-name'.
pytest.param("attribute.scope.name = 'attr-scope-name'", [2], id="scope_name_attribute_explicit_context"),
# An unprefixed `name` resolves to the intrinsic span `name` column and a
# `name` scope attribute, but NOT the scope.name field. Span 2's span
# name and span 1's scope attribute `name` both equal 'io.signoz.checkout';
# span 0's scope.name field equals it too but is NOT matched.
pytest.param("name = 'io.signoz.checkout'", [1, 2], id="bare_name_excludes_scope_name_field"),
# `scope.name` binds to the declared scope.name field ONLY (span 0). A same-named
# `name` scope attribute (span 1) does NOT shadow or union with it — query that
# attribute as `scope.attribute.name` instead.
pytest.param("scope.name = 'io.signoz.checkout'", [0], id="scope_name_collision"),
# `scope.name` is the declared path only; it does not cross-match a span attribute
# literally named `scope.name` (span 2's attribute scope.name='attr-scope-name'),
# whose declared scope.name is 'span-gamma'. So nothing matches.
pytest.param("scope.name = 'attr-scope-name'", [], id="scope_name_declared_only"),
# A `name`/`version` scope attribute is reachable only via the explicit
# `scope.attribute.` prefix. Span 1 has a `name` scope attribute = 'io.signoz.checkout'.
pytest.param("scope.attribute.name = 'io.signoz.checkout'", [1], id="scope_attribute_name"),
# `version` as a scope attribute: no span carries one (span 1's 4.5.6 is the declared
# scope.version, not a scope attribute), so this matches nothing.
pytest.param("scope.attribute.version = '4.5.6'", [], id="scope_attribute_version_none"),
# An unprefixed `name` resolves to the span `name` column only (span 2). It matches
# neither the scope.name field (span 0) nor a `name` scope attribute (span 1).
pytest.param("name = 'io.signoz.checkout'", [2], id="bare_name_excludes_scope_name_field"),
# A value that no resolvable key holds (scope.name/scope.version field,
# a `name`/`version` scope attribute, or a same-named attribute/resource)
# returns nothing.
@@ -1350,10 +1354,10 @@ def test_traces_list_with_scope_filter(
- Filtering on scope.name / scope.version / a scope attribute.
- An unprefixed key is resolved across contexts (scope checked alongside
attribute / intrinsic), while a `scope.`-prefixed key is scope-only.
- `scope.name` hits the declared scope.name field only, not a `name` scope
attribute (reserved-shadowed) nor a span attribute literally named
`scope.name` (addressed as `attribute.scope.name`); a bare `name` hits the
span name column and a `name` scope attribute but never the scope.name field.
- `scope.name`/`scope.version` bind to the declared JSON sub-columns only; a
same-named `name`/`version` scope attribute is reachable only via the explicit
`scope.attribute.` prefix, never via `scope.name` or a bare `name`.
- a bare `name` resolves to the span `name` column and never the scope.name field.
"""
now = datetime.now(tz=UTC).replace(microsecond=0)
trace_id = TraceIdGenerator.trace_id()