Compare commits

...

8 Commits

Author SHA1 Message Date
Nikhil Soni
145673b6ce refactor(traces-qb): resolve scope through the shared candidate path
Scope context names a column, not one of the two homes on it, so a short
declared name (`name`) meant either the declared path or a same-named scope
attribute. ColumnExpressionFor carried a scope branch to resolve that, because
its FieldFor probe answers resolvability and was standing in for uniqueness:
the scope column always resolves, so the probe reported a unique field for an
ambiguous name and skipped the union the filter path built.

Give each home a spelling instead, the way a real column and a same-named
attribute are addressed elsewhere: `scope.name` and `scope.version` resolve to
the declared paths, and a scope attribute they shadow stays reachable as
`scope.attributes.<name>`. Normalize already strips only the context prefix, so
the `attributes.` spelling arrives intact and needs no boundary change.

Every scope key now names exactly one home, which makes the probe honest and
removes the branch: scope resolves through the same path as every other
context. Declared paths take precedence over same-named scope attributes rather
than unioning with them.

Assisted-by: Claude Opus 5
2026-08-20 15:24:59 +05:30
Nikhil Soni
3eb4c26164 fix(traces-qb): resolve scope keys by their qualified intrinsic name
Scope intrinsics are registered fully qualified (scope.name, scope.version)
while span intrinsics are registered bare, so looking a scope key up by its
bare name matched the span intrinsic and flipped the key to span context.
Look intrinsics up by the key's qualified name instead, which lets scope keys
take the same intrinsic path as every other context; a scope key that misses
stays qualified rather than falling back to the bare name, since a scope
attribute named duration_nano is not the duration_nano column.

AdjustKey now adopts the canonical name of the intrinsic it resolved to, the
way its metadata path already does. This is a no-op for every lookup keyed by
the key's own name.

CandidateKeys stopped falling through to the name-only metadata match for
scope keys, which was pulling in same-named span and attribute entries.

Assisted-by: Claude Opus 5
2026-08-19 19:21:15 +05:30
Nikhil Soni
5868bdb6af docs(traces-qb): explain why scope skips the intrinsic override
Clarify that the intrinsic-override path keys on name alone and applies via
OverrideMetadataFrom (which cannot rename), so it structurally cannot resolve a
scope declared path and is skipped for scope keys.

Assisted-by: Claude Opus 4.8
2026-08-19 17:55:03 +05:30
Nikhil Soni
dffa81e72d fix(traces-qb): fetch scope keys under their full scope.-prefixed name
A scope attribute whose flattened name begins with 'scope.' (e.g. an OTel
scope attribute nested under 'scope', flattened to 'scope.prefixed') is
indistinguishable from the 'scope.' context prefix after Normalize strips it,
so QueryStringToKeysSelectors only fetched metadata for the stripped name and
resolution mis-generated scope.attributes.`prefixed` (empty) instead of
scope.attributes.`scope.prefixed`. Also emit a selector under the full
scope.-prefixed name (tracked by #11374).

Assisted-by: Claude Opus 4.8
2026-08-19 17:07:52 +05:30
Nikhil Soni
2e2fbf0ad4 docs(telemetrytypes): drop scope.attribute. from the scope prefix example
The scope.attribute(s). form is not supported; scope.<name> addresses the
scope attribute directly.

Assisted-by: Claude Opus 4.8
2026-08-19 15:49:34 +05:30
Nikhil Soni
34bbe72405 fix(traces-qb): keep scope-context keys in scope in adjustTraceKey
adjustTraceKey matched a short scope name (e.g. {name, scope}, also the
normalized form of {scope.name, ""}) against the span 'name' intrinsic and
overrode its context to span, resolving the span name column instead of the
declared scope.name path. Skip the intrinsic/calculated override for scope keys
and let the field mapper's scope handling resolve them.

Assisted-by: Claude Opus 4.8
2026-08-19 11:45:37 +05:30
Nikhil Soni
b277825701 chore(traces-qb): drop scope.attribute(s). prefix collapse
scope.<attr> already queries the <attr> scope attribute via the existing
scope. prefix handling in Normalize; the explicit scope.attribute(s). form
is not needed, so revert the Normalize change.

Assisted-by: Claude Opus 4.8
2026-08-19 11:38:34 +05:30
Nikhil Soni
e760755e1c feat(traces-qb): support querying the scope JSON column
Resolve scope.name/scope.version (declared typed paths) and arbitrary
scope.<attr> instrumentation-scope attributes in the traces query builder.

- register the scope JSON column and map FieldContextScope to it
- declared paths read scope.name::String (guard <> ''); attributes read
  scope.attributes.`<name>`::String (guard IS NOT NULL)
- short-name references union a same-named scope attribute (attribute-first)
  with the declared path via MatchingLogicalFields; CandidateKeys resolves a
  declared suffix without depending on the intrinsic being in the metadata map
- signal-specific exists predicate for the two scope homes
- Normalize() strips a leading scope.attribute(s). prefix
- scope.name/scope.version intrinsics + tagType='scope' discovery priority
- enable FieldContextScope in FieldContext.Enum()

Assisted-by: Claude Opus 4.8
2026-08-19 11:10:45 +05:30
13 changed files with 617 additions and 24 deletions

View File

@@ -147,6 +147,11 @@ func AdjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemet
// So we can safely override the context and data type
actions = append(actions, fmt.Sprintf("Overriding key: %s to %s", key, intrinsicOrCalculatedField))
// Adopt the canonical name of the field it resolved to, the same way the metadata
// path below does. This is a no-op when the caller looked the field up by the key's
// own name, and carries the qualified name for fields registered under one
// (`name` with scope context -> `scope.name`).
key.Name = intrinsicOrCalculatedField.Name
key.OverrideMetadataFrom(intrinsicOrCalculatedField)
return actions

View File

@@ -56,6 +56,21 @@ func QueryStringToKeysSelectors(query string) []*telemetrytypes.FieldKeySelector
FieldDataType: key.FieldDataType,
})
}
// A scope attribute whose flattened name begins with `scope.` (e.g. an OTel
// scope attribute nested under `scope`) is indistinguishable from the `scope.`
// context prefix after Normalize strips it. Also fetch the metadata key under
// its full `scope.`-prefixed name so resolution can find it.
// todo(tushar): consider reverting changes done to this method in below PR to avoid scope specific checks
// https://github.com/SigNoz/signoz/issues/11374
if key.FieldContext == telemetrytypes.FieldContextScope {
keys = append(keys, &telemetrytypes.FieldKeySelector{
Name: key.FieldContext.StringValue() + "." + key.Name,
Signal: key.Signal,
FieldContext: telemetrytypes.FieldContextUnspecified, // this allows 'scope.' prefix for keys with other context as well
FieldDataType: key.FieldDataType,
})
}
}
}

View File

@@ -72,6 +72,26 @@ func TestQueryToKeys(t *testing.T) {
},
},
},
{
// A scope reference also fetches its full `scope.`-prefixed name so a scope
// attribute whose flattened name begins with `scope.` (e.g. `scope.prefixed`)
// is discoverable after Normalize strips the prefix.
query: `scope.prefixed = 'local'`,
expectedKeys: []telemetrytypes.FieldKeySelector{
{
Name: "prefixed",
Signal: telemetrytypes.SignalUnspecified,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
{
Name: "scope.prefixed",
Signal: telemetrytypes.SignalUnspecified,
FieldContext: telemetrytypes.FieldContextUnspecified,
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
},
},
},
}
for _, testCase := range testCases {

View File

@@ -259,6 +259,23 @@ func adjustTraceKeys(keys map[string][]*telemetrytypes.TelemetryFieldKey, query
return actions
}
// intrinsicLookupName returns the name under which a key is registered in the intrinsic and
// calculated field tables. Span-context intrinsics are registered bare (`name`, `duration_nano`)
// while scope intrinsics are registered fully qualified (`scope.name`, `scope.version`), so a
// scope key must be looked up qualified — bare lookup would match the span intrinsic of the same
// name. A scope key that misses stays qualified rather than falling back to the bare name: a
// scope attribute named `duration_nano` is not the span `duration_nano` column.
func intrinsicLookupName(key *telemetrytypes.TelemetryFieldKey) string {
if key.FieldContext != telemetrytypes.FieldContextScope {
return key.Name
}
prefix := telemetrytypes.FieldContextScope.StringValue() + "."
if strings.HasPrefix(key.Name, prefix) {
return key.Name
}
return prefix + key.Name
}
// adjustTraceKey resolves a single TelemetryFieldKey against the keys map.
func adjustTraceKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
@@ -269,20 +286,22 @@ func adjustTraceKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*te
For example: trace_id (intrinsic), response_status_code (calculated).
*/
lookupName := intrinsicLookupName(key)
var isIntrinsicOrCalculatedField bool
var intrinsicOrCalculatedField telemetrytypes.TelemetryFieldKey
if _, ok := tracestelemetryschema.IntrinsicFields[key.Name]; ok {
if _, ok := tracestelemetryschema.IntrinsicFields[lookupName]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.IntrinsicFields[key.Name]
} else if _, ok := tracestelemetryschema.CalculatedFields[key.Name]; ok {
intrinsicOrCalculatedField = tracestelemetryschema.IntrinsicFields[lookupName]
} else if _, ok := tracestelemetryschema.CalculatedFields[lookupName]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.CalculatedFields[key.Name]
} else if _, ok := tracestelemetryschema.IntrinsicFieldsDeprecated[key.Name]; ok {
intrinsicOrCalculatedField = tracestelemetryschema.CalculatedFields[lookupName]
} else if _, ok := tracestelemetryschema.IntrinsicFieldsDeprecated[lookupName]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.IntrinsicFieldsDeprecated[key.Name]
} else if _, ok := tracestelemetryschema.CalculatedFieldsDeprecated[key.Name]; ok {
intrinsicOrCalculatedField = tracestelemetryschema.IntrinsicFieldsDeprecated[lookupName]
} else if _, ok := tracestelemetryschema.CalculatedFieldsDeprecated[lookupName]; ok {
isIntrinsicOrCalculatedField = true
intrinsicOrCalculatedField = tracestelemetryschema.CalculatedFieldsDeprecated[key.Name]
intrinsicOrCalculatedField = tracestelemetryschema.CalculatedFieldsDeprecated[lookupName]
}
if isIntrinsicOrCalculatedField {

View File

@@ -675,6 +675,107 @@ func TestStatementBuilderListQuery(t *testing.T) {
},
expectedErr: nil,
},
{
name: "List query selecting and filtering scope fields",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Filter: &qbtypes.Filter{
Expression: "scope.name = 'otelcol'",
},
Limit: 10,
SelectFields: []telemetrytypes.TelemetryFieldKey{
{
Name: "scope.name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
{
Name: "telemetry.sdk.language",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
},
},
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.name::String <> '', scope.name::String, NULL) AS `__SELECT_KEY_3_scope.name`, multiIf(scope.attributes.`telemetry.sdk.language` IS NOT NULL, scope.attributes.`telemetry.sdk.language`::String, NULL) AS `__SELECT_KEY_4_telemetry.sdk.language` FROM signoz_traces.distributed_signoz_index_v3 WHERE (scope.name::String = ? AND scope.name::String <> '') AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"otelcol", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
},
{
// Short scope names (`name`/`version`) collide with span intrinsics; adjustTraceKeys
// must keep them in scope and resolve the declared paths, not the span `name` column.
name: "List query selecting short scope declared names",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Limit: 10,
SelectFields: []telemetrytypes.TelemetryFieldKey{
{
Name: "name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
},
{
Name: "version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
},
},
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.name::String <> '', scope.name::String, NULL) AS `__SELECT_KEY_3_scope.name`, multiIf(scope.version::String <> '', scope.version::String, NULL) AS `__SELECT_KEY_4_scope.version` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
},
{
// `scope.name` resolves to the declared path; the scope attribute of the same name
// stays reachable by addressing the attributes home explicitly.
name: "List query selecting declared scope path and its shadowed attribute",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Limit: 10,
SelectFields: []telemetrytypes.TelemetryFieldKey{
{Name: "name", Signal: telemetrytypes.SignalTraces, FieldContext: telemetrytypes.FieldContextScope},
{Name: "attributes.name", Signal: telemetrytypes.SignalTraces, FieldContext: telemetrytypes.FieldContextScope},
},
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.name::String <> '', scope.name::String, NULL) AS `__SELECT_KEY_3_scope.name`, multiIf(scope.attributes.`name` IS NOT NULL, scope.attributes.`name`::String, NULL) AS `__SELECT_KEY_4_attributes.name` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
},
{
// A scope attribute may share its name with a span intrinsic. It must resolve to the
// scope attribute, never to the span column of that name.
name: "List query selecting scope attribute colliding with span intrinsic",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Limit: 10,
SelectFields: []telemetrytypes.TelemetryFieldKey{
{
Name: "duration_nano",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
},
},
},
expected: qbtypes.Statement{
Query: "SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, multiIf(scope.attributes.`duration_nano` IS NOT NULL, scope.attributes.`duration_nano`::String, NULL) AS `__SELECT_KEY_3_duration_nano` FROM signoz_traces.distributed_signoz_index_v3 WHERE timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
},
}
fl := flaggertest.New(t)

View File

@@ -180,7 +180,7 @@ func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelector
`CASE
// WHEN tagType = 'spanfield' THEN 1
WHEN tagType = 'resource' THEN 2
// WHEN tagType = 'scope' THEN 3
WHEN tagType = 'scope' THEN 3
WHEN tagType = 'tag' THEN 4
ELSE 5
END as priority`,

View File

@@ -585,3 +585,101 @@ func TestConditionForSynthesizedKeys(t *testing.T) {
assert.NotContains(t, sql, "mapContains")
})
}
// TestConditionForScope covers filters on the scope JSON column: declared paths, scope
// attributes, exists semantics, and the attribute-first union when a scope attribute
// shares a declared path's name.
func TestConditionForScope(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
cb := NewConditionBuilder(fm, flaggertest.New(t))
scopeName := IntrinsicFields["scope.name"]
declared := map[string][]*telemetrytypes.TelemetryFieldKey{"scope.name": {&scopeName}}
build := func(key telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) (string, []any) {
t.Helper()
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, keys, qbtypes.ConditionBuilderOptions{}, op, value, sb)
require.NoError(t, err)
sb.Where(sb.Or(conds...))
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
t.Run("declared scope.name equality is exists-guarded", func(t *testing.T) {
key := telemetrytypes.TelemetryFieldKey{Name: "name", FieldContext: telemetrytypes.FieldContextScope}
sql, args := build(key, declared, qbtypes.FilterOperatorEqual, "otelcol")
assert.Contains(t, sql, "scope.name::String = ?")
assert.Contains(t, sql, "scope.name::String <> ''")
assert.Contains(t, args, "otelcol")
})
t.Run("declared scope.name exists", func(t *testing.T) {
key := telemetrytypes.TelemetryFieldKey{Name: "name", FieldContext: telemetrytypes.FieldContextScope}
sql, _ := build(key, declared, qbtypes.FilterOperatorExists, nil)
assert.Contains(t, sql, "scope.name::String <> ''")
})
t.Run("scope attribute equality guards the raw JSON path", func(t *testing.T) {
key := telemetrytypes.TelemetryFieldKey{Name: "telemetry.sdk.language", FieldContext: telemetrytypes.FieldContextScope}
keys := map[string][]*telemetrytypes.TelemetryFieldKey{
"telemetry.sdk.language": {{Name: "telemetry.sdk.language", Signal: telemetrytypes.SignalTraces, FieldContext: telemetrytypes.FieldContextScope, FieldDataType: telemetrytypes.FieldDataTypeString}},
}
sql, args := build(key, keys, qbtypes.FilterOperatorEqual, "python")
assert.Contains(t, sql, "scope.attributes.`telemetry.sdk.language`::String = ?")
assert.Contains(t, sql, "scope.attributes.`telemetry.sdk.language` IS NOT NULL")
assert.Contains(t, args, "python")
assert.NotContains(t, sql, "scope.`scope.")
})
t.Run("declared path wins over a same-named scope attribute", func(t *testing.T) {
keys := map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.name": {&scopeName},
"name": {{Name: "name", Signal: telemetrytypes.SignalTraces, FieldContext: telemetrytypes.FieldContextScope, FieldDataType: telemetrytypes.FieldDataTypeString}},
}
key := telemetrytypes.TelemetryFieldKey{Name: "name", FieldContext: telemetrytypes.FieldContextScope}
sql, _ := build(key, keys, qbtypes.FilterOperatorEqual, "x")
assert.Contains(t, sql, "scope.name::String = ?")
assert.NotContains(t, sql, "scope.attributes.`name`")
explicit := telemetrytypes.TelemetryFieldKey{Name: "attributes.name", FieldContext: telemetrytypes.FieldContextScope}
sql, _ = build(explicit, keys, qbtypes.FilterOperatorEqual, "x")
assert.Contains(t, sql, "scope.attributes.`name`::String = ?")
})
t.Run("declared scope.version equality", func(t *testing.T) {
scopeVersion := IntrinsicFields["scope.version"]
key := telemetrytypes.TelemetryFieldKey{Name: "version", FieldContext: telemetrytypes.FieldContextScope}
keys := map[string][]*telemetrytypes.TelemetryFieldKey{"scope.version": {&scopeVersion}}
sql, args := build(key, keys, qbtypes.FilterOperatorEqual, "1.2.3")
assert.Contains(t, sql, "scope.version::String = ?")
assert.Contains(t, sql, "scope.version::String <> ''")
assert.Contains(t, args, "1.2.3")
})
t.Run("negative operator on declared path does not add existence guard", func(t *testing.T) {
key := telemetrytypes.TelemetryFieldKey{Name: "name", FieldContext: telemetrytypes.FieldContextScope}
sql, _ := build(key, declared, qbtypes.FilterOperatorNotEqual, "otelcol")
assert.Contains(t, sql, "scope.name::String <> ?")
assert.NotContains(t, sql, "= ''")
})
t.Run("IN on a scope attribute", func(t *testing.T) {
key := telemetrytypes.TelemetryFieldKey{Name: "telemetry.sdk.language", FieldContext: telemetrytypes.FieldContextScope}
keys := map[string][]*telemetrytypes.TelemetryFieldKey{
"telemetry.sdk.language": {{Name: "telemetry.sdk.language", Signal: telemetrytypes.SignalTraces, FieldContext: telemetrytypes.FieldContextScope, FieldDataType: telemetrytypes.FieldDataTypeString}},
}
sql, _ := build(key, keys, qbtypes.FilterOperatorIn, []any{"python", "go"})
assert.Contains(t, sql, "scope.attributes.`telemetry.sdk.language`::String = ?")
assert.Contains(t, sql, "scope.attributes.`telemetry.sdk.language` IS NOT NULL")
})
t.Run("numeric operand on a scope attribute coerces the string path to float", func(t *testing.T) {
key := telemetrytypes.TelemetryFieldKey{Name: "sampler.ratio", FieldContext: telemetrytypes.FieldContextScope}
keys := map[string][]*telemetrytypes.TelemetryFieldKey{
"sampler.ratio": {{Name: "sampler.ratio", Signal: telemetrytypes.SignalTraces, FieldContext: telemetrytypes.FieldContextScope, FieldDataType: telemetrytypes.FieldDataTypeString}},
}
sql, _ := build(key, keys, qbtypes.FilterOperatorGreaterThan, float64(0.5))
assert.Contains(t, sql, "toFloat64OrNull(scope.attributes.`sampler.ratio`::String) > ?")
})
}

View File

@@ -121,6 +121,20 @@ var (
FieldContext: telemetrytypes.FieldContextSpan,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"scope.name": {
Name: "scope.name",
Description: "Instrumentation scope name",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
"scope.version": {
Name: "scope.version",
Description: "Instrumentation scope version",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
}
IntrinsicFieldsDeprecated = map[string]telemetrytypes.TelemetryFieldKey{
"traceID": {

View File

@@ -53,6 +53,7 @@ var (
ValueType: schema.ColumnTypeString,
}},
"resource": {Name: "resource", Type: schema.JSONColumnType{}},
"scope": {Name: "scope", Type: schema.JSONColumnType{}},
"events": {Name: "events", Type: schema.ArrayColumnType{
ElementType: schema.ColumnTypeString,
@@ -181,7 +182,7 @@ func (m *fieldMapper) getColumn(
case telemetrytypes.FieldContextResource:
return []*schema.Column{indexV3Columns["resource"], indexV3Columns["resources_string"]}, nil
case telemetrytypes.FieldContextScope:
return []*schema.Column{}, qbtypes.ErrColumnNotFound
return []*schema.Column{indexV3Columns["scope"]}, nil
case telemetrytypes.FieldContextAttribute:
switch key.FieldDataType {
case telemetrytypes.FieldDataTypeString:
@@ -292,14 +293,25 @@ func (m *fieldMapper) resolveColumnExprs(
switch column.Type.GetType() {
case schema.ColumnTypeEnumJSON:
// json is only supported for resource context as of now
if key.FieldContext != telemetrytypes.FieldContextResource {
return nil, nil, nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource context fields are supported for json columns, got %s", key.FieldContext.String)
// The ::String cast is required because ClickHouse rejects Variant/Dynamic
// types in GROUP BY; revisit once the clickHouse dependency is updated.
switch key.FieldContext {
case telemetrytypes.FieldContextResource:
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
case telemetrytypes.FieldContextScope:
if declared, ok := resolvedScopeDeclaredPath(key.Name); ok {
// declared typed String paths are non-Nullable: absent reads '' not NULL.
exprs = append(exprs, fmt.Sprintf("%s::String", declared))
existExprs = append(existExprs, fmt.Sprintf("%s::String <> ''", declared))
} else {
attribute := scopeAttributeName(key.Name)
exprs = append(exprs, fmt.Sprintf("%s.attributes.`%s`::String", columnName, attribute))
existExprs = append(existExprs, fmt.Sprintf("%s.attributes.`%s` IS NOT NULL", columnName, attribute))
}
default:
return nil, nil, nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource and scope context fields are supported for json columns, got %s", key.FieldContext.String)
}
// have to add ::string as clickHouse throws an error :- data types Variant/Dynamic are not allowed in GROUP BY
// once clickHouse dependency is updated, we need to check if we can remove it.
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
existExprs = append(existExprs, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
case schema.ColumnTypeEnumString,
schema.ColumnTypeEnumUInt64,
schema.ColumnTypeEnumUInt32,
@@ -455,10 +467,13 @@ func (m *fieldMapper) ColumnExpressionFor(
return "", err
}
coerced := value
// a time column keeps its native type; coercing it would yield seconds
// a time column keeps its native type; coercing it would yield seconds, and a
// scope expression is already ::String so a String cast would be redundant
alreadyString := field.FieldContext == telemetrytypes.FieldContextScope &&
requiredDataType == telemetrytypes.FieldDataTypeString
if temporal, err := m.logicalIsTemporal(ctx, startNs, endNs, logical); err != nil {
return "", err
} else if !temporal {
} else if !temporal && !alreadyString {
coerced, _ = querybuilder.DataTypeCollisionHandledFieldName(logical.Single(), dummyValue, value, qbtypes.FilterOperatorUnknown)
}
stmts = append(stmts, guard, coerced)
@@ -484,7 +499,9 @@ func (m *fieldMapper) ColumnExpressionFor(
}
// Multiple candidates (collision / synth): multiIf picks the first that exists,
// stringified so branches share a type.
// stringified so branches share a type. Scope value expressions are already
// ::String, so they skip the redundant toString wrap.
scopeContext := field.FieldContext == telemetrytypes.FieldContextScope
args := make([]string, 0, len(candidates))
for _, logical := range candidates {
value, err := querybuilder.LogicalValueExpr(ctx, orgID, startNs, endNs, m, logical)
@@ -495,7 +512,11 @@ func (m *fieldMapper) ColumnExpressionFor(
if err != nil {
return "", err
}
args = append(args, fmt.Sprintf("%s, toString(%s)", guard, value))
if scopeContext {
args = append(args, fmt.Sprintf("%s, %s", guard, value))
} else {
args = append(args, fmt.Sprintf("%s, toString(%s)", guard, value))
}
}
return fmt.Sprintf("multiIf(%s, NULL)", strings.Join(args, ", ")), nil
}
@@ -577,6 +598,13 @@ func (m *fieldMapper) CandidateKeys(ctx context.Context, _ valuer.UUID, field *t
}
}
// Scope keys resolve only against the scope JSON column, so they never fall through to the
// name-only metadata match below: a same-named span or attribute entry is a different
// field (`scope.duration_nano` is not the duration_nano column).
if field.FieldContext == telemetrytypes.FieldContextScope {
return scopeCandidateKeys(field, keys)
}
// Metadata match by name, then the literal `{context}.{name}` spelling (a context can be
// a legitimate prefix in user data, e.g. `metric.max_count`). For a forgiving context
// this is the correction step (span.http.method -> attribute http.method).
@@ -600,10 +628,99 @@ func (m *fieldMapper) CandidateKeys(ctx context.Context, _ valuer.UUID, field *t
literal := telemetrytypes.NewTelemetryFieldKey(field.FieldContext.StringValue()+"."+field.Name, field.FieldContext, field.FieldDataType)
return append(querybuilder.SynthesizeKeys(field, value), querybuilder.SynthesizeKeys(literal, value)...)
}
// contexts that don't exist on spans (log, body, scope, …) have nothing to synthesize
// contexts that don't exist on spans (log, body, …) have nothing to synthesize
return nil
}
// scopeCandidateKeys resolves a scope-context key against the scope JSON column. A name that
// addresses one home explicitly — a declared path, or the `attributes.` prefix — resolves to
// it alone, even without metadata. A short declared name is ambiguous: the declared path
// comes first, joined by a same-named scope attribute that metadata knows about, the way a
// bare name puts its column ahead of same-named attributes.
func scopeCandidateKeys(field *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
scopeStringKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return telemetrytypes.NewTelemetryFieldKey(name, telemetrytypes.FieldContextScope, telemetrytypes.FieldDataTypeString)
}
if declared, ok := resolvedScopeDeclaredPath(field.Name); ok {
return []*telemetrytypes.TelemetryFieldKey{scopeStringKey(declared)}
}
for _, name := range []string{field.Name, telemetrytypes.FieldContextScope.StringValue() + "." + field.Name} {
scoped := []*telemetrytypes.TelemetryFieldKey{}
for _, match := range keys[name] {
if match.FieldContext == telemetrytypes.FieldContextScope {
scoped = append(scoped, match)
}
}
if len(scoped) > 0 {
return scoped
}
}
return []*telemetrytypes.TelemetryFieldKey{synthScopeAttributeKey(field)}
}
// synthScopeAttributeKey guesses a scope attribute (scope.attributes.<name>) for a name
// absent from metadata — the scope analog of querybuilder.SynthesizeKeys.
func synthScopeAttributeKey(field *telemetrytypes.TelemetryFieldKey) *telemetrytypes.TelemetryFieldKey {
return telemetrytypes.NewTelemetryFieldKey(field.Name, telemetrytypes.FieldContextScope, telemetrytypes.FieldDataTypeString)
}
// scopeAttributeNamePrefix addresses the attributes home explicitly, so a name that also
// exists as a declared path stays reachable (`scope.attributes.name`).
const scopeAttributeNamePrefix = "attributes."
// scopeAttributeName is the key inside scope.attributes that name refers to.
func scopeAttributeName(name string) string {
return strings.TrimPrefix(name, scopeAttributeNamePrefix)
}
// resolvedScopeDeclaredPath returns the declared path a scope name refers to, resolving the
// short spelling users type (`scope.name` normalizes to `name`) to it. The declared path wins
// over a same-named scope attribute, which stays reachable as `scope.attributes.name` — the
// precedence a real column has over a same-named attribute elsewhere in the builder.
func resolvedScopeDeclaredPath(name string) (string, bool) {
if strings.HasPrefix(name, scopeAttributeNamePrefix) {
return "", false
}
if isDeclaredScopePath(name) {
return name, true
}
qualified := telemetrytypes.FieldContextScope.StringValue() + "." + name
return qualified, isDeclaredScopePath(qualified)
}
// isDeclaredScopePath reports whether name is a declared typed sub-path of the scope JSON
// column (scope.name / scope.version), as opposed to an entry in scope.attributes.
func isDeclaredScopePath(name string) bool {
f, ok := IntrinsicFields[name]
return ok && f.FieldContext == telemetrytypes.FieldContextScope
}
// scopeJSONExistsExpression renders the presence predicate for a scope JSON key, whose
// two homes differ: declared typed paths are non-Nullable (absent reads ”), while
// scope.attributes.* are Dynamic/Nullable. Returns ok=false for non-scope keys so the
// caller falls back to the generic exists expression.
func scopeJSONExistsExpression(key *telemetrytypes.TelemetryFieldKey, fieldExpression string, exists bool) (string, bool) {
if key.FieldContext != telemetrytypes.FieldContextScope {
return "", false
}
if _, ok := resolvedScopeDeclaredPath(key.Name); ok {
if exists {
return fieldExpression + " <> ''", true
}
return fieldExpression + " = ''", true
}
// The value expression casts the JSON path to String, folding a missing key's NULL to
// '', so presence must test the raw path — drop the ::String cast.
path := strings.TrimSuffix(fieldExpression, "::String")
if exists {
return path + " IS NOT NULL", true
}
return path + " IS NULL", true
}
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
func (m *fieldMapper) ExistsFor(
ctx context.Context,
@@ -620,5 +737,8 @@ func (m *fieldMapper) ExistsFor(
if err != nil {
return "", err
}
if expr, ok := scopeJSONExistsExpression(key, fieldExpression, exists); ok {
return expr, nil
}
return querybuilder.ExistsExpression(columns, key, tsStart, tsEnd, fieldExpression, exists)
}

View File

@@ -304,3 +304,171 @@ func TestColumnExpressionForTimestampAttributeCollision(t *testing.T) {
assert.Contains(t, result, "attributes_number['timestamp']")
})
}
// scopeKey builds a TelemetryFieldKey the way the API boundary would after Normalize.
func scopeKey(name string) telemetrytypes.TelemetryFieldKey {
return telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
}
}
// declaredScopeKeys injects the scope.name/scope.version intrinsics into the metadata map
// the way metadata.go does at query time; resolution of the declared paths depends on it.
func declaredScopeKeys() map[string][]*telemetrytypes.TelemetryFieldKey {
scopeName := IntrinsicFields["scope.name"]
scopeVersion := IntrinsicFields["scope.version"]
return map[string][]*telemetrytypes.TelemetryFieldKey{
"scope.name": {&scopeName},
"scope.version": {&scopeVersion},
}
}
func scopeAttribute(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
}
// TestColumnExpressionForScope covers the scope resolution matrix from PR #10920: declared
// paths, scope attributes, and the precedence between them when a scope attribute shares its
// name with a declared path.
func TestColumnExpressionForScope(t *testing.T) {
ctx := context.Background()
tsStart := uint64(time.Date(2024, 6, 1, 0, 0, 0, 0, time.UTC).UnixNano())
tsEnd := uint64(time.Date(2024, 6, 5, 0, 0, 0, 0, time.UTC).UnixNano())
fm := NewFieldMapper(flaggertest.New(t))
run := func(field telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) string {
t.Helper()
result, err := fm.ColumnExpressionFor(ctx, valuer.UUID{}, tsStart, tsEnd, &field, telemetrytypes.FieldDataTypeUnspecified, keys)
require.NoError(t, err)
return result
}
t.Run("short name binds to declared scope.name when no attribute exists", func(t *testing.T) {
assert.Equal(t,
"multiIf(scope.name::String <> '', scope.name::String, NULL)",
run(scopeKey("name"), declaredScopeKeys()))
})
t.Run("fully-qualified scope.name isolates the declared path", func(t *testing.T) {
assert.Equal(t,
"multiIf(scope.name::String <> '', scope.name::String, NULL)",
run(scopeKey("scope.name"), declaredScopeKeys()))
})
t.Run("short version binds to declared scope.version", func(t *testing.T) {
assert.Equal(t,
"multiIf(scope.version::String <> '', scope.version::String, NULL)",
run(scopeKey("version"), declaredScopeKeys()))
})
t.Run("plain scope attribute", func(t *testing.T) {
keys := declaredScopeKeys()
keys["testing.env"] = []*telemetrytypes.TelemetryFieldKey{scopeAttribute("testing.env")}
assert.Equal(t,
"multiIf(scope.attributes.`testing.env` IS NOT NULL, scope.attributes.`testing.env`::String, NULL)",
run(scopeKey("testing.env"), keys))
})
t.Run("scope attribute synthesized when absent from metadata", func(t *testing.T) {
assert.Equal(t,
"multiIf(scope.attributes.`testing.env` IS NOT NULL, scope.attributes.`testing.env`::String, NULL)",
run(scopeKey("testing.env"), declaredScopeKeys()))
})
t.Run("declared path wins over a same-named scope attribute", func(t *testing.T) {
keys := declaredScopeKeys()
keys["name"] = []*telemetrytypes.TelemetryFieldKey{scopeAttribute("name")}
assert.Equal(t,
"multiIf(scope.name::String <> '', scope.name::String, NULL)",
run(scopeKey("name"), keys))
})
t.Run("attributes prefix reaches the shadowed scope attribute", func(t *testing.T) {
keys := declaredScopeKeys()
keys["name"] = []*telemetrytypes.TelemetryFieldKey{scopeAttribute("name")}
assert.Equal(t,
"multiIf(scope.attributes.`name` IS NOT NULL, scope.attributes.`name`::String, NULL)",
run(scopeKey("attributes.name"), keys))
})
t.Run("fully-qualified scope.version isolates declared even with conflicting attribute", func(t *testing.T) {
keys := declaredScopeKeys()
keys["version"] = []*telemetrytypes.TelemetryFieldKey{scopeAttribute("version")}
assert.Equal(t,
"multiIf(scope.version::String <> '', scope.version::String, NULL)",
run(scopeKey("scope.version"), keys))
})
t.Run("group by keeps the scope expression unwrapped by toString", func(t *testing.T) {
keys := declaredScopeKeys()
keys["name"] = []*telemetrytypes.TelemetryFieldKey{scopeAttribute("name")}
result, err := fm.ColumnExpressionFor(ctx, valuer.UUID{}, tsStart, tsEnd, &[]telemetrytypes.TelemetryFieldKey{scopeKey("attributes.name")}[0], telemetrytypes.FieldDataTypeString, keys)
require.NoError(t, err)
assert.Equal(t,
"multiIf(scope.attributes.`name` IS NOT NULL, scope.attributes.`name`::String, NULL)",
result)
})
}
// TestFieldForScope covers the per-key SQL for a resolved scope key.
func TestFieldForScope(t *testing.T) {
ctx := context.Background()
tsStart := uint64(time.Date(2024, 6, 1, 0, 0, 0, 0, time.UTC).UnixNano())
tsEnd := uint64(time.Date(2024, 6, 5, 0, 0, 0, 0, time.UTC).UnixNano())
fm := NewFieldMapper(flaggertest.New(t))
cases := map[string]string{
"scope.name": "scope.name::String",
"scope.version": "scope.version::String",
"custom.attr": "scope.attributes.`custom.attr`::String",
}
for name, want := range cases {
t.Run(name, func(t *testing.T) {
key := scopeKey(name)
got, err := fm.FieldFor(ctx, valuer.UUID{}, tsStart, tsEnd, &key)
require.NoError(t, err)
assert.Equal(t, want, got)
// A scope path must never double-prefix the JSON column.
assert.NotContains(t, got, "scope.`scope.")
})
}
}
// TestExistsForScope covers the presence predicates: declared paths test <> ” (non-Nullable),
// scope attributes test the raw JSON path IS NOT NULL.
func TestExistsForScope(t *testing.T) {
ctx := context.Background()
tsStart := uint64(time.Date(2024, 6, 1, 0, 0, 0, 0, time.UTC).UnixNano())
tsEnd := uint64(time.Date(2024, 6, 5, 0, 0, 0, 0, time.UTC).UnixNano())
fm := NewFieldMapper(flaggertest.New(t))
cases := []struct {
name string
key string
exists bool
want string
}{
{"declared exists", "scope.name", true, "scope.name::String <> ''"},
{"declared not exists", "scope.name", false, "scope.name::String = ''"},
{"short declared exists", "name", true, "scope.name::String <> ''"},
{"attribute exists", "exception.type", true, "scope.attributes.`exception.type` IS NOT NULL"},
{"attribute not exists", "exception.type", false, "scope.attributes.`exception.type` IS NULL"},
{"shadowed attribute exists", "attributes.name", true, "scope.attributes.`name` IS NOT NULL"},
{"shadowed attribute not exists", "attributes.name", false, "scope.attributes.`name` IS NULL"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
key := scopeKey(tc.key)
got, err := fm.ExistsFor(ctx, valuer.UUID{}, tsStart, tsEnd, &key, tc.exists)
require.NoError(t, err)
assert.Equal(t, tc.want, got)
})
}
}

View File

@@ -128,6 +128,21 @@ func BuildCompleteFieldKeyMap(releaseTime time.Time) map[string][]*telemetrytype
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
// declared scope paths, mirroring the intrinsics metadata.go injects at query time
"scope.name": {
{
Name: "scope.name",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
"scope.version": {
{
Name: "scope.version",
FieldContext: telemetrytypes.FieldContextScope,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
},
}
for _, keys := range keysMap {
for _, key := range keys {

View File

@@ -18,7 +18,7 @@ import (
// - Use `scope.` prefix to explicitly indicate and enforce scope context. Example
// - `scope.name`
// - `scope.version`
// - `scope.my.custom.attribute` and `scope.attribute.my.custom.attribute` resolve to same attribute
// - `scope.my.custom.attribute` resolves to the `my.custom.attribute` scope attribute
//
// - Use `attribute.` to explicitly indicate and enforce attribute context. Example
// - `attribute.http.method`
@@ -190,7 +190,7 @@ func (FieldContext) Enum() []any {
FieldContextSpan,
FieldContextTrace,
FieldContextResource,
// FieldContextScope,
FieldContextScope,
FieldContextAttribute,
// FieldContextEvent,
FieldContextBody,

View File

@@ -35,6 +35,24 @@ func TestGetFieldKeyFromKeyText(t *testing.T) {
FieldDataType: FieldDataTypeUnspecified,
},
},
{
// only the context prefix is stripped, so `attributes.` survives to address the
// scope attribute that a declared path of the same name would otherwise win
keyText: "scope.attributes.name",
expected: TelemetryFieldKey{
Name: "attributes.name",
FieldContext: FieldContextScope,
FieldDataType: FieldDataTypeUnspecified,
},
},
{
keyText: "scope.custom.attr:string",
expected: TelemetryFieldKey{
Name: "custom.attr",
FieldContext: FieldContextScope,
FieldDataType: FieldDataTypeString,
},
},
{
keyText: "attribute.http.method",
expected: TelemetryFieldKey{