Compare commits

..

1 Commits

Author SHA1 Message Date
Aditya Singh
095821264e fix(explorer): guard saved-view URL params against non-JSON values (#12706)
Some checks are pending
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
cacheci / tests (push) Waiting to run
Release Drafter / update_release_draft (push) Waiting to run
#### Description

- Opening an explorer with a `viewName`/`viewKey` in the URL that isn't
valid JSON crashed the whole page.
- The hook ran `JSON.parse` on the raw param during render, so a bare
saved-view name threw error and broke page.
- Fix: wrap `JSON.parse` in try/catch and fall back to the raw string.
- Renamed the hook to `useGetSavedViewParams` and moved it under
`hooks/saveViews`; it only reads `viewName`/`viewKey` so the old
query-builder name/location was misleading. Now returns `{ viewName,
viewKey }` directly.
- Behavior preserved for all consumers; added tests for the non-JSON
case.


Closes https://github.com/SigNoz/engineering-pod/issues/5638

Screenshots/Recording

Before:

Test url: Just remove quotes from viewKey or viewName:
[url](https://app.us.staging.signoz.cloud/logs/logs-explorer?relativeTime=1month&compositeQuery=%257B%2522queryType%2522%253A%2522builder%2522%252C%2522builder%2522%253A%257B%2522queryData%2522%253A%255B%257B%2522dataSource%2522%253A%2522logs%2522%252C%2522queryName%2522%253A%2522A%2522%252C%2522aggregateOperator%2522%253A%2522count%2522%252C%2522aggregateAttribute%2522%253A%257B%2522id%2522%253A%2522----%2522%252C%2522dataType%2522%253A%2522%2522%252C%2522key%2522%253A%2522%2522%252C%2522type%2522%253A%2522%2522%257D%252C%2522timeAggregation%2522%253A%2522rate%2522%252C%2522spaceAggregation%2522%253A%2522sum%2522%252C%2522filter%2522%253A%257B%2522expression%2522%253A%2522%2522%257D%252C%2522aggregations%2522%253A%255B%257B%2522expression%2522%253A%2522count%28%29%2522%257D%255D%252C%2522functions%2522%253Anull%252C%2522filters%2522%253A%257B%2522items%2522%253A%255B%257B%2522id%2522%253A%2522228b8a2f-d6ba-4704-9104-936e91a2c119%2522%252C%2522key%2522%253A%257B%2522id%2522%253A%2522code.function--string--tag%2522%252C%2522dataType%2522%253A%2522string%2522%252C%2522key%2522%253A%2522code.function%2522%252C%2522type%2522%253A%2522tag%2522%257D%252C%2522op%2522%253A%2522%253D%2522%252C%2522value%2522%253A%2522render_test%2522%257D%255D%252C%2522op%2522%253A%2522AND%2522%257D%252C%2522expression%2522%253A%2522A%2522%252C%2522disabled%2522%253Afalse%252C%2522stepInterval%2522%253A0%252C%2522having%2522%253A%257B%2522expression%2522%253A%2522%2522%257D%252C%2522limit%2522%253Anull%252C%2522orderBy%2522%253A%255B%255D%252C%2522groupBy%2522%253A%255B%255D%252C%2522legend%2522%253A%2522%2522%252C%2522reduceTo%2522%253A%2522avg%2522%252C%2522source%2522%253A%2522%2522%252C%2522name%2522%253A%2522A%2522%252C%2522signal%2522%253A%2522logs%2522%252C%2522order%2522%253Anull%252C%2522selectFields%2522%253Anull%252C%2522secondaryAggregations%2522%253Anull%257D%255D%252C%2522queryFormulas%2522%253A%255B%255D%252C%2522queryTraceOperator%2522%253A%255B%255D%257D%252C%2522promql%2522%253A%255B%257B%2522name%2522%253A%2522A%2522%252C%2522query%2522%253A%2522%2522%252C%2522legend%2522%253A%2522%2522%252C%2522disabled%2522%253Afalse%257D%255D%252C%2522clickhouse_sql%2522%253A%255B%257B%2522name%2522%253A%2522A%2522%252C%2522legend%2522%253A%2522%2522%252C%2522disabled%2522%253Afalse%252C%2522query%2522%253A%2522%2522%257D%255D%252C%2522id%2522%253A%2522bfb926e4-7b98-4cf4-bd4d-adcbd18a1da2%2522%252C%2522unit%2522%253A%2522%2522%257D&options=%7B%22selectColumns%22%3A%5B%7B%22name%22%3A%22timestamp%22%2C%22signal%22%3A%22logs%22%2C%22fieldContext%22%3A%22log%22%2C%22fieldDataType%22%3A%22%22%7D%2C%7B%22name%22%3A%22lkadsjfl%22%2C%22signal%22%3A%22%22%2C%22fieldContext%22%3A%22%22%2C%22fieldDataType%22%3A%22%22%7D%2C%7B%22name%22%3A%22body%22%2C%22signal%22%3A%22logs%22%2C%22fieldContext%22%3A%22log%22%2C%22fieldDataType%22%3A%22%22%7D%2C%7B%22name%22%3A%22test%22%2C%22signal%22%3A%22%22%2C%22fieldContext%22%3A%22%22%2C%22fieldDataType%22%3A%22%22%7D%2C%7B%22name%22%3A%22severity_text%22%2C%22description%22%3A%22Log+level.+Learn+more+%5Bhere%5D%28https%3A%2F%2Fopentelemetry.io%2Fdocs%2Fspecs%2Fotel%2Flogs%2Fdata-model%2F%23field-severitytext%29%22%2C%22signal%22%3A%22logs%22%2C%22fieldContext%22%3A%22log%22%2C%22fieldDataType%22%3A%22string%22%7D%5D%2C%22format%22%3A%22list%22%2C%22maxLines%22%3A1%2C%22fontSize%22%3A%22small%22%7D&panelTypes=%22list%22&viewName=%22test+manul+key%22&viewKey=068e4a96-5225-4abe-8f9b-a5009f26d4ce)

Breaks page

<img width="3456" height="1720" alt="image"
src="https://github.com/user-attachments/assets/4d7546a3-b5cb-4be3-bc9f-e0782446f7f1"
/>



#### Additional Information

Sentry: https://signoz-io.sentry.io/issues/7520808361
2026-08-28 06:40:25 +00:00
16 changed files with 120 additions and 420 deletions

View File

@@ -7,9 +7,8 @@ import axios from 'axios';
import TextToolTip from 'components/TextToolTip';
import { SOMETHING_WENT_WRONG } from 'constants/api';
import { LOCALSTORAGE } from 'constants/localStorage';
import { QueryParams } from 'constants/query';
import { useOptionsMenu } from 'container/OptionsMenu';
import { useGetSearchQueryParam } from 'hooks/queryBuilder/useGetSearchQueryParam';
import { useGetSavedViewParams } from 'hooks/saveViews/useGetSavedViewParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { useDeleteView } from 'hooks/saveViews/useDeleteView';
import { useGetAllViews } from 'hooks/saveViews/useGetAllViews';
@@ -69,9 +68,7 @@ function ExplorerCard({
setIsOpen(newOpen);
};
const viewName = useGetSearchQueryParam(QueryParams.viewName) || '';
const viewKey = useGetSearchQueryParam(QueryParams.viewKey) || '';
const { viewName, viewKey } = useGetSavedViewParams();
const { options } = useOptionsMenu({
storageKey:

View File

@@ -1,5 +1,3 @@
import { QueryParams } from 'constants/query';
export const ExploreHeaderToolTip = {
url: 'https://signoz.io/docs/querying/overview/?utm_source=product&utm_medium=new-query-builder',
text: 'More details on how to use query builder',
@@ -9,5 +7,3 @@ export const SaveButtonText = {
SAVE_AS_NEW_VIEW: 'Save as new view',
SAVE_VIEW: 'Save view',
};
export type QuerySearchParamNames = QueryParams.viewName | QueryParams.viewKey;

View File

@@ -54,7 +54,7 @@ import {
} from 'container/OptionsMenu/constants';
import { OptionsQuery } from 'container/OptionsMenu/types';
import { ExportDashboard } from 'hooks/dashboard/useExportDashboards';
import { useGetSearchQueryParam } from 'hooks/queryBuilder/useGetSearchQueryParam';
import { useGetSavedViewParams } from 'hooks/saveViews/useGetSavedViewParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { useGetAllViews } from 'hooks/saveViews/useGetAllViews';
import { useSaveView } from 'hooks/saveViews/useSaveView';
@@ -287,8 +287,7 @@ function ExplorerOptions({
const compositeQuery = mapCompositeQueryFromQuery(currentQuery, panelType);
const viewName = useGetSearchQueryParam(QueryParams.viewName) || '';
const viewKey = useGetSearchQueryParam(QueryParams.viewKey) || '';
const { viewName, viewKey } = useGetSavedViewParams();
const extraData = viewsData?.data?.data?.find(
(view) => view.id === viewKey,

View File

@@ -15,9 +15,8 @@ import {
QUERY_BUILDER_FUNCTIONS,
} from 'constants/antlrQueryConstants';
import { FeatureKeys } from 'constants/features';
import { QueryParams } from 'constants/query';
import { useActiveLog } from 'hooks/logs/useActiveLog';
import { useGetSearchQueryParam } from 'hooks/queryBuilder/useGetSearchQueryParam';
import { useGetSavedViewParams } from 'hooks/saveViews/useGetSavedViewParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { ICurrentQueryData } from 'hooks/useHandleExplorerTabChange';
import { useNotifications } from 'hooks/useNotifications';
@@ -50,7 +49,7 @@ function BodyTitleRenderer({
const { featureFlags } = useAppContext();
const [, setCopy] = useCopyToClipboard();
const { notifications } = useNotifications();
const viewName = useGetSearchQueryParam(QueryParams.viewName) || '';
const { viewName } = useGetSavedViewParams();
const cleanedNodeKey = removeObjectFromString(nodeKey);
const isBodyJsonQueryEnabled =

View File

@@ -7,13 +7,12 @@ import GroupByIcon from 'assets/CustomIcons/GroupByIcon';
import cx from 'classnames';
import CopyClipboardHOC from 'components/Logs/CopyClipboardHOC';
import { DATE_TIME_FORMATS } from 'constants/dateTimeFormats';
import { QueryParams } from 'constants/query';
import { OPERATORS } from 'constants/queryBuilder';
import ROUTES from 'constants/routes';
import { ChangeViewFunctionType } from 'container/ExplorerOptions/types';
import { RESTRICTED_SELECTED_FIELDS } from 'container/LogsFilters/config';
import { MetricsType } from 'container/MetricsApplication/constant';
import { useGetSearchQueryParam } from 'hooks/queryBuilder/useGetSearchQueryParam';
import { useGetSavedViewParams } from 'hooks/saveViews/useGetSavedViewParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { ICurrentQueryData } from 'hooks/useHandleExplorerTabChange';
import {
@@ -141,7 +140,7 @@ export default function TableViewActions(
const { pathname } = useLocation();
const { stagedQuery, updateQueriesData } = useQueryBuilder();
const viewName = useGetSearchQueryParam(QueryParams.viewName) || '';
const { viewName } = useGetSavedViewParams();
const { dataType, logType: fieldType } = getFieldAttributes(record.field);
// there is no option for where clause in old logs explorer and live logs page or infra monitoring

View File

@@ -1,6 +1,6 @@
import { fireEvent, render, screen } from '@testing-library/react';
import { RESTRICTED_SELECTED_FIELDS } from 'container/LogsFilters/config';
import { useGetSearchQueryParam } from 'hooks/queryBuilder/useGetSearchQueryParam';
import { useGetSavedViewParams } from 'hooks/saveViews/useGetSavedViewParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { ExplorerViews } from 'pages/LogsExplorer/utils';
@@ -88,7 +88,7 @@ jest.mock('react-router-dom', () => ({
}));
jest.mock('hooks/queryBuilder/useQueryBuilder');
jest.mock('hooks/queryBuilder/useGetSearchQueryParam');
jest.mock('hooks/saveViews/useGetSavedViewParams');
describe('TableViewActions', () => {
const TEST_VALUE = 'test value';
@@ -140,8 +140,10 @@ describe('TableViewActions', () => {
}),
} as any);
// Default mock for useGetSearchQueryParam
jest.mocked(useGetSearchQueryParam).mockReturnValue(null);
// Default mock for useGetSavedViewParams
jest
.mocked(useGetSavedViewParams)
.mockReturnValue({ viewName: '', viewKey: '' });
});
it('should render without crashing', () => {
@@ -249,7 +251,9 @@ describe('TableViewActions', () => {
updateQueriesData: mockUpdateQueriesData,
} as any);
jest.mocked(useGetSearchQueryParam).mockReturnValue(null);
jest
.mocked(useGetSavedViewParams)
.mockReturnValue({ viewName: '', viewKey: '' });
render(
<TableViewActions

View File

@@ -3,10 +3,9 @@ import { useLocation } from 'react-router-dom';
import { CircleMinus, CirclePlus, Layers, RefreshCw } from '@signozhq/icons';
import { convertFiltersToExpression } from 'components/QueryBuilderV2/utils';
import { FeatureKeys } from 'constants/features';
import { QueryParams } from 'constants/query';
import ROUTES from 'constants/routes';
import { ChangeViewFunctionType } from 'container/ExplorerOptions/types';
import { useGetSearchQueryParam } from 'hooks/queryBuilder/useGetSearchQueryParam';
import { useGetSavedViewParams } from 'hooks/saveViews/useGetSavedViewParams';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { ICurrentQueryData } from 'hooks/useHandleExplorerTabChange';
import { ExplorerViews } from 'pages/LogsExplorer/utils';
@@ -58,7 +57,7 @@ export function useLogAttributeActions({
const { pathname } = useLocation();
const { stagedQuery, updateQueriesData } = useQueryBuilder();
const { featureFlags } = useAppContext();
const viewName = useGetSearchQueryParam(QueryParams.viewName) || '';
const { viewName } = useGetSavedViewParams();
const isBodyJsonQueryEnabled =
featureFlags?.find((flag) => flag.name === FeatureKeys.USE_JSON_BODY)

View File

@@ -1,15 +0,0 @@
import { useMemo } from 'react';
import { QuerySearchParamNames } from 'components/ExplorerCard/constants';
import useUrlQuery from 'hooks/useUrlQuery';
export const useGetSearchQueryParam = (
searchParams: QuerySearchParamNames,
): string | null => {
const urlQuery = useUrlQuery();
return useMemo(() => {
const searchQuery = urlQuery.get(searchParams);
return searchQuery ? JSON.parse(searchQuery) : null;
}, [urlQuery, searchParams]);
};

View File

@@ -0,0 +1,60 @@
import { renderHook } from '@testing-library/react';
import useUrlQuery from 'hooks/useUrlQuery';
import { useGetSavedViewParams } from '../useGetSavedViewParams';
jest.mock('hooks/useUrlQuery');
const mockedUseUrlQuery = useUrlQuery as jest.Mock;
const setSearch = (search: string): void => {
mockedUseUrlQuery.mockReturnValue(new URLSearchParams(search));
};
describe('useGetSavedViewParams', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('returns empty strings when no params are present', () => {
setSearch('');
const { result } = renderHook(() => useGetSavedViewParams());
expect(result.current).toStrictEqual({ viewName: '', viewKey: '' });
});
it('parses JSON-stringified values', () => {
setSearch(
`viewName=${encodeURIComponent(
JSON.stringify('Hindsight'),
)}&viewKey=${encodeURIComponent(JSON.stringify('abc-123'))}`,
);
const { result } = renderHook(() => useGetSavedViewParams());
expect(result.current).toStrictEqual({
viewName: 'Hindsight',
viewKey: 'abc-123',
});
});
it('falls back to the raw string when a value is not valid JSON', () => {
setSearch('viewName=Hindsight&viewKey=some-uuid-value');
const { result } = renderHook(() => useGetSavedViewParams());
expect(result.current).toStrictEqual({
viewName: 'Hindsight',
viewKey: 'some-uuid-value',
});
});
it('does not throw and keeps the raw string for non-string JSON', () => {
setSearch('viewName=123');
const { result } = renderHook(() => useGetSavedViewParams());
expect(result.current).toStrictEqual({ viewName: '123', viewKey: '' });
});
});

View File

@@ -0,0 +1,33 @@
import { useMemo } from 'react';
import { QueryParams } from 'constants/query';
import useUrlQuery from 'hooks/useUrlQuery';
interface SavedViewParams {
viewName: string;
viewKey: string;
}
const parseViewParam = (value: string | null): string => {
if (!value) {
return '';
}
try {
const parsed = JSON.parse(value);
return typeof parsed === 'string' ? parsed : value;
} catch {
return value;
}
};
export const useGetSavedViewParams = (): SavedViewParams => {
const urlQuery = useUrlQuery();
return useMemo(
() => ({
viewName: parseViewParam(urlQuery.get(QueryParams.viewName)),
viewKey: parseViewParam(urlQuery.get(QueryParams.viewKey)),
}),
[urlQuery],
);
};

View File

@@ -6,7 +6,7 @@ import { SIGNOZ_VALUE } from 'container/QueryBuilder/filters/OrderByFilter/const
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
import { useGetSearchQueryParam } from './queryBuilder/useGetSearchQueryParam';
import { useGetSavedViewParams } from './saveViews/useGetSavedViewParams';
import { useQueryBuilder } from './queryBuilder/useQueryBuilder';
export interface ICurrentQueryData {
@@ -31,9 +31,7 @@ export const useHandleExplorerTabChange = (): {
updateQueriesData,
} = useQueryBuilder();
const viewName = useGetSearchQueryParam(QueryParams.viewName) || '';
const viewKey = useGetSearchQueryParam(QueryParams.viewKey) || '';
const { viewName, viewKey } = useGetSavedViewParams();
const getUpdateQuery = useCallback(
(newPanelType: PANEL_TYPES): Query => {

View File

@@ -43,7 +43,7 @@ func ExistsExpression(columns []*schema.Column, key *telemetrytypes.TelemetryFie
if len(evolutionsEntries) > 0 && evolutionsEntries[0] != nil {
columnName = evolutionsEntries[0].ColumnName
}
rawPath := fmt.Sprintf("%s.%s", columnName, ClickHouseIdentifier(key.Name))
rawPath := fmt.Sprintf("%s.`%s`", columnName, key.Name)
if exists {
return rawPath + " IS NOT NULL", nil
}

View File

@@ -43,7 +43,6 @@ const (
SpanAttributesStringColumn = "attributes_string"
SpanAttributesNumberColumn = "attributes_number"
SpanAttributesBoolColumn = "attributes_bool"
SpanAttributesColumn = "attributes"
SpanResourcesStringColumn = "resources_string"
)

View File

@@ -52,9 +52,8 @@ var (
KeyType: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString},
ValueType: schema.ColumnTypeString,
}},
"resource": {Name: "resource", Type: schema.JSONColumnType{}},
"scope": {Name: "scope", Type: schema.JSONColumnType{}},
"attributes": {Name: "attributes", Type: schema.JSONColumnType{}},
"resource": {Name: "resource", Type: schema.JSONColumnType{}},
"scope": {Name: "scope", Type: schema.JSONColumnType{}},
"events": {Name: "events", Type: schema.ArrayColumnType{
ElementType: schema.ColumnTypeString,
@@ -185,32 +184,16 @@ func (m *fieldMapper) getColumn(
case telemetrytypes.FieldContextScope:
return []*schema.Column{indexV3Columns["scope"]}, nil
case telemetrytypes.FieldContextAttribute:
// Only typed keys resolve here: a data-type-unspecified attribute key
// falls through to ErrColumnNotFound so bare keys keep taking the legacy
// CandidateKeys/synthesis path rather than flipping to metadata-first.
var mapCol *schema.Column
switch key.FieldDataType {
case telemetrytypes.FieldDataTypeString:
mapCol = indexV3Columns["attributes_string"]
return []*schema.Column{indexV3Columns["attributes_string"]}, nil
case telemetrytypes.FieldDataTypeInt64,
telemetrytypes.FieldDataTypeFloat64,
telemetrytypes.FieldDataTypeNumber:
mapCol = indexV3Columns["attributes_number"]
return []*schema.Column{indexV3Columns["attributes_number"]}, nil
case telemetrytypes.FieldDataTypeBool:
mapCol = indexV3Columns["attributes_bool"]
default:
return nil, qbtypes.ErrColumnNotFound
return []*schema.Column{indexV3Columns["attributes_bool"]}, nil
}
// Dual-read from the JSON column only once the attributes evolution entry
// is registered for this key; without it, resolution is byte-for-byte the
// legacy Map path. The JSON column is returned first so it wins the multiIf
// when both homes hold the key; SelectEvolutionsForColumns narrows the pair
// by the query's time range. This makes the evolution entry the rollout
// control, exactly as it is for resource/scope.
if attributeJSONEvolutionRegistered(key) {
return []*schema.Column{indexV3Columns["attributes"], mapCol}, nil
}
return []*schema.Column{mapCol}, nil
case telemetrytypes.FieldContextSpan:
// Check if this is a span scope field
if strings.ToLower(key.Name) == SpanSearchScopeRoot || strings.ToLower(key.Name) == SpanSearchScopeEntryPoint {
@@ -296,7 +279,6 @@ func (m *fieldMapper) resolveColumnExprs(
return nil, nil, nil, err
}
key = narrowEvolutionsToColumns(key, columns)
newColumns, evolutionsEntries, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
if err != nil {
return nil, nil, nil, err
@@ -327,17 +309,6 @@ func (m *fieldMapper) resolveColumnExprs(
exprs = append(exprs, fmt.Sprintf("%s.attributes.%s::String", columnName, querybuilder.ClickHouseIdentifier(attributeName)))
existExprs = append(existExprs, fmt.Sprintf("%s.attributes.%s IS NOT NULL", columnName, querybuilder.ClickHouseIdentifier(attributeName)))
}
case telemetrytypes.FieldContextAttribute:
// Span attributes are flat shared-data paths keyed by the attribute name
// verbatim (no nested object), so the name addresses the path directly.
// String casts to ::String so an absent path folds to '' — matching the
// Map column's default and preserving negative-operator parity; typed
// numeric/bool cast to Nullable so a missing or wrong-typed path reads
// NULL rather than 0/false. Existence tests the raw path (the cast folds
// NULL) and is index-eligible via attributes_paths_tokenbf.
path := fmt.Sprintf("%s.%s", columnName, querybuilder.ClickHouseIdentifier(key.Name))
exprs = append(exprs, fmt.Sprintf("%s::%s", path, attributeJSONCast(key.FieldDataType)))
existExprs = append(existExprs, fmt.Sprintf("%s IS NOT NULL", path))
default:
return nil, nil, nil, errors.NewInternalf(errors.CodeInternal, "only resource and scope context fields are supported for json columns, got %s", key.FieldContext.String)
}
@@ -382,66 +353,6 @@ func (m *fieldMapper) resolveColumnExprs(
return exprs, existExprs, columns, nil
}
// attributeJSONEvolutionRegistered reports whether the key carries an evolution entry for the
// `attributes` JSON column. Until that entry is registered, attribute resolution stays on the
// legacy Map column and the JSON column is never touched.
func attributeJSONEvolutionRegistered(key *telemetrytypes.TelemetryFieldKey) bool {
for _, e := range key.Evolutions {
if e != nil && e.ColumnName == SpanAttributesColumn {
return true
}
}
return false
}
// narrowEvolutionsToColumns returns key.Evolutions filtered to entries whose column is in cols.
// A metadata attribute key carries the `__all__` evolutions for every attribute-context column
// (all three legacy maps plus the JSON column), but getColumn resolves a typed key to only its
// own map + the JSON column; without this filter SelectEvolutionsForColumns would reject the
// in-range sibling-map entries as columns not present in the slice. A no-op for every other
// context, where getColumn already returns exactly the columns the evolutions name.
func narrowEvolutionsToColumns(key *telemetrytypes.TelemetryFieldKey, cols []*schema.Column) *telemetrytypes.TelemetryFieldKey {
if len(key.Evolutions) == 0 {
return key
}
allowed := make(map[string]struct{}, len(cols))
for _, c := range cols {
allowed[c.Name] = struct{}{}
}
filtered := make([]*telemetrytypes.EvolutionEntry, 0, len(key.Evolutions))
for _, e := range key.Evolutions {
if e == nil {
continue
}
if _, ok := allowed[e.ColumnName]; ok {
filtered = append(filtered, e)
}
}
if len(filtered) == len(key.Evolutions) {
return key
}
narrowed := *key
narrowed.Evolutions = filtered
return &narrowed
}
// attributeJSONCast returns the ClickHouse cast target for a span attribute read from the
// JSON column. String (and data-type-unspecified) casts to non-nullable String so an absent
// path folds to ” the way the Map column's default does, keeping negative-operator parity;
// numeric and bool cast to Nullable so an absent or wrong-typed path reads NULL instead of a
// spurious 0/false. GROUP BY accepts these Nullable scalar casts (unlike a raw Dynamic).
func attributeJSONCast(dataType telemetrytypes.FieldDataType) string {
switch dataType {
case telemetrytypes.FieldDataTypeInt64,
telemetrytypes.FieldDataTypeFloat64,
telemetrytypes.FieldDataTypeNumber,
telemetrytypes.FieldDataTypeBool:
return fmt.Sprintf("Nullable(%s)", telemetrytypes.MappingFieldDataTypeToJSONDataType[dataType].StringValue())
default:
return "String"
}
}
// upgradeToFamilies swaps single-member candidates for their family when the
// metadata map proves membership. Candidate order and every non-family
// candidate stay exactly as the legacy flow produced them; sibling candidates
@@ -605,7 +516,6 @@ func (m *fieldMapper) columnIsTemporal(ctx context.Context, startNs, endNs uint6
if err != nil {
return false, err
}
key = narrowEvolutionsToColumns(key, columns)
newColumns, _, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
if err != nil {
return false, err
@@ -728,7 +638,6 @@ func (m *fieldMapper) ExistsFor(
if err != nil {
return "", err
}
key = narrowEvolutionsToColumns(key, columns)
fieldExpression, err := m.FieldFor(ctx, orgID, tsStart, tsEnd, key)
if err != nil {
return "", err

View File

@@ -1,254 +0,0 @@
package tracestelemetryschema
import (
"context"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var (
attrJSONRelease = time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC)
attrWindowBefore = [2]uint64{tsNano(2024, 1), tsNano(2024, 6)}
attrWindowAfter = [2]uint64{tsNano(2025, 6), tsNano(2025, 7)}
attrWindowStraddle = [2]uint64{tsNano(2024, 6), tsNano(2025, 6)}
)
func tsNano(y int, m time.Month) uint64 {
return uint64(time.Date(y, m, 1, 0, 0, 0, 0, time.UTC).UnixNano())
}
func attrKey(name string, dt telemetrytypes.FieldDataType, evo []*telemetrytypes.EvolutionEntry) telemetrytypes.TelemetryFieldKey {
return telemetrytypes.TelemetryFieldKey{
Name: name,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldDataType: dt,
Evolutions: evo,
}
}
// TestFieldForAttributeJSONEvolution asserts the value expression across the rollout window:
// before release the legacy Map lookup (byte-for-byte today), after release the type-aware JSON
// cast, straddling a dual-read multiIf with the JSON column first.
func TestFieldForAttributeJSONEvolution(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
evo := MockAttributeEvolutionData(attrJSONRelease)
testCases := []struct {
name string
dataType telemetrytypes.FieldDataType
window [2]uint64
expected string
}{
{"string before -> map", telemetrytypes.FieldDataTypeString, attrWindowBefore, "attributes_string['user.id']"},
{"string after -> json", telemetrytypes.FieldDataTypeString, attrWindowAfter, "attributes.`user.id`::String"},
{"string straddle -> dual", telemetrytypes.FieldDataTypeString, attrWindowStraddle, "multiIf(attributes.`user.id` IS NOT NULL, attributes.`user.id`::String, mapContains(attributes_string, 'user.id'), attributes_string['user.id'], NULL)"},
{"number before -> map", telemetrytypes.FieldDataTypeNumber, attrWindowBefore, "attributes_number['user.id']"},
{"number after -> json", telemetrytypes.FieldDataTypeNumber, attrWindowAfter, "attributes.`user.id`::Nullable(Float64)"},
{"number straddle -> dual", telemetrytypes.FieldDataTypeNumber, attrWindowStraddle, "multiIf(attributes.`user.id` IS NOT NULL, attributes.`user.id`::Nullable(Float64), mapContains(attributes_number, 'user.id'), attributes_number['user.id'], NULL)"},
{"int64 after -> json", telemetrytypes.FieldDataTypeInt64, attrWindowAfter, "attributes.`user.id`::Nullable(Int64)"},
{"bool before -> map", telemetrytypes.FieldDataTypeBool, attrWindowBefore, "attributes_bool['user.id']"},
{"bool after -> json", telemetrytypes.FieldDataTypeBool, attrWindowAfter, "attributes.`user.id`::Nullable(Bool)"},
{"bool straddle -> dual", telemetrytypes.FieldDataTypeBool, attrWindowStraddle, "multiIf(attributes.`user.id` IS NOT NULL, attributes.`user.id`::Nullable(Bool), mapContains(attributes_bool, 'user.id'), attributes_bool['user.id'], NULL)"},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
key := attrKey("user.id", tc.dataType, evo)
got, err := fm.FieldFor(ctx, valuer.UUID{}, tc.window[0], tc.window[1], &key)
require.NoError(t, err)
assert.Equal(t, tc.expected, got)
})
}
}
// TestFieldForAttributeNoEvolutionParity proves the JSON column is untouched until the evolution
// entry is registered: a key with no evolutions resolves to the Map column for every window.
func TestFieldForAttributeNoEvolutionParity(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
for _, dt := range []struct {
dataType telemetrytypes.FieldDataType
expected string
}{
{telemetrytypes.FieldDataTypeString, "attributes_string['user.id']"},
{telemetrytypes.FieldDataTypeNumber, "attributes_number['user.id']"},
{telemetrytypes.FieldDataTypeBool, "attributes_bool['user.id']"},
} {
key := attrKey("user.id", dt.dataType, nil)
got, err := fm.FieldFor(ctx, valuer.UUID{}, attrWindowAfter[0], attrWindowAfter[1], &key)
require.NoError(t, err)
assert.Equal(t, dt.expected, got, "no evolution entry must keep the Map path")
}
}
// TestConditionForAttributeJSON asserts the emitted WHERE fragment per operator against the JSON
// column (window fully after release). Positive operators carry the raw-path existence guard;
// numeric comparisons keep numeric semantics; existence never tests the ::String cast.
func TestConditionForAttributeJSON(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
cb := NewConditionBuilder(fm, flaggertest.New(t))
evo := MockAttributeEvolutionData(attrJSONRelease)
testCases := []struct {
name string
key telemetrytypes.TelemetryFieldKey
operator qbtypes.FilterOperator
value any
expected string
}{
{
name: "equal string",
key: attrKey("user.id", telemetrytypes.FieldDataTypeString, evo),
operator: qbtypes.FilterOperatorEqual, value: "admin",
expected: "(attributes.`user.id`::String = ? AND attributes.`user.id` IS NOT NULL)",
},
{
name: "not equal string has no exists guard",
key: attrKey("user.id", telemetrytypes.FieldDataTypeString, evo),
operator: qbtypes.FilterOperatorNotEqual, value: "admin",
expected: "attributes.`user.id`::String <> ?",
},
{
name: "greater than number",
key: attrKey("http.status_code", telemetrytypes.FieldDataTypeInt64, evo),
operator: qbtypes.FilterOperatorGreaterThan, value: float64(200),
expected: "toFloat64(attributes.`http.status_code`::Nullable(Int64)) > ?",
},
{
name: "ilike string",
key: attrKey("user.id", telemetrytypes.FieldDataTypeString, evo),
operator: qbtypes.FilterOperatorILike, value: "%adm%",
expected: "LOWER(attributes.`user.id`::String) LIKE LOWER(?)",
},
{
name: "exists uses raw path",
key: attrKey("user.id", telemetrytypes.FieldDataTypeString, evo),
operator: qbtypes.FilterOperatorExists, value: nil,
expected: "attributes.`user.id` IS NOT NULL",
},
{
name: "not exists uses raw path",
key: attrKey("user.id", telemetrytypes.FieldDataTypeString, evo),
operator: qbtypes.FilterOperatorNotExists, value: nil,
expected: "attributes.`user.id` IS NULL",
},
{
name: "in string",
key: attrKey("user.id", telemetrytypes.FieldDataTypeString, evo),
operator: qbtypes.FilterOperatorIn, value: []any{"a", "b"},
expected: "((attributes.`user.id`::String = ? OR attributes.`user.id`::String = ?) AND attributes.`user.id` IS NOT NULL)",
},
{
name: "not in string has no exists guard",
key: attrKey("user.id", telemetrytypes.FieldDataTypeString, evo),
operator: qbtypes.FilterOperatorNotIn, value: []any{"a", "b"},
expected: "(attributes.`user.id`::String <> ? AND attributes.`user.id`::String <> ?)",
},
{
name: "between number",
key: attrKey("latency", telemetrytypes.FieldDataTypeNumber, evo),
operator: qbtypes.FilterOperatorBetween, value: []any{float64(1), float64(9)},
expected: "toFloat64(attributes.`latency`::Nullable(Float64)) BETWEEN ? AND ?",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, attrWindowAfter[0], attrWindowAfter[1], &tc.key,
map[string][]*telemetrytypes.TelemetryFieldKey{tc.key.Name: {&tc.key}}, qbtypes.ConditionBuilderOptions{}, tc.operator, tc.value, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, tc.expected)
})
}
}
// TestConditionForAttributeJSONNotExistsDualRead covers NOT EXISTS across both homes during the
// dual-read window: it must AND the JSON IS NULL with NOT mapContains so a row present in either
// home is excluded (De Morgan), including rows that predate the JSON column.
func TestConditionForAttributeJSONNotExistsDualRead(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
cb := NewConditionBuilder(fm, flaggertest.New(t))
evo := MockAttributeEvolutionData(attrJSONRelease)
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, attrWindowStraddle[0], attrWindowStraddle[1], &key,
map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorNotExists, nil, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
// the value multiIf resolves the row's home; NOT EXISTS negates the whole thing to IS NULL
assert.Contains(t, sql, "IS NULL")
assert.Contains(t, sql, "attributes.`user.id` IS NOT NULL")
assert.Contains(t, sql, "mapContains(attributes_string, 'user.id')")
}
// TestColumnExpressionForAttributeJSON covers group-by (coerced to String) and aggregation
// (coerced to Float64) over a JSON attribute after release: both are exists-guarded so an absent
// path is NULL rather than a spurious ”/0, and the numeric branch keeps its toFloat64 coercion.
func TestColumnExpressionForAttributeJSON(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
evo := MockAttributeEvolutionData(attrJSONRelease)
t.Run("group by string", func(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
got, err := fm.ColumnExpressionFor(ctx, valuer.UUID{}, attrWindowAfter[0], attrWindowAfter[1], &key, telemetrytypes.FieldDataTypeString, nil)
require.NoError(t, err)
assert.Equal(t, "multiIf(attributes.`user.id` IS NOT NULL, attributes.`user.id`::String, NULL)", got)
})
t.Run("aggregation numeric", func(t *testing.T) {
key := attrKey("latency", telemetrytypes.FieldDataTypeNumber, evo)
got, err := fm.ColumnExpressionFor(ctx, valuer.UUID{}, attrWindowAfter[0], attrWindowAfter[1], &key, telemetrytypes.FieldDataTypeFloat64, nil)
require.NoError(t, err)
assert.Equal(t, "multiIf(attributes.`latency` IS NOT NULL, toFloat64(attributes.`latency`::Nullable(Float64)), NULL)", got)
})
}
// TestAttributeJSONNoAmbiguityWarning guards against a visible regression: the JSON column is a
// second physical home for the same logical field, not a second logical field, so a plain
// attribute filter must not emit the "ambiguous key" warning.
func TestAttributeJSONNoAmbiguityWarning(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
cb := NewConditionBuilder(fm, flaggertest.New(t))
evo := MockAttributeEvolutionData(attrJSONRelease)
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
sb := sqlbuilder.NewSelectBuilder()
_, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, attrWindowAfter[0], attrWindowAfter[1], &key,
map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "x", sb)
require.NoError(t, err)
assert.Empty(t, warnings, "a plain attribute filter must not emit an ambiguity warning")
}
// TestColumnForUnspecifiedAttributeNoBranchFlip pins the branch-flip decision: a
// data-type-unspecified attribute key resolves to no column (even with the evolution present), so
// bare attribute keys keep taking the legacy CandidateKeys/synthesis path rather than becoming
// metadata-first resolvable.
func TestColumnForUnspecifiedAttributeNoBranchFlip(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper(flaggertest.New(t))
evo := MockAttributeEvolutionData(attrJSONRelease)
key := attrKey("user.id", telemetrytypes.FieldDataTypeUnspecified, evo)
_, err := fm.ColumnFor(ctx, valuer.UUID{}, attrWindowAfter[0], attrWindowAfter[1], &key)
assert.ErrorIs(t, err, qbtypes.ErrColumnNotFound)
}

View File

@@ -154,29 +154,6 @@ func BuildCompleteFieldKeyMap(releaseTime time.Time) map[string][]*telemetrytype
return keysMap
}
// MockAttributeEvolutionData returns the attribute-context evolution timeline: the three legacy
// map columns at epoch 0 and the JSON `attributes` column released at releaseTime. Every entry
// is field_name "__all__", so a typed attribute key carries all four; getColumn keeps only its
// own map plus the JSON column and narrowEvolutionsToColumns drops the rest before selection.
func MockAttributeEvolutionData(releaseTime time.Time) []*telemetrytypes.EvolutionEntry {
entry := func(col, typ string, rt time.Time) *telemetrytypes.EvolutionEntry {
return &telemetrytypes.EvolutionEntry{
Signal: telemetrytypes.SignalTraces,
ColumnName: col,
ColumnType: typ,
FieldContext: telemetrytypes.FieldContextAttribute,
FieldName: "__all__",
ReleaseTime: rt,
}
}
return []*telemetrytypes.EvolutionEntry{
entry("attributes_string", "Map(LowCardinality(String), String)", time.Unix(0, 0)),
entry("attributes_number", "Map(LowCardinality(String), Float64)", time.Unix(0, 0)),
entry("attributes_bool", "Map(LowCardinality(String), Bool)", time.Unix(0, 0)),
entry("attributes", "JSON()", releaseTime),
}
}
// MockEvolutionData returns the canonical resource-column evolution timeline used in tests:
// the legacy resources_string map at epoch 0 and the JSON resource column released at releaseTime.
func MockEvolutionData(releaseTime time.Time) []*telemetrytypes.EvolutionEntry {