Compare commits

..

16 Commits

Author SHA1 Message Date
Nikhil Soni
7e01566687 refactor(qb): skip non-candidate evolutions in place instead of filtering a copy
Assisted-by: Claude Opus 5.5
2026-09-24 22:40:06 +05:30
Nikhil Soni
1cbd6f3fe2 refactor(qb): ignore evolutions of non-candidate columns in evolution selection
The column mapper decides the candidate columns, so an evolution for a column it did not return is not an error.

Assisted-by: Claude Opus 5.5
2026-09-24 22:40:06 +05:30
Nikhil Soni
b1cce4d6a7 feat(traces-qb): gate JSON span attribute reads behind use_trace_attributes_json
The evolution entry is deployment-wide, so a per-deployment flag controls the rollout.

Assisted-by: Claude Opus 5.5
2026-09-24 22:40:06 +05:30
Nikhil Soni
00932bc04e refactor(promote): group target constructors; drop redundant and unsupported-feature tests
- move NewTargetFromPath next to the other target constructors
- drop TestNewTargetFromPath: thin glue over SignalFromText/FieldContextFromText/TargetFor
- drop traces index rejection cases: per-path indexes are simply not supported for traces yet
2026-09-24 19:57:13 +05:30
Nikhil Soni
5d5387f42d test(promote): cover the per-path skip index creation of the logs body domain 2026-09-24 19:57:13 +05:30
Nikhil Soni
3cb4802d84 refactor(promote): move the path resolution to types with a validate method, table-drive the tests 2026-09-24 19:57:13 +05:30
Nikhil Soni
291f1a49ac chore: regenerate openapi spec and api clients 2026-09-24 19:57:13 +05:30
Nikhil Soni
d8120f02f7 test: align the subtest names with the table format rule 2026-09-24 19:54:40 +05:30
Nikhil Soni
d5bf4ac6ff fix(promote): rename the signal path variable to telemetry_signal
orval generates an AbortSignal parameter named signal for every client
method, so a {signal} path variable produced a duplicate identifier in
the generated client (tsc error). The URL itself is unchanged in
behavior: /api/v1/promote_paths/{telemetry_signal}/{context}.
2026-09-24 19:54:39 +05:30
Nikhil Soni
e55f005805 refactor(promote): inline the promote and list helpers into their sole callers 2026-09-24 19:54:39 +05:30
Nikhil Soni
00c176e5bf refactor(promote)!: drop the legacy logs promote_paths routes
There are no consumers of /api/v1/logs/promote_paths, so no backward
compatibility is needed: the logs body domain is served by the generic
/api/v1/promote_paths/{signal}/{context} routes and the legacy routes
and handler methods are removed.
2026-09-24 19:54:39 +05:30
Nikhil Soni
48b047757c refactor(promote): move Target into target.go, enum-style SignalFromText, rename handler method
- Target type definition moves from types.go to target.go alongside its
  constructors, with inline comments
- SignalFromText follows the codebase enum pattern (switch over the
  declared values + Enum method) instead of a string-to-signal map
- generic route handler method renamed HandlePromotePaths -> PromotePaths
2026-09-24 19:54:39 +05:30
Nikhil Soni
5098cfb474 refactor(promote): centralize domain construction and generalize routes
- target construction moves to promotetypes: a generic NewTarget plus
  per-domain constructors (NewLogsBodyTarget, NewTracesAttributesTarget)
  and a TargetFor registry keyed by (signal, context); implpromote and
  telemetrymetadata no longer hand-roll domain literals
- routes generalize to /api/v1/promote_paths/{signal}/{context}: the
  legacy logs body route (/api/v1/logs/promote_paths) is kept for
  compatibility but the domain now travels in the path, so a future logs
  attribute domain does not collide with the logs body route; supersedes
  the /api/v1/traces/promote_paths routes
- add telemetrytypes.SignalFromText for parsing the signal path variable
2026-09-24 19:54:39 +05:30
Nikhil Soni
5a8cbb7347 refactor(promote): template the promotion record with EvolutionEntry
Target now carries an EvolutionEntry template (signal, promoted column
name and type, field context) instead of loose signal/context/column
fields, so the store write is exactly row template + field names +
release time and the hardcoded JSON() column type moves to the domain
definitions. DBName/LocalTableName stay on Target explicitly as index
DDL config, used only by targets with index support.
2026-09-24 19:54:39 +05:30
Nikhil Soni
c95c591e1f refactor(promote): collapse module interface to target-parameterized methods
The per-domain methods were pure delegates; the promotion domain now
travels as promotetypes.Target through Module.ListPromotedPaths /
Module.PromotePaths, with the handler methods (one per route) passing
their domain's target.
2026-09-24 19:54:39 +05:30
Nikhil Soni
5e11b1490f feat(promote): add traces attributes promotion API
Refactor the promote module into a target-parameterized core so the logs
body_v2 flow and future promotion domains share one implementation, and
add the spans attributes JSON column (attributes -> attributes_promoted)
as a second domain behind POST/GET /api/v1/traces/promote_paths.

- promotetypes.Target describes a promotion domain: signal, field
  context, db/table, base/promoted columns, path prefix rule and whether
  per-path skip indexes are supported
- index support is optional per target; traces starts promotion-only
  since the traces query builder does not consume per-path skip indexes
- metadata store GetPromotedPaths/PromotePaths take (signal, column,
  context) instead of being hardcoded to the logs body column
- fix the list response never attaching indexes to promoted entries
  (aggregated by unprefixed name but looked up by prefixed path) and
  reporting indexed+promoted paths twice
2026-09-24 19:54:39 +05:30
129 changed files with 1560 additions and 1483 deletions

View File

@@ -38,6 +38,7 @@ jobs:
fail-fast: false
matrix:
suite:
- alerts
- alertmanager
- alertmanagerrotation
- basepath
@@ -63,7 +64,6 @@ jobs:
- querierauthz
- role
- rootuser
- ruler
- savedview
- semconvfamilies
- serviceaccount

View File

@@ -13100,110 +13100,6 @@ paths:
tags:
- llmpricingrules
x-signoz-stability: alpha
/api/v1/logs/promote_paths:
get:
deprecated: false
description: This endpoints promotes and indexes paths
operationId: ListPromotedAndIndexedPaths
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/PromotetypesPromotePath'
nullable: true
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- VIEWER
- tokenizer:
- VIEWER
summary: Promote and index paths
tags:
- logs
x-signoz-stability: alpha
post:
deprecated: false
description: This endpoints promotes and indexes paths
operationId: HandlePromoteAndIndexPaths
requestBody:
content:
application/json:
schema:
items:
$ref: '#/components/schemas/PromotetypesPromotePath'
nullable: true
type: array
responses:
"201":
description: Created
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Promote and index paths
tags:
- logs
x-signoz-stability: alpha
/api/v1/org/preferences:
get:
deprecated: false
@@ -13375,6 +13271,136 @@ paths:
tags:
- preferences
x-signoz-stability: alpha
/api/v1/promote_paths/{telemetry_signal}/{context}:
get:
deprecated: false
description: This endpoint lists the promoted paths of a JSON column. The promotion
domain is identified by the telemetry_signal and context path variables, e.g.
traces/attribute.
operationId: ListPromotedPaths
parameters:
- in: path
name: telemetry_signal
required: true
schema:
type: string
- in: path
name: context
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/PromotetypesPromotePath'
nullable: true
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- VIEWER
- tokenizer:
- VIEWER
summary: List promoted paths
tags:
- promote
x-signoz-stability: alpha
post:
deprecated: false
description: This endpoint promotes paths of a JSON column to its promoted column.
The promotion domain is identified by the telemetry_signal and context path
variables, e.g. traces/attribute.
operationId: PromotePaths
parameters:
- in: path
name: telemetry_signal
required: true
schema:
type: string
- in: path
name: context
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
items:
$ref: '#/components/schemas/PromotetypesPromotePath'
nullable: true
type: array
responses:
"201":
description: Created
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- EDITOR
- tokenizer:
- EDITOR
summary: Promote paths
tags:
- promote
x-signoz-stability: alpha
/api/v1/roles:
get:
deprecated: false

View File

@@ -4,23 +4,15 @@
* * regenerate with 'pnpm generate:api'
* SigNoz
*/
import { useMutation, useQuery } from 'react-query';
import { useMutation } from 'react-query';
import type {
InvalidateOptions,
MutationFunction,
QueryClient,
QueryFunction,
QueryKey,
UseMutationOptions,
UseMutationResult,
UseQueryOptions,
UseQueryResult,
} from 'react-query';
import type {
HandleExportRawDataPOSTParams,
ListPromotedAndIndexedPaths200,
PromotetypesPromotePathDTO,
Querybuildertypesv5QueryRangeRequestDTO,
RenderErrorResponseDTO,
} from '../sigNoz.schemas';
@@ -28,26 +20,6 @@ import type {
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
const withQueryKey = <T extends object, K>(
query: T,
queryKey: K,
): T & { queryKey: K } => {
const result = { queryKey } as T & { queryKey: K };
for (const key of Object.keys(query)) {
// The explicit queryKey always wins, matching the previous
// `{ ...query, queryKey }` spread where it was set last.
if (key === 'queryKey') {
continue;
}
Object.defineProperty(result, key, {
enumerable: true,
configurable: true,
get: () => (query as Record<string, unknown>)[key],
});
}
return result;
};
/**
* This endpoints allows complex query exporting raw data for traces and logs
* @summary Export raw data
@@ -149,175 +121,3 @@ export const useHandleExportRawDataPOST = <
> => {
return useMutation(getHandleExportRawDataPOSTMutationOptions(options));
};
/**
* This endpoints promotes and indexes paths
* @summary Promote and index paths
*/
export const listPromotedAndIndexedPaths = (signal?: AbortSignal) => {
return GeneratedAPIInstance<ListPromotedAndIndexedPaths200>({
url: `/api/v1/logs/promote_paths`,
method: 'GET',
signal,
});
};
export const getListPromotedAndIndexedPathsQueryKey = () => {
return [`/api/v1/logs/promote_paths`] as const;
};
export const getListPromotedAndIndexedPathsQueryOptions = <
TData = Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError,
TData
>;
}) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ?? getListPromotedAndIndexedPathsQueryKey();
const queryFn: QueryFunction<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>
> = ({ signal }) => listPromotedAndIndexedPaths(signal);
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListPromotedAndIndexedPathsQueryResult = NonNullable<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>
>;
export type ListPromotedAndIndexedPathsQueryError =
ErrorType<RenderErrorResponseDTO>;
/**
* @summary Promote and index paths
*/
export function useListPromotedAndIndexedPaths<
TData = Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError,
TData
>;
}): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListPromotedAndIndexedPathsQueryOptions(options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return withQueryKey(query, queryOptions.queryKey);
}
/**
* @summary Promote and index paths
*/
export const invalidateListPromotedAndIndexedPaths = async (
queryClient: QueryClient,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListPromotedAndIndexedPathsQueryKey() },
options,
);
return queryClient;
};
/**
* This endpoints promotes and indexes paths
* @summary Promote and index paths
*/
export const handlePromoteAndIndexPaths = (
promotetypesPromotePathDTONull?: BodyType<
PromotetypesPromotePathDTO[] | null
> | null,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v1/logs/promote_paths`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: promotetypesPromotePathDTONull,
signal,
});
};
export const getHandlePromoteAndIndexPathsMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
> => {
const mutationKey = ['handlePromoteAndIndexPaths'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> }
> = (props) => {
const { data } = props ?? {};
return handlePromoteAndIndexPaths(data);
};
return { mutationFn, ...mutationOptions };
};
export type HandlePromoteAndIndexPathsMutationResult = NonNullable<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>
>;
export type HandlePromoteAndIndexPathsMutationBody =
| BodyType<PromotetypesPromotePathDTO[] | null>
| undefined;
export type HandlePromoteAndIndexPathsMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @summary Promote and index paths
*/
export const useHandlePromoteAndIndexPaths = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
> => {
return useMutation(getHandlePromoteAndIndexPathsMutationOptions(options));
};

View File

@@ -0,0 +1,262 @@
/**
* ! Do not edit manually
* * The file has been auto-generated using Orval for SigNoz
* * regenerate with 'pnpm generate:api'
* SigNoz
*/
import { useMutation, useQuery } from 'react-query';
import type {
InvalidateOptions,
MutationFunction,
QueryClient,
QueryFunction,
QueryKey,
UseMutationOptions,
UseMutationResult,
UseQueryOptions,
UseQueryResult,
} from 'react-query';
import type {
ListPromotedPaths200,
ListPromotedPathsPathParameters,
PromotePathsPathParameters,
PromotetypesPromotePathDTO,
RenderErrorResponseDTO,
} from '../sigNoz.schemas';
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
const withQueryKey = <T extends object, K>(
query: T,
queryKey: K,
): T & { queryKey: K } => {
const result = { queryKey } as T & { queryKey: K };
for (const key of Object.keys(query)) {
// The explicit queryKey always wins, matching the previous
// `{ ...query, queryKey }` spread where it was set last.
if (key === 'queryKey') {
continue;
}
Object.defineProperty(result, key, {
enumerable: true,
configurable: true,
get: () => (query as Record<string, unknown>)[key],
});
}
return result;
};
/**
* This endpoint lists the promoted paths of a JSON column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.
* @summary List promoted paths
*/
export const listPromotedPaths = (
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<ListPromotedPaths200>({
url: `/api/v1/promote_paths/${telemetrySignal}/${context}`,
method: 'GET',
signal,
});
};
export const getListPromotedPathsQueryKey = ({
telemetrySignal,
context,
}: ListPromotedPathsPathParameters) => {
return [`/api/v1/promote_paths/${telemetrySignal}/${context}`] as const;
};
export const getListPromotedPathsQueryOptions = <
TData = Awaited<ReturnType<typeof listPromotedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedPaths>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ??
getListPromotedPathsQueryKey({ telemetrySignal, context });
const queryFn: QueryFunction<
Awaited<ReturnType<typeof listPromotedPaths>>
> = ({ signal }) => listPromotedPaths({ telemetrySignal, context }, signal);
return {
queryKey,
queryFn,
enabled:
telemetrySignal !== null &&
telemetrySignal !== undefined &&
context !== null &&
context !== undefined,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof listPromotedPaths>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListPromotedPathsQueryResult = NonNullable<
Awaited<ReturnType<typeof listPromotedPaths>>
>;
export type ListPromotedPathsQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary List promoted paths
*/
export function useListPromotedPaths<
TData = Awaited<ReturnType<typeof listPromotedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedPaths>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListPromotedPathsQueryOptions(
{ telemetrySignal, context },
options,
);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return withQueryKey(query, queryOptions.queryKey);
}
/**
* @summary List promoted paths
*/
export const invalidateListPromotedPaths = async (
queryClient: QueryClient,
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListPromotedPathsQueryKey({ telemetrySignal, context }) },
options,
);
return queryClient;
};
/**
* This endpoint promotes paths of a JSON column to its promoted column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.
* @summary Promote paths
*/
export const promotePaths = (
{ telemetrySignal, context }: PromotePathsPathParameters,
promotetypesPromotePathDTONull?: BodyType<
PromotetypesPromotePathDTO[] | null
> | null,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v1/promote_paths/${telemetrySignal}/${context}`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: promotetypesPromotePathDTONull,
signal,
});
};
export const getPromotePathsMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
> => {
const mutationKey = ['promotePaths'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof promotePaths>>,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
}
> = (props) => {
const { pathParams, data } = props ?? {};
return promotePaths(pathParams, data);
};
return { mutationFn, ...mutationOptions };
};
export type PromotePathsMutationResult = NonNullable<
Awaited<ReturnType<typeof promotePaths>>
>;
export type PromotePathsMutationBody =
| BodyType<PromotetypesPromotePathDTO[] | null>
| undefined;
export type PromotePathsMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Promote paths
*/
export const usePromotePaths = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
> => {
return useMutation(getPromotePathsMutationOptions(options));
};

View File

@@ -12470,17 +12470,6 @@ export type ListUnmappedLLMModels200 = {
status: string;
};
export type ListPromotedAndIndexedPaths200 = {
/**
* @type array,null
*/
data: PromotetypesPromotePathDTO[] | null;
/**
* @type string
*/
status: string;
};
export type ListOrgPreferences200 = {
/**
* @type array
@@ -12506,6 +12495,25 @@ export type GetOrgPreference200 = {
export type UpdateOrgPreferencePathParameters = {
name: string;
};
export type ListPromotedPathsPathParameters = {
telemetrySignal: string;
context: string;
};
export type ListPromotedPaths200 = {
/**
* @type array,null
*/
data: PromotetypesPromotePathDTO[] | null;
/**
* @type string
*/
status: string;
};
export type PromotePathsPathParameters = {
telemetrySignal: string;
context: string;
};
export type ListRoles200 = {
/**
* @type array

View File

@@ -47,5 +47,4 @@ export enum LOCALSTORAGE {
DASHBOARDS_LIST_VIEWS = 'DASHBOARDS_LIST_VIEWS',
DASHBOARD_V2_PANEL_COLUMN_WIDTHS = 'DASHBOARD_V2_PANEL_COLUMN_WIDTHS',
LLM_ATTRIBUTE_MAPPING_TEST_SPAN = 'LLM_ATTRIBUTE_MAPPING_TEST_SPAN',
SAVED_VIEW_ENABLED = 'SAVED_VIEW_ENABLED',
}

View File

@@ -3,22 +3,15 @@ import {
MessageActionKindDTO,
SavedViewEntityDTO,
} from 'api/ai-assistant/sigNozAIAssistantAPI.schemas';
import {
getSavedView,
listSavedViews,
} from 'api/generated/services/saved-view';
import {
GetSavedView200,
ListSavedViews200,
SavedviewtypesPanelTypeDTO,
SavedviewtypesSavedViewDTO,
SavedviewtypesSchemaVersionDTO,
SavedviewtypesSourceDTO,
} from 'api/generated/services/sigNoz.schemas';
import { getAllViews } from 'api/saveView/getAllViews';
import { getViewById } from 'api/saveView/getViewById';
import ROUTES from 'constants/routes';
import { QueryParams } from 'constants/query';
import { PANEL_TYPES } from 'constants/queryBuilder';
import { ICompositeMetricQuery } from 'types/api/alerts/compositeQuery';
import { AllViewsProps, ViewProps } from 'types/api/saveViews/types';
import { DataSource } from 'types/common/queryBuilder';
import { AxiosResponse } from 'axios';
import type { History } from 'history';
import {
@@ -38,7 +31,8 @@ import {
} from '../resolveOpenResource';
import { resourceRoute, ResourceType } from '../resourceRoute';
jest.mock('api/generated/services/saved-view');
jest.mock('api/saveView/getAllViews');
jest.mock('api/saveView/getViewById');
jest.mock(
'lib/newQueryBuilder/queryBuilderMappers/mapQueryDataFromApi',
@@ -54,45 +48,43 @@ jest.mock(
}),
);
const mockedListSavedViews = listSavedViews as jest.MockedFunction<
typeof listSavedViews
const mockedGetAllViews = getAllViews as jest.MockedFunction<
typeof getAllViews
>;
const mockedGetSavedView = getSavedView as jest.MockedFunction<
typeof getSavedView
const mockedGetViewById = getViewById as jest.MockedFunction<
typeof getViewById
>;
function makeView(
id: string,
source: SavedviewtypesSourceDTO,
): SavedviewtypesSavedViewDTO {
function makeView(id: string, sourcePage: DataSource): ViewProps {
return {
id,
name: `view-${id}`,
source,
schemaVersion: SavedviewtypesSchemaVersionDTO.v2,
name: `View ${id}`,
category: 'test',
createdAt: '2021-07-07T06:31:00.000Z',
createdBy: 'user',
updatedAt: '2021-07-07T06:33:00.000Z',
updatedBy: 'user',
spec: {
displayName: `View ${id}`,
panelType: SavedviewtypesPanelTypeDTO.list,
requestType: 'raw',
queries: [{ type: 'builder_query', spec: { name: 'A', signal: source } }],
},
} as unknown as SavedviewtypesSavedViewDTO;
sourcePage,
tags: [],
extraData: '',
compositeQuery: {
panelType: PANEL_TYPES.LIST,
} as ICompositeMetricQuery,
};
}
function mockViewsResponse(
views: SavedviewtypesSavedViewDTO[],
): ListSavedViews200 {
return { status: 'success', data: views };
function mockViewsResponse(views: ViewProps[]): AxiosResponse<AllViewsProps> {
return {
data: { status: 'success', data: views },
} as AxiosResponse<AllViewsProps>;
}
function mockViewByIdResponse(
view: SavedviewtypesSavedViewDTO,
): GetSavedView200 {
return { status: 'success', data: view };
view: ViewProps,
): AxiosResponse<{ status: string; data: ViewProps }> {
return {
data: { status: 'success', data: view },
} as AxiosResponse<{ status: string; data: ViewProps }>;
}
describe('resourceRoute', () => {
@@ -198,33 +190,18 @@ describe('resolveOpenResource', () => {
describe('findSavedViewInLists', () => {
beforeEach(() => {
mockedListSavedViews.mockReset();
mockedGetAllViews.mockReset();
});
it('loads only the hinted source when entity is provided', async () => {
const tracesView = makeView('view-traces', SavedviewtypesSourceDTO.traces);
mockedListSavedViews.mockResolvedValueOnce(mockViewsResponse([tracesView]));
const tracesView = makeView('view-traces', DataSource.TRACES);
mockedGetAllViews.mockResolvedValueOnce(mockViewsResponse([tracesView]));
const result = await findSavedViewInLists('view-traces', DataSource.TRACES);
expect(result).toStrictEqual(tracesView);
expect(mockedListSavedViews).toHaveBeenCalledTimes(1);
expect(mockedListSavedViews).toHaveBeenCalledWith({
source: SavedviewtypesSourceDTO.traces,
});
});
it('treats a null list as empty and probes the next source', async () => {
const metricsView = makeView('view-metrics', SavedviewtypesSourceDTO.metrics);
mockedListSavedViews
.mockResolvedValueOnce({ status: 'success', data: null })
.mockResolvedValueOnce(mockViewsResponse([]))
.mockResolvedValueOnce(mockViewsResponse([metricsView]));
const result = await findSavedViewInLists('view-metrics');
expect(result).toStrictEqual(metricsView);
expect(mockedListSavedViews).toHaveBeenCalledTimes(3);
expect(mockedGetAllViews).toHaveBeenCalledTimes(1);
expect(mockedGetAllViews).toHaveBeenCalledWith(DataSource.TRACES);
});
});
@@ -250,75 +227,52 @@ describe('openSavedView', () => {
it('navigates with history.push and view query params', () => {
const push = jest.fn();
const history = { push } as unknown as History;
const view = makeView('view-logs', SavedviewtypesSourceDTO.logs);
const view = makeView('view-logs', DataSource.LOGS);
openSavedView(view, history);
expect(push).toHaveBeenCalledTimes(1);
const pushedUrl = push.mock.calls[0][0] as string;
expect(pushedUrl).toContain(ROUTES.LOGS_EXPLORER);
const params = new URLSearchParams(pushedUrl.split('?')[1]);
expect(params.get(QueryParams.viewKey)).toBe('"view-logs"');
expect(params.get(QueryParams.viewName)).toBe('"View view-logs"');
expect(params.get(QueryParams.panelTypes)).toBe('"list"');
});
it('throws when the view has no source', () => {
const view = makeView('view-logs', SavedviewtypesSourceDTO.logs);
delete view.source;
expect(() =>
openSavedView(view, { push: jest.fn() } as unknown as History),
).toThrow('Unsupported saved view source');
});
it('throws when the view has no queries', () => {
const view = makeView('view-logs', SavedviewtypesSourceDTO.logs);
view.spec.queries = [];
expect(() =>
openSavedView(view, { push: jest.fn() } as unknown as History),
).toThrow('Saved view is missing query data');
expect(pushedUrl).toContain(QueryParams.viewKey);
});
});
describe('openSavedViewByKey', () => {
beforeEach(() => {
mockedListSavedViews.mockReset();
mockedGetSavedView.mockReset();
mockedGetAllViews.mockReset();
mockedGetViewById.mockReset();
});
it('prefers the direct view lookup endpoint', async () => {
const view = makeView('view-logs', SavedviewtypesSourceDTO.logs);
mockedGetSavedView.mockResolvedValueOnce(mockViewByIdResponse(view));
const view = makeView('view-logs', DataSource.LOGS);
mockedGetViewById.mockResolvedValueOnce(mockViewByIdResponse(view));
const push = jest.fn();
const history = { push } as unknown as History;
await openSavedViewByKey('view-logs', DataSource.LOGS, history);
expect(mockedGetSavedView).toHaveBeenCalledWith({ id: 'view-logs' });
expect(mockedListSavedViews).not.toHaveBeenCalled();
expect(mockedGetViewById).toHaveBeenCalledWith('view-logs');
expect(mockedGetAllViews).not.toHaveBeenCalled();
expect(push).toHaveBeenCalled();
});
it('falls back to list probing when direct lookup fails', async () => {
const view = makeView('view-traces', SavedviewtypesSourceDTO.traces);
mockedGetSavedView.mockRejectedValueOnce(new Error('not found'));
mockedListSavedViews.mockResolvedValueOnce(mockViewsResponse([view]));
const view = makeView('view-traces', DataSource.TRACES);
mockedGetViewById.mockRejectedValueOnce(new Error('not found'));
mockedGetAllViews.mockResolvedValueOnce(mockViewsResponse([view]));
const push = jest.fn();
const history = { push } as unknown as History;
await openSavedViewByKey('view-traces', DataSource.TRACES, history);
expect(mockedListSavedViews).toHaveBeenCalledWith({
source: SavedviewtypesSourceDTO.traces,
});
expect(mockedGetAllViews).toHaveBeenCalledWith(DataSource.TRACES);
expect(push).toHaveBeenCalled();
});
it('throws when the saved view does not exist', async () => {
mockedGetSavedView.mockRejectedValueOnce(new Error('not found'));
mockedListSavedViews.mockResolvedValue(mockViewsResponse([]));
mockedGetViewById.mockRejectedValueOnce(new Error('not found'));
mockedGetAllViews.mockResolvedValue(mockViewsResponse([]));
await expect(
openSavedViewByKey('missing', DataSource.LOGS, {

View File

@@ -1,22 +1,15 @@
import {
getSavedView,
listSavedViews,
} from 'api/generated/services/saved-view';
import { SavedviewtypesSavedViewDTO } from 'api/generated/services/sigNoz.schemas';
import { getAllViews } from 'api/saveView/getAllViews';
import { getViewById } from 'api/saveView/getViewById';
import { QueryParams } from 'constants/query';
import { PANEL_TYPES } from 'constants/queryBuilder';
import {
findSavedView,
getSavedViewQuery,
SavedViewSourcePage,
toSavedViewSource,
} from 'container/SavedViews/utils';
import { mapQueryDataFromApi } from 'lib/newQueryBuilder/queryBuilderMappers/mapQueryDataFromApi';
import { SOURCEPAGE_VS_ROUTES } from 'pages/SaveView/constants';
import { ViewProps } from 'types/api/saveViews/types';
import { DataSource } from 'types/common/queryBuilder';
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import { History } from 'history';
type SavedViewSourceHint = SavedViewSourcePage;
type SavedViewSourceHint = DataSource | 'meter';
const DEFAULT_PROBE_SOURCES: SavedViewSourceHint[] = [
DataSource.LOGS,
@@ -27,15 +20,13 @@ const DEFAULT_PROBE_SOURCES: SavedViewSourceHint[] = [
export async function findSavedViewInLists(
viewKey: string,
sourceHint?: SavedViewSourceHint | null,
): Promise<SavedviewtypesSavedViewDTO | null> {
): Promise<ViewProps | null> {
const sources = sourceHint ? [sourceHint] : DEFAULT_PROBE_SOURCES;
for (const source of sources) {
try {
const response = await listSavedViews({
source: toSavedViewSource(source),
});
const match = findSavedView(response.data, viewKey);
const response = await getAllViews(source);
const match = response.data.data.find((view) => view.id === viewKey);
if (match) {
return match;
}
@@ -50,11 +41,11 @@ export async function findSavedViewInLists(
async function loadSavedView(
viewKey: string,
sourceHint?: SavedViewSourceHint | null,
): Promise<SavedviewtypesSavedViewDTO> {
): Promise<ViewProps> {
try {
const response = await getSavedView({ id: viewKey });
if (response.data) {
return response.data;
const response = await getViewById(viewKey);
if (response.data?.data) {
return response.data.data;
}
} catch {
// Fall back to list probing when the direct lookup fails.
@@ -94,23 +85,20 @@ export function buildExplorerNavigationUrl(
return `${route}?${params.toString()}`;
}
export function openSavedView(
view: SavedviewtypesSavedViewDTO,
history: History,
): void {
const route = view.source ? explorerRouteForSourcePage(view.source) : null;
export function openSavedView(view: ViewProps, history: History): void {
const route = explorerRouteForSourcePage(view.sourcePage);
if (!route) {
throw new Error('Unsupported saved view source');
}
if (!view.spec.queries?.length) {
if (!view.compositeQuery) {
throw new Error('Saved view is missing query data');
}
const query = getSavedViewQuery(view);
const query = mapQueryDataFromApi(view.compositeQuery);
const url = buildExplorerNavigationUrl(route, query, {
[QueryParams.panelTypes]: view.spec.panelType as unknown as PANEL_TYPES,
[QueryParams.viewName]: view.spec.displayName,
[QueryParams.panelTypes]: view.compositeQuery.panelType as PANEL_TYPES,
[QueryParams.viewName]: view.name,
[QueryParams.viewKey]: view.id,
});
history.push(url);
@@ -124,3 +112,6 @@ export async function openSavedViewByKey(
const view = await loadSavedView(viewKey, sourceHint);
openSavedView(view, history);
}
/** @deprecated Use findSavedViewInLists — kept for tests. */
export const findSavedView = findSavedViewInLists;

View File

@@ -53,10 +53,6 @@
z-index: 0;
background: var(--l1-background);
// Column so the bottom strip sits under the scrolling content, not inside it.
display: flex;
flex-direction: column;
&.full-screen-content {
width: 100%;
}
@@ -74,9 +70,7 @@
.chat-support-gateway {
position: fixed;
// Lifted above the bottom strip. Don't extend this pattern — new fixed-bottom
// UI belongs in the bounded layout, not in another offset here.
bottom: calc(20px + var(--bottom-strip-height, 0px));
bottom: 20px;
right: 20px;
z-index: 1000;

View File

@@ -43,7 +43,6 @@ import { USER_PREFERENCES } from 'constants/userPreferences';
import AIAssistantModal from 'container/AIAssistant/AIAssistantModal';
import AIAssistantPanel from 'container/AIAssistant/AIAssistantPanel';
import { useAIAssistantStore } from 'container/AIAssistant/store/useAIAssistantStore';
import BottomStrip from 'container/BottomStrip';
import SideNav from 'container/SideNav';
import TopNav from 'container/TopNav';
import dayjs from 'dayjs';
@@ -52,7 +51,6 @@ import { useIsDarkMode } from 'hooks/useDarkMode';
import { useGetTenantLicense } from 'hooks/useGetTenantLicense';
import { useIsAIAssistantEnabled } from 'hooks/useIsAIAssistantEnabled';
import { useNotifications } from 'hooks/useNotifications';
import { useSavedViewEnabled } from 'hooks/useSavedViewEnabled';
import useTabVisibility from 'hooks/useTabFocus';
import history from 'lib/history';
import { isNull } from 'lodash-es';
@@ -404,7 +402,6 @@ function AppLayout(props: AppLayoutProps): JSX.Element {
}, [pathname]);
const isToDisplayLayout = isLoggedIn;
const isSavedViewEnabled = useSavedViewEnabled();
const routeKey = useMemo(() => getRouteKey(pathname), [pathname]);
const pageTitle = t(routeKey);
@@ -871,10 +868,6 @@ function AppLayout(props: AppLayoutProps): JSX.Element {
</OverlayScrollbar>
</LayoutContent>
</Sentry.ErrorBoundary>
{isSavedViewEnabled && isToDisplayLayout && !renderFullScreen && (
<BottomStrip />
)}
</div>
{isLoggedIn && isAIAssistantEnabled && (

View File

@@ -12,12 +12,8 @@ export const Layout = styled(LayoutComponent)`
}
`;
// Takes the height left in `.app-content` after the bottom strip.
// `min-height: 0` is not needed right now, overlayscrollbars already sets
// `overflow: auto` here. Kept so this does not break if that goes away.
export const LayoutContent = styled(LayoutComponent.Content)`
flex: 1;
min-height: 0;
height: 100%;
&::-webkit-scrollbar {
width: 0.1rem;
}

View File

@@ -1,36 +0,0 @@
.strip {
display: flex;
align-items: center;
justify-content: space-between;
gap: var(--spacing-6);
flex-shrink: 0;
height: var(--bottom-strip-height);
padding: 0 var(--spacing-6);
background: var(--l2-background);
border-top: 1px solid var(--l2-border);
font-family: var(--font-family-sf-mono, monospace);
// Above page content, below the body-portalled overlays that are meant to
// cover the strip.
position: relative;
z-index: 1;
}
.left,
.right {
display: flex;
align-items: center;
gap: var(--spacing-6);
min-width: 0;
}
// Temporary placeholder for the left slot. Replaced later.
.version {
color: var(--l2-foreground);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}

View File

@@ -1,49 +0,0 @@
import { render } from 'tests/test-utils';
import BottomStrip, {
BOTTOM_STRIP_HEIGHT,
BOTTOM_STRIP_HEIGHT_VAR,
BOTTOM_STRIP_ON_CLASS,
} from '..';
describe('BottomStrip', () => {
it('publishes the body class and height property while mounted', () => {
const { unmount } = render(<BottomStrip />);
expect(document.body.classList.contains(BOTTOM_STRIP_ON_CLASS)).toBe(true);
expect(document.body.style.getPropertyValue(BOTTOM_STRIP_HEIGHT_VAR)).toBe(
`${BOTTOM_STRIP_HEIGHT}px`,
);
unmount();
expect(document.body.classList.contains(BOTTOM_STRIP_ON_CLASS)).toBe(false);
expect(document.body.style.getPropertyValue(BOTTOM_STRIP_HEIGHT_VAR)).toBe(
'',
);
});
// The string is whatever the Go build injected, so it is rendered untouched —
// same as SideNav. Release tags carry the "v", local builds do not.
it.each([['v0.134.67'], ['main-64f1c2a']])(
'renders the build version %p exactly as given',
(version) => {
const { getByTestId } = render(<BottomStrip />, undefined, {
appContextOverrides: {
versionData: { version, ee: 'Y', setupCompleted: true },
},
});
expect(getByTestId('bottom-strip-version')).toHaveTextContent(version);
},
);
it('renders the strip without a version when none is available', () => {
const { getByTestId, queryByTestId } = render(<BottomStrip />, undefined, {
appContextOverrides: { versionData: null },
});
expect(getByTestId('bottom-strip')).toBeInTheDocument();
expect(queryByTestId('bottom-strip-version')).not.toBeInTheDocument();
});
});

View File

@@ -1,42 +0,0 @@
import { useLayoutEffect } from 'react';
import { useAppContext } from 'providers/App/App';
import styles from './BottomStrip.module.scss';
export const BOTTOM_STRIP_HEIGHT = 24;
export const BOTTOM_STRIP_ON_CLASS = 'bottom-strip-on';
export const BOTTOM_STRIP_HEIGHT_VAR = '--bottom-strip-height';
function BottomStrip(): JSX.Element {
const { versionData } = useAppContext();
const version = versionData?.version?.trim();
useLayoutEffect(() => {
document.body.classList.add(BOTTOM_STRIP_ON_CLASS);
document.body.style.setProperty(
BOTTOM_STRIP_HEIGHT_VAR,
`${BOTTOM_STRIP_HEIGHT}px`,
);
return (): void => {
document.body.classList.remove(BOTTOM_STRIP_ON_CLASS);
document.body.style.removeProperty(BOTTOM_STRIP_HEIGHT_VAR);
};
}, []);
return (
<div className={styles.strip} data-testid="bottom-strip">
<div className={styles.left}>
{version && (
<span className={styles.version} data-testid="bottom-strip-version">
{version}
</span>
)}
</div>
<div className={styles.right} />
</div>
);
}
export default BottomStrip;

View File

@@ -1,8 +1,6 @@
.create-alert-v2-footer {
position: fixed;
// Lifted above the bottom strip. Don't extend this pattern — new fixed-bottom
// UI belongs in the bounded layout, not in another offset here.
bottom: var(--bottom-strip-height, 0px);
bottom: 0;
left: 63px;
right: 0;
background-color: var(--l1-background);

View File

@@ -1,8 +1,6 @@
.explorer-options-container {
position: fixed;
// Lifted above the bottom strip. Don't extend this pattern — new fixed-bottom
// UI belongs in the bounded layout, not in another offset here.
bottom: var(--bottom-strip-height, 0px);
bottom: 0px;
left: calc(50% + 240px);
transform: translate(calc(-50% - 120px), 0);
transition: left 0.2s linear;

View File

@@ -1,8 +1,6 @@
.explorer-option-droppable-container {
position: fixed;
// Lifted above the bottom strip. Don't extend this pattern — new fixed-bottom
// UI belongs in the bounded layout, not in another offset here.
bottom: var(--bottom-strip-height, 0px);
bottom: 0;
width: -webkit-fill-available;
height: 24px;
display: flex;

View File

@@ -1,6 +1,7 @@
.home-container {
display: flex;
flex-direction: column;
min-height: 100vh;
overflow-y: auto;
height: 100%;
width: 100%;

View File

@@ -1,18 +1,17 @@
import { useEffect, useMemo, useState } from 'react';
import { Link } from 'react-router-dom';
import { Button, Skeleton } from 'antd';
import { Badge } from '@signozhq/ui/badge';
import logEvent from 'api/common/logEvent';
import { useListSavedViews } from 'api/generated/services/saved-view';
import {
SavedviewtypesSavedViewDTO,
SavedviewtypesSourceDTO,
} from 'api/generated/services/sigNoz.schemas';
import { getViewDetailsUsingViewKey } from 'components/ExplorerCard/utils';
import ROUTES from 'constants/routes';
import { getSavedViewQuery } from 'container/SavedViews/utils';
import { useGetAllViews } from 'hooks/saveViews/useGetAllViews';
import { useHandleExplorerTabChange } from 'hooks/useHandleExplorerTabChange';
import { SOURCEPAGE_VS_ROUTES } from 'pages/SaveView/constants';
import Card from 'periscope/components/Card/Card';
import { useAppContext } from 'providers/App/App';
import { ViewProps } from 'types/api/saveViews/types';
import { DataSource } from 'types/common/queryBuilder';
import { USER_ROLES } from 'types/roles';
import floppyDiscUrl from '@/assets/Icons/floppy-disc.svg';
@@ -36,40 +35,38 @@ export default function SavedViews({
}): JSX.Element {
const { user } = useAppContext();
const [selectedEntity, setSelectedEntity] = useState<string>('logs');
const [selectedEntityViews, setSelectedEntityViews] = useState<
SavedviewtypesSavedViewDTO[]
>([]);
const [selectedEntityViews, setSelectedEntityViews] = useState<any[]>([]);
const {
data: logsViewsData,
isLoading: logsViewsLoading,
isError: logsViewsError,
} = useListSavedViews({ source: SavedviewtypesSourceDTO.logs });
} = useGetAllViews(DataSource.LOGS);
const {
data: tracesViewsData,
isLoading: tracesViewsLoading,
isError: tracesViewsError,
} = useListSavedViews({ source: SavedviewtypesSourceDTO.traces });
} = useGetAllViews(DataSource.TRACES);
const {
data: metricsViewsData,
isLoading: metricsViewsLoading,
isError: metricsViewsError,
} = useListSavedViews({ source: SavedviewtypesSourceDTO.metrics });
} = useGetAllViews(DataSource.METRICS);
const logsViews = useMemo(
() => [...(logsViewsData?.data || [])],
() => [...(logsViewsData?.data.data || [])],
[logsViewsData],
);
const tracesViews = useMemo(
() => [...(tracesViewsData?.data || [])],
() => [...(tracesViewsData?.data.data || [])],
[tracesViewsData],
);
const metricsViews = useMemo(
() => [...(metricsViewsData?.data || [])],
() => [...(metricsViewsData?.data.data || [])],
[metricsViewsData],
);
@@ -91,22 +88,39 @@ export default function SavedViews({
const { handleExplorerTabChange } = useHandleExplorerTabChange();
const handleRedirectQuery = (view: SavedviewtypesSavedViewDTO): void => {
const handleRedirectQuery = (view: ViewProps): void => {
logEvent('Homepage: Saved view clicked', {
viewId: view.id,
viewName: view.spec.displayName,
viewName: view.name,
entity: selectedEntity,
});
handleExplorerTabChange(
view.spec.panelType,
{
query: getSavedViewQuery(view),
viewName: view.spec.displayName,
viewKey: view.id,
},
SOURCEPAGE_VS_ROUTES[selectedEntity],
);
let currentViews: ViewProps[] = [];
if (selectedEntity === 'logs') {
currentViews = logsViews;
} else if (selectedEntity === 'traces') {
currentViews = tracesViews;
} else if (selectedEntity === 'metrics') {
currentViews = metricsViews;
}
const currentViewDetails = getViewDetailsUsingViewKey(view.id, currentViews);
if (!currentViewDetails) {
return;
}
const { query, name, id, panelType: currentPanelType } = currentViewDetails;
if (selectedEntity) {
handleExplorerTabChange(
currentPanelType,
{
query,
viewName: name,
viewKey: id,
},
SOURCEPAGE_VS_ROUTES[selectedEntity],
);
}
};
useEffect(() => {
@@ -225,10 +239,24 @@ export default function SavedViews({
/>
<div className="saved-view-item-name home-data-item-name">
{view.spec.displayName}
{view.name}
</div>
</div>
<div className="saved-view-item-description home-data-item-tag">
{view.tags?.map((tag: string) => {
if (tag === '') {
return null;
}
return (
<Badge color="sienna" key={tag}>
{tag}
</Badge>
);
})}
</div>
<Button
type="link"
size="small"
@@ -279,7 +307,7 @@ export default function SavedViews({
logEvent('Homepage: Saved views switched', {
tab,
});
let currentViews: SavedviewtypesSavedViewDTO[] = [];
let currentViews: ViewProps[] = [];
if (tab === 'logs') {
currentViews = logsViews;
} else if (tab === 'traces') {

View File

@@ -1,4 +1,7 @@
.licenses-page {
max-height: 100vh;
overflow: hidden;
.licenses-page-header {
border-bottom: 1px solid var(--l1-border);
background: var(--l1-background);
@@ -29,6 +32,7 @@
.licenses-page-content {
flex: 1;
height: calc(100vh - 48px);
background: var(--l1-background);
padding: 10px 8px;
overflow-y: auto;

View File

@@ -2,7 +2,7 @@
display: flex;
flex-direction: column;
gap: 1rem;
flex: 1;
height: calc(100vh - 62px);
min-height: 400px;
}

View File

@@ -181,9 +181,7 @@
.ant-pagination {
position: fixed;
// Lifted above the bottom strip. Don't extend this pattern — new
// fixed-bottom UI belongs in the bounded layout, not in another offset here.
bottom: var(--bottom-strip-height, 0px);
bottom: 0;
width: calc(100% - 54px);
background: var(--l1-background);
padding: 16px;

View File

@@ -1,126 +0,0 @@
import {
SavedviewtypesPanelTypeDTO,
SavedviewtypesSavedViewDTO,
SavedviewtypesSchemaVersionDTO,
SavedviewtypesSourceDTO,
} from 'api/generated/services/sigNoz.schemas';
import { PANEL_TYPES } from 'constants/queryBuilder';
import { EQueryType } from 'types/common/dashboard';
import { DataSource } from 'types/common/queryBuilder';
import { findSavedView, getSavedViewQuery, toSavedViewSource } from '../utils';
jest.mock('uuid', () => ({
v4: (): string => 'test-id',
}));
function makeView(): SavedviewtypesSavedViewDTO {
return {
id: 'view-1',
name: 'errors-by-service-abc123',
source: SavedviewtypesSourceDTO.traces,
schemaVersion: SavedviewtypesSchemaVersionDTO.v2,
createdBy: 'a@b.c',
updatedBy: 'a@b.c',
spec: {
displayName: 'Errors by service',
panelType: SavedviewtypesPanelTypeDTO.list,
requestType: 'raw',
queries: [
{
type: 'builder_query',
spec: {
name: 'A',
signal: 'traces',
stepInterval: 60,
filter: { expression: 'has_error = true' },
// v2 reads back fully defaulted envelopes; nulls must not break the mapper
groupBy: null,
order: null,
selectFields: null,
functions: null,
legend: '',
disabled: false,
},
},
],
selectedFields: [{ name: 'service.name' }],
display: { color: 'red' },
},
} as SavedviewtypesSavedViewDTO;
}
describe('getSavedViewQuery', () => {
it('maps the v2 spec through the v5 branch of mapQueryDataFromApi', () => {
const query = getSavedViewQuery(makeView());
expect(query.queryType).toBe(EQueryType.QUERY_BUILDER);
expect(query.promql).toStrictEqual([]);
expect(query.clickhouse_sql).toStrictEqual([]);
expect(query.builder.queryData).toHaveLength(1);
const [queryData] = query.builder.queryData;
expect(queryData.queryName).toBe('A');
expect(queryData.dataSource).toBe(DataSource.TRACES);
expect(queryData.filter).toStrictEqual({ expression: 'has_error = true' });
expect(queryData.groupBy).toStrictEqual([]);
expect(queryData.orderBy).toStrictEqual([]);
});
it('keeps formulas alongside builder queries', () => {
const view = makeView();
view.spec.queries.push({
type: 'builder_formula',
spec: { name: 'F1', expression: 'A / 2' },
} as SavedviewtypesSavedViewDTO['spec']['queries'][number]);
const query = getSavedViewQuery(view);
expect(query.builder.queryData).toHaveLength(1);
expect(query.builder.queryFormulas).toHaveLength(1);
expect(query.builder.queryFormulas[0].queryName).toBe('F1');
});
it('does not read the panel type into the query', () => {
const view = makeView();
view.spec.panelType = SavedviewtypesPanelTypeDTO.graph;
const query = getSavedViewQuery(view);
// panelType travels separately (url param), the Query itself has no such field
expect(query).not.toHaveProperty('panelType', PANEL_TYPES.TIME_SERIES);
});
});
describe('toSavedViewSource', () => {
it('maps every explorer source page to the v2 source', () => {
expect(toSavedViewSource(DataSource.LOGS)).toBe(SavedviewtypesSourceDTO.logs);
expect(toSavedViewSource(DataSource.TRACES)).toBe(
SavedviewtypesSourceDTO.traces,
);
expect(toSavedViewSource(DataSource.METRICS)).toBe(
SavedviewtypesSourceDTO.metrics,
);
expect(toSavedViewSource('meter')).toBe(SavedviewtypesSourceDTO.meter);
});
});
describe('findSavedView', () => {
const views = [
{ ...makeView(), id: 'a' },
{ ...makeView(), id: 'b' },
];
it('returns the view with the matching id', () => {
expect(findSavedView(views, 'b')?.id).toBe('b');
});
it('returns undefined when the id is not in the list', () => {
expect(findSavedView(views, 'c')).toBeUndefined();
});
it('returns undefined for a null or not yet loaded list', () => {
expect(findSavedView(null, 'a')).toBeUndefined();
expect(findSavedView(undefined, 'a')).toBeUndefined();
});
});

View File

@@ -1,49 +0,0 @@
import {
SavedviewtypesSavedViewDTO,
SavedviewtypesSourceDTO,
} from 'api/generated/services/sigNoz.schemas';
import { PANEL_TYPES } from 'constants/queryBuilder';
import { mapQueryDataFromApi } from 'lib/newQueryBuilder/queryBuilderMappers/mapQueryDataFromApi';
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import { QueryEnvelope } from 'types/api/v5/queryRange';
import { EQueryType } from 'types/common/dashboard';
import { DataSource } from 'types/common/queryBuilder';
export type SavedViewSourcePage = DataSource | 'meter';
// Explorers and the preferences module are keyed by DataSource (the signal),
// the api keys views by source page. Same values today, so this is the one
// place they meet. AI observability views will come with their own source and
// DataSource cannot tell them apart from traces, so preferences should move to
// source page at that point and this map goes with it.
const SAVED_VIEW_SOURCE: Record<SavedViewSourcePage, SavedviewtypesSourceDTO> =
{
[DataSource.LOGS]: SavedviewtypesSourceDTO.logs,
[DataSource.TRACES]: SavedviewtypesSourceDTO.traces,
[DataSource.METRICS]: SavedviewtypesSourceDTO.metrics,
meter: SavedviewtypesSourceDTO.meter,
};
export function toSavedViewSource(
sourcePage: SavedViewSourcePage,
): SavedviewtypesSourceDTO {
return SAVED_VIEW_SOURCE[sourcePage];
}
// Explorers only save builder queries; v2 carries no queryType, so it is fixed here.
export function getSavedViewQuery(view: SavedviewtypesSavedViewDTO): Query {
const { queries, panelType } = view.spec;
return mapQueryDataFromApi({
queries: queries as QueryEnvelope[],
panelType: panelType as unknown as PANEL_TYPES,
queryType: EQueryType.QUERY_BUILDER,
unit: undefined,
});
}
export function findSavedView(
views: SavedviewtypesSavedViewDTO[] | null | undefined,
id: string,
): SavedviewtypesSavedViewDTO | undefined {
return views?.find((view) => view.id === id);
}

View File

@@ -2,7 +2,7 @@
display: flex;
flex-direction: column;
gap: 1rem;
flex: 1;
height: calc(100vh - 62px);
min-height: 400px;
padding-top: var(--spacing-8);
}

View File

@@ -1,4 +1,7 @@
.version-container {
max-height: 100vh;
overflow: hidden;
.version-page-header {
border-bottom: 1px solid var(--l1-border);
background: var(--l1-background);

View File

@@ -1,18 +1,11 @@
import { useMutation, UseMutationResult, useQueryClient } from 'react-query';
import { invalidateListSavedViews } from 'api/generated/services/saved-view';
import { useMutation, UseMutationResult } from 'react-query';
import { deleteView } from 'api/saveView/deleteView';
import { DeleteViewPayloadProps } from 'types/api/saveViews/types';
export const useDeleteView = (
uuid: string,
): UseMutationResult<DeleteViewPayloadProps, Error, string> => {
const queryClient = useQueryClient();
return useMutation({
): UseMutationResult<DeleteViewPayloadProps, Error, string> =>
useMutation({
mutationKey: [uuid],
mutationFn: () => deleteView(uuid),
// v1 and v2 share storage; consumers already on v2 must see this write.
// Temporary till the v1 client is deleted with the explorer bar.
onSuccess: () => invalidateListSavedViews(queryClient),
});
};

View File

@@ -1,5 +1,4 @@
import { useMutation, UseMutationResult, useQueryClient } from 'react-query';
import { invalidateListSavedViews } from 'api/generated/services/saved-view';
import { useMutation, UseMutationResult } from 'react-query';
import { saveView } from 'api/saveView/saveView';
import { AxiosResponse } from 'axios';
import { SaveViewPayloadProps, SaveViewProps } from 'types/api/saveViews/types';
@@ -14,14 +13,8 @@ export const useSaveView = ({
Error,
SaveViewProps,
SaveViewPayloadProps
> => {
const queryClient = useQueryClient();
return useMutation({
> =>
useMutation({
mutationKey: [viewName, sourcePage, compositeQuery, extraData],
mutationFn: saveView,
// v1 and v2 share storage; consumers already on v2 must see this write.
// Temporary till the v1 client is deleted with the explorer bar.
onSuccess: () => invalidateListSavedViews(queryClient),
});
};

View File

@@ -1,5 +1,4 @@
import { useMutation, UseMutationResult, useQueryClient } from 'react-query';
import { invalidateListSavedViews } from 'api/generated/services/saved-view';
import { useMutation, UseMutationResult } from 'react-query';
import { updateView } from 'api/saveView/updateView';
import {
UpdateViewPayloadProps,
@@ -17,10 +16,8 @@ export const useUpdateView = ({
Error,
UpdateViewProps,
UpdateViewPayloadProps
> => {
const queryClient = useQueryClient();
return useMutation({
> =>
useMutation({
mutationKey: [viewName, sourcePage, compositeQuery, extraData],
mutationFn: () =>
updateView({
@@ -30,8 +27,4 @@ export const useUpdateView = ({
sourcePage,
viewKey,
}),
// v1 and v2 share storage; consumers already on v2 must see this write.
// Temporary till the v1 client is deleted with the explorer bar.
onSuccess: () => invalidateListSavedViews(queryClient),
});
};

View File

@@ -1,11 +0,0 @@
import getLocalStorageKey from 'api/browser/localstorage/get';
import { LOCALSTORAGE } from 'constants/localStorage';
import { useState } from 'react';
export function useSavedViewEnabled(): boolean {
const [isEnabled] = useState(
() => getLocalStorageKey(LOCALSTORAGE.SAVED_VIEW_ENABLED) === 'true',
);
return isEnabled;
}

View File

@@ -1,29 +1,4 @@
.alerts-container {
// Hands the page height down to the active tab so its content can bound itself
// instead of guessing with 100vh. Child combinators only, nested Tabs
// (Configuration) must not be caught.
flex: 1;
min-height: 0;
> .ant-tabs-content-holder {
display: flex;
flex-direction: column;
> .ant-tabs-content {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
> .ant-tabs-tabpane-active {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}
}
}
.top-level-tab.periscope-tab {
padding: 2px 0;
}
@@ -65,9 +40,5 @@
.alert-rules-container {
margin-top: 10px;
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}
}

View File

@@ -8,7 +8,6 @@ import {
QuickfiltertypesSourceDTO,
TelemetrytypesFieldContextDTO,
TelemetrytypesFieldDataTypeDTO,
TelemetrytypesSignalDTO,
} from 'api/generated/services/sigNoz.schemas';
import ROUTES from 'constants/routes';
import { VIEWS } from 'container/ApiMonitoring/Explorer/Domains/DomainDetails/constants';
@@ -25,10 +24,7 @@ import {
toggleControl,
} from '@/storybook/controls/controls';
import { defineStoryMocks } from '@/storybook/controls/defineStoryMocks';
import {
fieldKeysResponse,
fieldValuesResponse,
} from '@/storybook/msw/__story_mockdata__/fields';
import { fieldValuesResponse } from '@/storybook/msw/__story_mockdata__/fields';
import { quickFiltersResponse } from '@/storybook/msw/__story_mockdata__/quickFilters';
import {
@@ -321,21 +317,6 @@ export const apiMonitoringMocks = defineStoryMocks({
})),
),
rest.get(
'http://localhost/api/v1/fields/keys',
response.json((req) =>
fieldKeysResponse(
groupByAttributeKeys(req.url.searchParams.get('searchText') ?? '').map(
({ key }) => key,
),
{
signal: TelemetrytypesSignalDTO.traces,
fieldContext: TelemetrytypesFieldContextDTO.attribute,
},
),
),
),
rest.get(
'http://localhost/api/v1/fields/values',
response.json((req) =>

View File

@@ -2,9 +2,7 @@
display: flex;
flex-direction: column;
position: fixed;
// Lifted above the bottom strip. Don't extend this pattern — new fixed-bottom
// UI belongs in the bounded layout, not in another offset here.
bottom: var(--bottom-strip-height, 0px);
bottom: 0;
left: 0;
width: 100%;
z-index: 100;

View File

@@ -164,10 +164,10 @@ export const homeMocks = defineStoryMocks({
),
rest.get(
'http://localhost/api/v2/saved_views',
'http://localhost/api/v1/explorer/views',
response.json((req) => {
const source = req.url.searchParams.get('source') ?? 'logs';
const signal = isSavedViewSignal(source) ? source : 'logs';
const sourcePage = req.url.searchParams.get('sourcePage') ?? 'logs';
const signal = isSavedViewSignal(sourcePage) ? sourcePage : 'logs';
return savedViewsResponse(
values.savedViewSignals.includes(signal) ? values.savedViews : 0,

View File

@@ -6,21 +6,10 @@
import { FeatureKeys } from 'constants/features';
import { ORG_PREFERENCES } from 'constants/orgPreferences';
import { checkListStepToPreferenceKeyMap } from 'container/Home/constants';
import {
type ListSavedViews200,
Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5LogAggregationDTOSignal as LogsSignal,
Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5MetricAggregationDTOSignal as MetricsSignal,
Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregationDTOSignal as TracesSignal,
Querybuildertypesv5QueryEnvelopeBuilderDTOType,
type Querybuildertypesv5QueryEnvelopeDTO,
Querybuildertypesv5RequestTypeDTO,
type RuletypesRuleDTO,
SavedviewtypesPanelTypeDTO,
SavedviewtypesSchemaVersionDTO,
SavedviewtypesSourceDTO,
} from 'api/generated/services/sigNoz.schemas';
import type { RuletypesRuleDTO } from 'api/generated/services/sigNoz.schemas';
import type { ServiceDataProps } from 'api/metrics/getTopLevelOperations';
import { alertRulesFixture } from 'mocks-server/__mockdata__/alert_rules';
import { explorerView } from 'mocks-server/__mockdata__/explorer_views';
import { defaultFeatureFlags } from 'tests/fixtures/appContextMock';
import type { FeatureFlagProps } from 'types/api/features/getFeaturesFlags';
import type { MetricRangePayloadV3 } from 'types/api/metrics/getQueryRange';
@@ -176,53 +165,20 @@ const VIEW_NAMES: Record<SavedViewSignal, string[]> = {
export const isSavedViewSignal = (value: string): value is SavedViewSignal =>
SAVED_VIEW_SIGNALS.includes(value as SavedViewSignal);
const SAVED_VIEW_SOURCE: Record<SavedViewSignal, SavedviewtypesSourceDTO> = {
logs: SavedviewtypesSourceDTO.logs,
traces: SavedviewtypesSourceDTO.traces,
metrics: SavedviewtypesSourceDTO.metrics,
};
const SAVED_VIEW_QUERY: Record<
SavedViewSignal,
Querybuildertypesv5QueryEnvelopeDTO
> = {
logs: {
type: Querybuildertypesv5QueryEnvelopeBuilderDTOType.builder_query,
spec: { name: 'A', signal: LogsSignal.logs },
},
traces: {
type: Querybuildertypesv5QueryEnvelopeBuilderDTOType.builder_query,
spec: { name: 'A', signal: TracesSignal.traces },
},
metrics: {
type: Querybuildertypesv5QueryEnvelopeBuilderDTOType.builder_query,
spec: { name: 'A', signal: MetricsSignal.metrics },
},
};
export const savedViewsResponse = (
count: number,
signal: SavedViewSignal,
): ListSavedViews200 => {
const names = VIEW_NAMES[signal];
sourcePage: SavedViewSignal,
): Record<string, unknown> => {
const names = VIEW_NAMES[sourcePage];
return {
status: 'success',
data: Array.from({ length: Math.min(count, names.length) }, (_, index) => ({
id: `storybook-${signal}-view-${index + 1}`,
name: `storybook-${signal}-view-${index + 1}`,
source: SAVED_VIEW_SOURCE[signal],
schemaVersion: SavedviewtypesSchemaVersionDTO.v2,
createdAt: '2026-08-20T09:00:00Z',
createdBy: 'storybook@signoz.io',
updatedAt: '2026-08-20T09:00:00Z',
updatedBy: 'storybook@signoz.io',
spec: {
displayName: names[index],
panelType: SavedviewtypesPanelTypeDTO.list,
requestType: Querybuildertypesv5RequestTypeDTO.raw,
queries: [SAVED_VIEW_QUERY[signal]],
},
...explorerView.data[0],
id: `storybook-${sourcePage}-view-${index + 1}`,
name: names[index],
sourcePage,
tags: [sourcePage],
})),
};
};

View File

@@ -1,4 +1,7 @@
.support-page-container {
max-height: 100vh;
overflow: hidden;
.support-page-header {
border-bottom: 1px solid var(--l1-border);
background: var(--l1-background);

View File

@@ -1,6 +1,5 @@
.root {
flex: 1;
min-height: 0;
height: calc(100vh);
display: flex;
flex-direction: column;
}

View File

@@ -1,24 +1,13 @@
.traces-funnel-details {
display: flex;
height: 100%;
// 45px -> height of the tab bar
height: calc(100vh - 45px);
&__steps-config {
flex-shrink: 0;
width: 600px;
border-right: 1px solid var(--l1-border);
// Positioning context for the absolute .steps-footer.
position: relative;
display: flex;
flex-direction: column;
// Scoped here so the modal usage of FunnelConfiguration on trace details
// stays in normal flow.
.funnel-configuration {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
}
}
&__steps-results {
width: 100%;

View File

@@ -4,17 +4,14 @@
flex-direction: column;
justify-content: flex-start;
&.funnel-details-page {
flex: 1;
min-height: 0;
// .steps-footer is absolute against the config column, so its 64px is
// reserved rather than laid out.
margin-bottom: 64px;
height: calc(
100vh - 170px
); // 64px bottom bar + 61px configuration header + 45px page navbar
overflow: auto;
}
}
&__header {
flex-shrink: 0;
display: flex;
align-items: center;
justify-content: space-between;

View File

@@ -1,202 +0,0 @@
import { renderHook } from '@testing-library/react';
import { useListSavedViews } from 'api/generated/services/saved-view';
import {
SavedviewtypesSavedViewDTO,
SavedviewtypesSourceDTO,
} from 'api/generated/services/sigNoz.schemas';
import {
defaultLogsSelectedColumns,
defaultTraceSelectedColumns,
ensureLogsRequiredColumns,
} from 'container/OptionsMenu/constants';
import { DataSource } from 'types/common/queryBuilder';
import { usePreferenceSync } from '../sync/usePreferenceSync';
import { PreferenceMode } from '../types';
jest.mock('api/generated/services/saved-view');
const loaderPreferences = { columns: [{ name: 'from-loader' }] };
jest.mock('../loader/usePreferenceLoader', () => ({
usePreferenceLoader: jest.fn(() => ({
preferences: loaderPreferences,
loading: false,
error: null,
})),
}));
jest.mock('../updater/usePreferenceUpdater', () => ({
usePreferenceUpdater: jest.fn(() => ({
updateColumns: jest.fn(),
updateFormatting: jest.fn(),
})),
}));
const mockedUseListSavedViews = useListSavedViews as jest.MockedFunction<
typeof useListSavedViews
>;
function makeView(
id: string,
source: SavedviewtypesSourceDTO,
spec: Partial<SavedviewtypesSavedViewDTO['spec']>,
): SavedviewtypesSavedViewDTO {
return {
id,
source,
schemaVersion: 'v2',
spec: {
displayName: id,
panelType: 'list',
requestType: 'raw',
queries: [],
...spec,
},
} as unknown as SavedviewtypesSavedViewDTO;
}
function mockViews(views: SavedviewtypesSavedViewDTO[]): void {
mockedUseListSavedViews.mockReturnValue({
data: { status: 'success', data: views },
} as unknown as ReturnType<typeof useListSavedViews>);
}
describe('usePreferenceSync in saved view mode', () => {
beforeEach(() => {
mockedUseListSavedViews.mockReset();
});
it('fetches the list for the data source only in saved view mode', () => {
mockViews([]);
renderHook(() =>
usePreferenceSync({
mode: PreferenceMode.DIRECT,
dataSource: DataSource.LOGS,
savedViewId: undefined,
}),
);
expect(mockedUseListSavedViews).toHaveBeenCalledWith(
{ source: 'logs' },
{ query: { enabled: false } },
);
});
it('returns loader preferences outside saved view mode', () => {
mockViews([]);
const { result } = renderHook(() =>
usePreferenceSync({
mode: PreferenceMode.DIRECT,
dataSource: DataSource.LOGS,
savedViewId: undefined,
}),
);
expect(result.current.preferences).toBe(loaderPreferences);
});
it('applies selectedFields and display of the active logs view', () => {
mockViews([
makeView('view-1', SavedviewtypesSourceDTO.logs, {
selectedFields: [{ name: 'service.name' }, { name: 'body' }],
display: { maxLines: 3, format: 'raw', fontSize: 'large', color: 'red' },
}),
]);
const { result } = renderHook(() =>
usePreferenceSync({
mode: PreferenceMode.SAVED_VIEW,
dataSource: DataSource.LOGS,
savedViewId: 'view-1',
}),
);
expect(result.current.preferences?.columns).toStrictEqual(
ensureLogsRequiredColumns([{ name: 'service.name' }, { name: 'body' }]),
);
expect(result.current.preferences?.formatting).toStrictEqual({
maxLines: 3,
format: 'raw',
fontSize: 'large',
version: 1,
});
});
it('falls back to defaults when the view has zero-valued display and no fields', () => {
mockViews([
makeView('view-1', SavedviewtypesSourceDTO.logs, {
selectedFields: undefined,
display: { maxLines: 0, format: '', fontSize: '', color: '' },
}),
]);
const { result } = renderHook(() =>
usePreferenceSync({
mode: PreferenceMode.SAVED_VIEW,
dataSource: DataSource.LOGS,
savedViewId: 'view-1',
}),
);
expect(result.current.preferences?.columns).toStrictEqual(
ensureLogsRequiredColumns(defaultLogsSelectedColumns),
);
expect(result.current.preferences?.formatting).toStrictEqual({
maxLines: 1,
format: 'table',
fontSize: 'small',
version: 1,
});
});
it('passes trace selectedFields through and defaults when absent', () => {
mockViews([
makeView('with-fields', SavedviewtypesSourceDTO.traces, {
selectedFields: [{ name: 'name' }, { name: 'durationNano' }],
}),
makeView('without-fields', SavedviewtypesSourceDTO.traces, {}),
]);
const withFields = renderHook(() =>
usePreferenceSync({
mode: PreferenceMode.SAVED_VIEW,
dataSource: DataSource.TRACES,
savedViewId: 'with-fields',
}),
);
const withoutFields = renderHook(() =>
usePreferenceSync({
mode: PreferenceMode.SAVED_VIEW,
dataSource: DataSource.TRACES,
savedViewId: 'without-fields',
}),
);
expect(withFields.result.current.preferences?.columns).toStrictEqual([
{ name: 'name' },
{ name: 'durationNano' },
]);
expect(withFields.result.current.preferences?.formatting).toBeUndefined();
expect(withoutFields.result.current.preferences?.columns).toBe(
defaultTraceSelectedColumns,
);
});
it('uses defaults when the saved view id is not in the list', () => {
mockViews([makeView('other', SavedviewtypesSourceDTO.logs, {})]);
const { result } = renderHook(() =>
usePreferenceSync({
mode: PreferenceMode.SAVED_VIEW,
dataSource: DataSource.LOGS,
savedViewId: 'missing',
}),
);
expect(result.current.preferences?.columns).toStrictEqual(
ensureLogsRequiredColumns(defaultLogsSelectedColumns),
);
});
});

View File

@@ -1,14 +1,12 @@
/* eslint-disable sonarjs/cognitive-complexity */
import { useEffect, useState } from 'react';
import { useListSavedViews } from 'api/generated/services/saved-view';
import { TelemetryFieldKey } from 'api/v5/v5';
import {
defaultLogsSelectedColumns,
defaultTraceSelectedColumns,
ensureLogsRequiredColumns,
} from 'container/OptionsMenu/constants';
import { FontSize, LogViewMode } from 'container/OptionsMenu/types';
import { findSavedView, toSavedViewSource } from 'container/SavedViews/utils';
import { defaultSelectedColumns as defaultTracesSelectedColumns } from 'container/TracesExplorer/ListView/configs';
import { useGetAllViews } from 'hooks/saveViews/useGetAllViews';
import { DataSource } from 'types/common/queryBuilder';
import { usePreferenceLoader } from '../loader/usePreferenceLoader';
@@ -30,16 +28,16 @@ export function usePreferenceSync({
updateColumns: (newColumns: TelemetryFieldKey[]) => void;
updateFormatting: (newFormatting: FormattingOptions) => void;
} {
const { data: viewsData } = useListSavedViews(
{ source: toSavedViewSource(dataSource) },
{ query: { enabled: mode === PreferenceMode.SAVED_VIEW } },
const { data: viewsData } = useGetAllViews(
dataSource,
mode === PreferenceMode.SAVED_VIEW,
);
const [savedViewPreferences, setSavedViewPreferences] =
useState<Preferences | null>(null);
const withColumnNames = (
columns: TelemetryFieldKey[] | undefined,
const updateExtraDataSelectColumns = (
columns: TelemetryFieldKey[],
): TelemetryFieldKey[] | null => {
if (!columns) {
return null;
@@ -51,28 +49,27 @@ export function usePreferenceSync({
};
useEffect(() => {
const spec = savedViewId
? findSavedView(viewsData?.data, savedViewId)?.spec
: undefined;
const selectedFields = spec?.selectedFields as
| TelemetryFieldKey[]
| undefined;
const extraData = viewsData?.data?.data?.find(
(view) => view.id === savedViewId,
)?.extraData;
const parsedExtraData = JSON.parse(extraData || '{}');
let columns: TelemetryFieldKey[] = [];
let formatting: FormattingOptions | undefined;
if (dataSource === DataSource.LOGS) {
columns = ensureLogsRequiredColumns(
withColumnNames(selectedFields) || defaultLogsSelectedColumns,
updateExtraDataSelectColumns(parsedExtraData?.selectColumns) ||
defaultLogsSelectedColumns,
);
formatting = {
maxLines: spec?.display?.maxLines || 1,
format: (spec?.display?.format as LogViewMode) || 'table',
fontSize: (spec?.display?.fontSize as FontSize) || FontSize.SMALL,
version: 1,
maxLines: parsedExtraData?.maxLines ?? 1,
format: parsedExtraData?.format ?? 'table',
fontSize: parsedExtraData?.fontSize ?? 'small',
version: parsedExtraData?.version ?? 1,
};
}
if (dataSource === DataSource.TRACES) {
columns = selectedFields || defaultTraceSelectedColumns;
columns = parsedExtraData?.selectColumns || defaultTracesSelectedColumns;
}
setSavedViewPreferences({ columns, formatting });
}, [viewsData, dataSource, savedViewId, mode]);

View File

@@ -10,11 +10,11 @@ import (
)
func (provider *provider) addPromoteRoutes(router *mux.Router) error {
if err := router.Handle("/api/v1/logs/promote_paths", handler.New(provider.authzMiddleware.EditAccess(provider.promoteHandler.HandlePromoteAndIndexPaths), handler.OpenAPIDef{
ID: "HandlePromoteAndIndexPaths",
Tags: []string{"logs"},
Summary: "Promote and index paths",
Description: "This endpoints promotes and indexes paths",
if err := router.Handle("/api/v1/promote_paths/{telemetry_signal}/{context}", handler.New(provider.authzMiddleware.EditAccess(provider.promoteHandler.PromotePaths), handler.OpenAPIDef{
ID: "PromotePaths",
Tags: []string{"promote"},
Summary: "Promote paths",
Description: "This endpoint promotes paths of a JSON column to its promoted column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.",
Request: new([]*promotetypes.PromotePath),
RequestContentType: "application/json",
Response: nil,
@@ -26,11 +26,11 @@ func (provider *provider) addPromoteRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/logs/promote_paths", handler.New(provider.authzMiddleware.ViewAccess(provider.promoteHandler.ListPromotedAndIndexedPaths), handler.OpenAPIDef{
ID: "ListPromotedAndIndexedPaths",
Tags: []string{"logs"},
Summary: "Promote and index paths",
Description: "This endpoints promotes and indexes paths",
if err := router.Handle("/api/v1/promote_paths/{telemetry_signal}/{context}", handler.New(provider.authzMiddleware.ViewAccess(provider.promoteHandler.ListPromotedPaths), handler.OpenAPIDef{
ID: "ListPromotedPaths",
Tags: []string{"promote"},
Summary: "List promoted paths",
Description: "This endpoint lists the promoted paths of a JSON column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.",
Request: nil,
RequestContentType: "",
Response: new([]*promotetypes.PromotePath),

View File

@@ -11,6 +11,7 @@ var (
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureEnableMetricsReduction = featuretypes.MustNewName("enable_metrics_reduction")
FeatureResolveSemconvFamilies = featuretypes.MustNewName("resolve_semconv_families")
FeatureUseTraceAttributesJSON = featuretypes.MustNewName("use_trace_attributes_json")
)
func MustNewRegistry() featuretypes.Registry {
@@ -79,6 +80,14 @@ func MustNewRegistry() featuretypes.Registry {
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureUseTraceAttributesJSON,
Kind: featuretypes.KindBoolean,
Stage: featuretypes.StageExperimental,
Description: "Controls whether trace queries read span attributes from the JSON columns",
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
)
if err != nil {
panic(err)

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/promotetypes"
"github.com/gorilla/mux"
)
type handler struct {
@@ -19,9 +20,16 @@ func NewHandler(module promote.Module) promote.Handler {
return &handler{module: module}
}
func (h *handler) HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Request) {
func (h *handler) PromotePaths(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
target, err := promotetypes.NewTargetFromPath(vars["telemetry_signal"], vars["context"])
if err != nil {
render.Error(w, err)
return
}
// TODO(Nitya): Use in multi tenant setup
_, err := authtypes.ClaimsFromContext(r.Context())
_, err = authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, errors.NewInternalf(errors.CodeInternal, "failed to get org id from context"))
return
@@ -33,7 +41,7 @@ func (h *handler) HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Requ
return
}
err = h.module.PromoteAndIndexPaths(r.Context(), req...)
err = h.module.PromotePaths(r.Context(), target, req...)
if err != nil {
render.Error(w, err)
return
@@ -42,15 +50,22 @@ func (h *handler) HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Requ
render.Success(w, http.StatusCreated, nil)
}
func (h *handler) ListPromotedAndIndexedPaths(w http.ResponseWriter, r *http.Request) {
func (h *handler) ListPromotedPaths(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
target, err := promotetypes.NewTargetFromPath(vars["telemetry_signal"], vars["context"])
if err != nil {
render.Error(w, err)
return
}
// TODO(Nitya): Use in multi tenant setup
_, err := authtypes.ClaimsFromContext(r.Context())
_, err = authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, errors.NewInternalf(errors.CodeInternal, "failed to get org id from context"))
return
}
paths, err := h.module.ListPromotedAndIndexedPaths(r.Context())
paths, err := h.module.ListPromotedPaths(r.Context(), target)
if err != nil {
render.Error(w, err)
return

View File

@@ -2,14 +2,11 @@ package implpromote
import (
"context"
"maps"
"slices"
"strings"
schemamigrator "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
@@ -31,45 +28,57 @@ func NewModule(metadataStore telemetrytypes.MetadataStore, telemetrystore teleme
return &module{metadataStore: metadataStore, telemetryStore: telemetrystore}
}
func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetypes.PromotePath, error) {
// ListPromotedPaths lists the promoted paths of the target JSON column,
// merged with per-path index metadata where the target supports indexes.
func (m *module) ListPromotedPaths(ctx context.Context, target promotetypes.Target) ([]promotetypes.PromotePath, error) {
promotedPaths, err := m.metadataStore.GetPromotedPaths(ctx, target.Entry)
if err != nil {
return nil, err
}
response := make([]promotetypes.PromotePath, 0, len(promotedPaths))
for path := range promotedPaths {
response = append(response, promotetypes.PromotePath{
Path: target.RequiredPathPrefix + path,
Promote: true,
})
}
// Index metadata is optional per target; merge it in only where supported.
if !target.IndexesSupported {
return response, nil
}
indexes, err := m.metadataStore.ListLogsJSONIndexes(ctx)
if err != nil {
return nil, err
}
// index.Name is the bare path and index.BaseColumn carries the column
// prefix, so the aggregate key is the full sub-column path.
aggr := map[string][]promotetypes.WrappedIndex{}
for _, index := range indexes {
aggr[index.Name] = append(aggr[index.Name], promotetypes.WrappedIndex{
fullPath := index.BaseColumn + index.Name
aggr[fullPath] = append(aggr[fullPath], promotetypes.WrappedIndex{
FieldDataType: index.FieldDataType,
Type: index.IndexType,
Granularity: index.Granularity,
})
}
promotedPaths, err := m.listPromotedPaths(ctx)
if err != nil {
return nil, err
}
response := []promotetypes.PromotePath{}
for _, path := range promotedPaths {
fullPath := logstelemetryschema.BodyPromotedColumnPrefix + path
path = telemetrytypes.BodyJSONStringSearchPrefix + path
item := promotetypes.PromotePath{
Path: path,
Promote: true,
}
indexes, ok := aggr[fullPath]
if ok {
item.Indexes = indexes
for i := range response {
fullPath := target.PromotedColumnPrefix() + strings.TrimPrefix(response[i].Path, target.RequiredPathPrefix)
if indexes, ok := aggr[fullPath]; ok {
response[i].Indexes = indexes
delete(aggr, fullPath)
}
response = append(response, item)
}
// add the paths that are not promoted but have indexes
for path, indexes := range aggr {
path := strings.TrimPrefix(path, logstelemetryschema.BodyV2ColumnPrefix)
path = telemetrytypes.BodyJSONStringSearchPrefix + path
for fullPath, indexes := range aggr {
path := strings.TrimPrefix(fullPath, target.BaseColumnPrefix())
path = strings.TrimPrefix(path, target.PromotedColumnPrefix())
path = target.RequiredPathPrefix + path
response = append(response, promotetypes.PromotePath{
Path: path,
Indexes: indexes,
@@ -78,54 +87,10 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
return response, nil
}
func (m *module) listPromotedPaths(ctx context.Context) ([]string, error) {
paths, err := m.metadataStore.GetPromotedPaths(ctx)
if err != nil {
return nil, err
}
return slices.Collect(maps.Keys(paths)), nil
}
// PromotePaths inserts provided JSON paths into the promoted paths table for logs queries.
func (m *module) PromotePaths(ctx context.Context, paths []string) error {
if len(paths) == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "paths cannot be empty")
}
return m.metadataStore.PromotePaths(ctx, paths...)
}
// createIndexes creates string ngram + token filter indexes on JSON path subcolumns for LIKE queries.
func (m *module) createIndexes(ctx context.Context, indexes []schemamigrator.Index) error {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalLogs.StringValue(),
instrumentationtypes.CodeNamespace: "promote",
instrumentationtypes.CodeFunctionName: "createIndexes",
})
if len(indexes) == 0 {
return nil
}
for _, index := range indexes {
alterStmt := schemamigrator.AlterTableAddIndex{
Database: logstelemetryschema.DBName,
Table: logstelemetryschema.LogsV2LocalTableName,
Index: index,
}
op := alterStmt.OnCluster(m.telemetryStore.Cluster())
if err := m.telemetryStore.ClickhouseDB().Exec(ctx, op.ToSQL()); err != nil {
return errors.WrapInternalf(err, CodeFailedToCreateIndex, "failed to create index")
}
}
return nil
}
// PromoteAndIndexPaths handles promoting paths and creating indexes in one call.
func (m *module) PromoteAndIndexPaths(
ctx context.Context,
paths ...*promotetypes.PromotePath,
) error {
// PromotePaths records new promotions of the target JSON column in the column
// evolution table and, for targets with index support, creates the requested
// per-path skip indexes.
func (m *module) PromotePaths(ctx context.Context, target promotetypes.Target, paths ...*promotetypes.PromotePath) error {
if len(paths) == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "paths cannot be empty")
}
@@ -133,13 +98,13 @@ func (m *module) PromoteAndIndexPaths(
pathsStr := []string{}
// validate the paths
for _, path := range paths {
if err := path.ValidateAndSetDefaults(); err != nil {
if err := path.ValidateAndSetDefaults(target); err != nil {
return err
}
pathsStr = append(pathsStr, path.Path)
}
existingPromotedPaths, err := m.metadataStore.GetPromotedPaths(ctx, pathsStr...)
existingPromotedPaths, err := m.metadataStore.GetPromotedPaths(ctx, target.Entry, pathsStr...)
if err != nil {
return err
}
@@ -153,10 +118,10 @@ func (m *module) PromoteAndIndexPaths(
}
}
if len(it.Indexes) > 0 {
parentColumn := logstelemetryschema.LogsV2BodyV2Column
parentColumn := target.BaseColumn
// if the path is already promoted or is being promoted, add it to the promoted column
if _, promoted := existingPromotedPaths[it.Path]; promoted || it.Promote {
parentColumn = logstelemetryschema.LogsV2BodyPromotedColumn
parentColumn = target.PromotedColumn()
}
for _, index := range it.Indexes {
@@ -182,17 +147,43 @@ func (m *module) PromoteAndIndexPaths(
}
if len(toInsert) > 0 {
err := m.PromotePaths(ctx, toInsert)
err := m.metadataStore.PromotePaths(ctx, target.Entry, toInsert...)
if err != nil {
return err
}
}
if len(indexes) > 0 {
if err := m.createIndexes(ctx, indexes); err != nil {
if err := m.createIndexes(ctx, target, indexes); err != nil {
return err
}
}
return nil
}
// createIndexes creates string ngram + token filter indexes on JSON path subcolumns for LIKE queries.
func (m *module) createIndexes(ctx context.Context, target promotetypes.Target, indexes []schemamigrator.Index) error {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: target.Entry.Signal.StringValue(),
instrumentationtypes.CodeNamespace: "promote",
instrumentationtypes.CodeFunctionName: "createIndexes",
})
if len(indexes) == 0 {
return nil
}
for _, index := range indexes {
alterStmt := schemamigrator.AlterTableAddIndex{
Database: target.DBName,
Table: target.LocalTableName,
Index: index,
}
op := alterStmt.OnCluster(m.telemetryStore.Cluster())
if err := m.telemetryStore.ClickhouseDB().Exec(ctx, op.ToSQL()); err != nil {
return errors.WrapInternalf(err, CodeFailedToCreateIndex, "failed to create index")
}
}
return nil
}

View File

@@ -0,0 +1,234 @@
package implpromote
import (
"context"
"regexp"
"testing"
sqlmock "github.com/DATA-DOG/go-sqlmock"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
"github.com/SigNoz/signoz/pkg/types/promotetypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestPromotePaths(t *testing.T) {
ctx := context.Background()
testCases := []struct {
name string
target promotetypes.Target
paths []*promotetypes.PromotePath
promoteTwice bool
wantErr bool
wantPromoted []string
}{
{
name: "PromotesNewAttributes_Idempotent",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{
{Path: "http.method", Promote: true},
{Path: "span.operation", Promote: true},
},
promoteTwice: true,
wantPromoted: []string{"http.method", "span.operation"},
},
{
name: "NonPromoteEntries_NotRecorded",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{{Path: "http.method"}},
},
{
name: "ColumnPrefixedPath_Rejected",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{{Path: "attributes.http.method", Promote: true}},
wantErr: true,
},
{
name: "EmptyPath_Rejected",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{{Path: "", Promote: true}},
wantErr: true,
},
{
name: "EmptyRequest_Rejected",
target: promotetypes.NewTracesAttributesTarget(),
wantErr: true,
},
{
name: "PromotesBodyPath_PrefixStripped",
target: promotetypes.NewLogsBodyTarget(),
paths: []*promotetypes.PromotePath{{Path: "body.user.name", Promote: true}},
wantPromoted: []string{"user.name"},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
store := telemetrytypestest.NewMockMetadataStore()
m := NewModule(store, nil)
err := m.PromotePaths(ctx, testCase.target, testCase.paths...)
if testCase.wantErr {
assert.Error(t, err)
assert.Empty(t, store.PromotedPathsMap)
return
}
require.NoError(t, err)
require.Len(t, store.PromotedPathsMap, len(testCase.wantPromoted))
for _, path := range testCase.wantPromoted {
assert.True(t, store.PromotedPathsMap[path], path)
}
if testCase.promoteTwice {
// promoting again must not fail
require.NoError(t, m.PromotePaths(ctx, testCase.target, testCase.paths...))
assert.Len(t, store.PromotedPathsMap, len(testCase.wantPromoted))
}
})
}
}
func TestPromotePathsCreatesIndexes(t *testing.T) {
ctx := context.Background()
testCases := []struct {
name string
promoted map[string]bool
path *promotetypes.PromotePath
wantDDLColumn string
}{
{
name: "NewPromotion_IndexesPromotedColumn",
path: &promotetypes.PromotePath{
Path: "body.user.name",
Promote: true,
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantDDLColumn: "dynamicElement(body_promoted.user.name",
},
{
name: "AlreadyPromoted_IndexesPromotedColumn",
promoted: map[string]bool{"user.name": true},
path: &promotetypes.PromotePath{
Path: "body.user.name",
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantDDLColumn: "dynamicElement(body_promoted.user.name",
},
{
name: "UnpromotedPath_IndexesBaseColumn",
path: &promotetypes.PromotePath{
Path: "body.user.name",
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantDDLColumn: "dynamicElement(body_v2.user.name",
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
ts := telemetrystoretest.New(telemetrystore.Config{}, sqlmock.QueryMatcherRegexp)
store := telemetrytypestest.NewMockMetadataStore()
if testCase.promoted != nil {
store.PromotedPathsMap = testCase.promoted
}
m := NewModule(store, ts)
ts.Mock().ExpectExec("ADD INDEX (.+)" + regexp.QuoteMeta(testCase.wantDDLColumn)).WillReturnError(nil)
require.NoError(t, m.PromotePaths(ctx, promotetypes.NewLogsBodyTarget(), testCase.path))
assert.NoError(t, ts.Mock().ExpectationsWereMet())
})
}
}
func TestListPromotedPaths(t *testing.T) {
ctx := context.Background()
testCases := []struct {
name string
target promotetypes.Target
promoted map[string]bool
indexes []telemetrytypes.TelemetryFieldKeySkipIndex
wantPaths []promotetypes.PromotePath
}{
{
name: "TracesAttributes_PromotedPaths",
target: promotetypes.NewTracesAttributesTarget(),
promoted: map[string]bool{"http.method": true},
wantPaths: []promotetypes.PromotePath{
{Path: "http.method", Promote: true},
},
},
{
name: "LogsBody_PromotedAndIndexedPaths",
target: promotetypes.NewLogsBodyTarget(),
promoted: map[string]bool{"user.name": true},
indexes: []telemetrytypes.TelemetryFieldKeySkipIndex{
{
Name: "user.name",
FieldContext: telemetrytypes.FieldContextBody,
FieldDataType: telemetrytypes.FieldDataTypeString,
BaseColumn: "body_promoted.",
IndexType: "ngrambf_v1(4, 1024, 2, 0)",
Granularity: 1,
},
{
Name: "request.duration",
FieldContext: telemetrytypes.FieldContextBody,
FieldDataType: telemetrytypes.FieldDataTypeFloat64,
BaseColumn: "body_v2.",
IndexType: "minmax",
Granularity: 1,
},
},
wantPaths: []promotetypes.PromotePath{
{
Path: "body.user.name",
Promote: true,
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
{
Path: "body.request.duration",
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeFloat64, Type: "minmax", Granularity: 1},
},
},
},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
store := telemetrytypestest.NewMockMetadataStore()
store.PromotedPathsMap = testCase.promoted
store.LogsJSONIndexes = testCase.indexes
m := NewModule(store, nil)
paths, err := m.ListPromotedPaths(ctx, testCase.target)
require.NoError(t, err)
require.Len(t, paths, len(testCase.wantPaths))
byPath := map[string]promotetypes.PromotePath{}
for _, path := range paths {
byPath[path.Path] = path
}
for _, want := range testCase.wantPaths {
require.Contains(t, byPath, want.Path)
assert.Equal(t, want, byPath[want.Path])
}
})
}
}

View File

@@ -8,11 +8,11 @@ import (
)
type Module interface {
ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetypes.PromotePath, error)
PromoteAndIndexPaths(ctx context.Context, paths ...*promotetypes.PromotePath) error
ListPromotedPaths(ctx context.Context, target promotetypes.Target) ([]promotetypes.PromotePath, error)
PromotePaths(ctx context.Context, target promotetypes.Target, paths ...*promotetypes.PromotePath) error
}
type Handler interface {
HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Request)
ListPromotedAndIndexedPaths(w http.ResponseWriter, r *http.Request)
PromotePaths(w http.ResponseWriter, r *http.Request)
ListPromotedPaths(w http.ResponseWriter, r *http.Request)
}

View File

@@ -24,7 +24,9 @@ func NewQueryInfo(ctx context.Context, orgID valuer.UUID, fl flagger.Flagger, si
FamiliesOn: semconvFamiliesEnabled(ctx, orgID, fl),
}
if fl != nil {
q.BodyJSONOn = fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID))
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)
q.BodyJSONOn = fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, evalCtx)
q.TraceAttrsJSONOn = fl.BooleanOrEmpty(ctx, flagger.FeatureUseTraceAttributesJSON, evalCtx)
}
return q
}

View File

@@ -7,6 +7,7 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
@@ -24,7 +25,7 @@ var jsonAttrColRe = regexp.MustCompile(`,\s*attributes\s*(,| FROM )`)
func newBulkTestBuilder(t *testing.T, releaseTime time.Time) *traceQueryStatementBuilder {
t.Helper()
fl := flaggertest.New(t)
fl := flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureUseTraceAttributesJSON.String(): true})
storage := tracestelemetryschema.NewStorage()
store := telemetrytypestest.NewMockMetadataStore()
store.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)

View File

@@ -16,6 +16,7 @@ import (
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/types/promotetypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/huandu/go-sqlbuilder"
)
@@ -33,6 +34,10 @@ var (
CodeFailedToAppendPath = errors.MustNewCode("failed_to_append_path_promoted_paths")
)
// logsBodyPromotedEntry templates the column evolution rows recorded for
// logs body promotions.
var logsBodyPromotedEntry = promotetypes.NewLogsBodyTarget().Entry
// enrichJSONKeys enriches body-context keys with promoted path info, indexes,
// and JSON access plans. parentTypeCache contains parent array types (ArrayJSON/ArrayDynamic)
// pre-fetched in the main UNION query.
@@ -67,7 +72,7 @@ func (t *telemetryMetaStore) enrichJSONKeys(ctx context.Context, selectors []*te
}
// fetch promoted paths
promoted, err := t.GetPromotedPaths(ctx, paths...)
promoted, err := t.GetPromotedPaths(ctx, logsBodyPromotedEntry, paths...)
if err != nil {
return err
}
@@ -157,7 +162,7 @@ func buildListLogsJSONIndexesQuery(cluster string, filters ...string) (string, [
}
func (t *telemetryMetaStore) ListLogsJSONIndexes(ctx context.Context, filters ...string) ([]telemetrytypes.TelemetryFieldKeySkipIndex, error) {
ctx = withTelemetryContext(ctx, "ListLogsJSONIndexes")
ctx = withTelemetryContext(ctx, telemetrytypes.SignalLogs, "ListLogsJSONIndexes")
query, args := buildListLogsJSONIndexesQuery(t.telemetrystore.Cluster(), filters...)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
@@ -215,14 +220,14 @@ func (t *telemetryMetaStore) ListLogsJSONIndexes(ctx context.Context, filters ..
// TODO(Piyush): Remove this if not used in future.
func (t *telemetryMetaStore) ListJSONValues(ctx context.Context, path string, limit int) (*telemetrytypes.TelemetryFieldValues, bool, error) {
ctx = withTelemetryContext(ctx, "ListJSONValues")
ctx = withTelemetryContext(ctx, telemetrytypes.SignalLogs, "ListJSONValues")
path = CleanPathPrefixes(path)
if strings.Contains(path, telemetrytypes.ArraySep) || strings.Contains(path, telemetrytypes.ArrayAnyIndex) {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput, "array paths are not supported")
}
promoted, err := t.IsPathPromoted(ctx, path)
promoted, err := t.isPathPromoted(ctx, logsBodyPromotedEntry, path)
if err != nil {
return nil, false, err
}
@@ -376,13 +381,13 @@ func derefValue(v any) any {
return val.Interface()
}
// IsPathPromoted checks if a specific path is promoted (Column Evolution table: field_name for logs body).
func (t *telemetryMetaStore) IsPathPromoted(ctx context.Context, path string) (bool, error) {
ctx = withTelemetryContext(ctx, "IsPathPromoted")
// isPathPromoted checks if a specific path is promoted (Column Evolution table: field_name for the entry's column).
func (t *telemetryMetaStore) isPathPromoted(ctx context.Context, entry telemetrytypes.EvolutionEntry, path string) (bool, error) {
ctx = withTelemetryContext(ctx, entry.Signal, "isPathPromoted")
split := strings.Split(path, telemetrytypes.ArraySep)
pathSegment := split[0]
query := fmt.Sprintf("SELECT 1 FROM %s.%s WHERE signal = ? AND column_name = ? AND field_context = ? AND field_name = ? LIMIT 1", DBName, PromotedPathsTableName)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, telemetrytypes.SignalLogs, logstelemetryschema.LogsV2BodyPromotedColumn, telemetrytypes.FieldContextBody, pathSegment)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, entry.Signal, entry.ColumnName, entry.FieldContext, pathSegment)
if err != nil {
return false, errors.WrapInternalf(err, CodeFailCheckPathPromoted, "failed to check if path %s is promoted", path)
}
@@ -391,14 +396,14 @@ func (t *telemetryMetaStore) IsPathPromoted(ctx context.Context, path string) (b
return rows.Next(), nil
}
// GetPromotedPaths returns promoted paths from the Column Evolution table (field_name for logs body).
func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error) {
ctx = withTelemetryContext(ctx, "GetPromotedPaths")
// GetPromotedPaths returns promoted paths from the Column Evolution table (field_name for the entry's column).
func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, entry telemetrytypes.EvolutionEntry, paths ...string) (map[string]bool, error) {
ctx = withTelemetryContext(ctx, entry.Signal, "GetPromotedPaths")
sb := sqlbuilder.Select("field_name").From(fmt.Sprintf("%s.%s", DBName, PromotedPathsTableName))
conditions := []string{
sb.Equal("signal", telemetrytypes.SignalLogs),
sb.Equal("column_name", logstelemetryschema.LogsV2BodyPromotedColumn),
sb.Equal("field_context", telemetrytypes.FieldContextBody),
sb.Equal("signal", entry.Signal),
sb.Equal("column_name", entry.ColumnName),
sb.Equal("field_context", entry.FieldContext),
sb.NotEqual("field_name", "__all__"),
}
if len(paths) > 0 {
@@ -438,9 +443,10 @@ func CleanPathPrefixes(path string) string {
return path
}
// PromotePaths inserts promoted paths into the Column Evolution table (same schema as signoz-otel-collector metadata_migrations).
func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string) error {
ctx = withTelemetryContext(ctx, "PromotePaths")
// PromotePaths inserts promoted paths into the Column Evolution table as rows templated by entry
// (same schema as signoz-otel-collector metadata_migrations); FieldName and ReleaseTime are set per path.
func (t *telemetryMetaStore) PromotePaths(ctx context.Context, entry telemetrytypes.EvolutionEntry, paths ...string) error {
ctx = withTelemetryContext(ctx, entry.Signal, "PromotePaths")
batch, err := t.telemetrystore.ClickhouseDB().PrepareBatch(ctx,
fmt.Sprintf("INSERT INTO %s.%s (signal, column_name, column_type, field_context, field_name, version, release_time) VALUES", DBName,
PromotedPathsTableName))
@@ -454,7 +460,7 @@ func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string)
if trimmed == "" {
continue
}
if err := batch.Append(telemetrytypes.SignalLogs, logstelemetryschema.LogsV2BodyPromotedColumn, "JSON()", telemetrytypes.FieldContextBody, trimmed, 0, releaseTime); err != nil {
if err := batch.Append(entry.Signal, entry.ColumnName, entry.ColumnType, entry.FieldContext, trimmed, entry.Version, releaseTime); err != nil {
_ = batch.Abort()
return errors.WrapInternalf(err, CodeFailedToAppendPath, "failed to append path")
}
@@ -466,9 +472,9 @@ func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string)
return nil
}
func withTelemetryContext(ctx context.Context, functionName string) context.Context {
func withTelemetryContext(ctx context.Context, signal telemetrytypes.Signal, functionName string) context.Context {
return ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalLogs.StringValue(),
instrumentationtypes.TelemetrySignal: signal.StringValue(),
instrumentationtypes.CodeNamespace: "metadata",
instrumentationtypes.CodeFunctionName: functionName,
})

View File

@@ -172,7 +172,7 @@ func NewStorage() qbtypes.Storage {
func (m *storage) getColumn(
_ context.Context,
_, _ uint64,
q qbtypes.QueryInfo,
key *telemetrytypes.TelemetryFieldKey,
) ([]*schema.Column, error) {
switch key.FieldContext {
@@ -194,8 +194,8 @@ func (m *storage) getColumn(
default:
return nil, qbtypes.ErrColumnNotFound
}
// The `attributes` evolution entry is the rollout control.
if attributeColumnEvolutionRegistered(key, SpanAttributesColumn) {
// The use_trace_attributes_json flag and the `attributes` evolution entry are the rollout control.
if q.TraceAttrsJSONOn && attributeColumnEvolutionRegistered(key, SpanAttributesColumn) {
cols := make([]*schema.Column, 0, 3)
if attributeColumnEvolutionRegistered(key, SpanAttributesPromotedColumn) {
cols = append(cols, indexV3Columns["attributes_promoted"])
@@ -233,15 +233,15 @@ func (m *storage) getColumn(
// (after evolution selection); existExprs only carries guards for guardable column types.
func (m *storage) resolveColumnExprs(
ctx context.Context,
startNs, endNs uint64,
q qbtypes.QueryInfo,
key *telemetrytypes.TelemetryFieldKey,
) (exprs []string, existExprs []string, columns []*schema.Column, err error) {
columns, err = m.getColumn(ctx, startNs, endNs, key)
columns, err = m.getColumn(ctx, q, key)
if err != nil {
return nil, nil, nil, err
}
newColumns, evolutionsEntries, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
newColumns, evolutionsEntries, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, q.StartNs, q.EndNs)
if err != nil {
return nil, nil, nil, err
}
@@ -355,12 +355,12 @@ func attributeJSONValueExpr(path string, dataType telemetrytypes.FieldDataType)
// columnIsTemporal reports whether key resolves to a single time column, after evolution
// selection. Multiple columns mean an attribute-map union, which is never temporal.
func (m *storage) columnIsTemporal(ctx context.Context, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (bool, error) {
columns, err := m.getColumn(ctx, startNs, endNs, key)
func (m *storage) columnIsTemporal(ctx context.Context, q qbtypes.QueryInfo, key *telemetrytypes.TelemetryFieldKey) (bool, error) {
columns, err := m.getColumn(ctx, q, key)
if err != nil {
return false, err
}
newColumns, _, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
newColumns, _, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, q.StartNs, q.EndNs)
if err != nil {
return false, err
}
@@ -394,7 +394,7 @@ func (m *storage) read(ctx context.Context, q qbtypes.QueryInfo, key *telemetryt
return key.Name, nil
}
exprs, existExpr, columns, err := m.resolveColumnExprs(ctx, q.StartNs, q.EndNs, key)
exprs, existExpr, columns, err := m.resolveColumnExprs(ctx, q, key)
if err != nil {
return "", err
}
@@ -457,7 +457,7 @@ func (m *storage) Read(ctx context.Context, q qbtypes.QueryInfo, key *telemetryt
if isSpanSearchScopeField(key.Name) {
return qbtypes.Read{SQL: key.Name, Presence: "true", Absence: "false", WhenAbsent: qbtypes.AlwaysPresent}, nil
}
exprs, existExprs, columns, err := m.resolveColumnExprs(ctx, q.StartNs, q.EndNs, key)
exprs, existExprs, columns, err := m.resolveColumnExprs(ctx, q, key)
if err != nil {
return qbtypes.Read{}, err
}
@@ -469,7 +469,7 @@ func (m *storage) Read(ctx context.Context, q qbtypes.QueryInfo, key *telemetryt
if err != nil {
return qbtypes.Read{}, err
}
temporal, err := m.columnIsTemporal(ctx, q.StartNs, q.EndNs, key)
temporal, err := m.columnIsTemporal(ctx, q, key)
if err != nil {
return qbtypes.Read{}, err
}
@@ -532,18 +532,18 @@ func foldAbsentJSONReadToTypeDefault(key *telemetrytypes.TelemetryFieldKey, oper
// and corrects to the attribute maps when it names no column. A strict
// context synthesizes its type variants under the stripped and the literal
// spelling.
func (m *storage) Fallback(ctx context.Context, _ qbtypes.QueryInfo, key *telemetrytypes.TelemetryFieldKey, _ qbtypes.FilterOperator, value any) ([]*telemetrytypes.LogicalField, error) {
func (m *storage) Fallback(ctx context.Context, q qbtypes.QueryInfo, key *telemetrytypes.TelemetryFieldKey, _ qbtypes.FilterOperator, value any) ([]*telemetrytypes.LogicalField, error) {
var keys []*telemetrytypes.TelemetryFieldKey
switch key.FieldContext {
case telemetrytypes.FieldContextUnspecified:
probe := telemetrytypes.NewTelemetryFieldKey(key.Name, telemetrytypes.FieldContextSpan, key.FieldDataType)
if columns, err := m.getColumn(ctx, 0, 0, probe); err == nil {
if columns, err := m.getColumn(ctx, q, probe); err == nil {
keys = []*telemetrytypes.TelemetryFieldKey{stampColumnType(probe, columns)}
} else {
keys = querybuilder.SynthesizeKeys(key, value)
}
case telemetrytypes.FieldContextSpan, telemetrytypes.FieldContextTrace:
if columns, err := m.getColumn(ctx, 0, 0, key); err == nil {
if columns, err := m.getColumn(ctx, q, key); err == nil {
column := telemetrytypes.NewTelemetryFieldKey(key.Name, key.FieldContext, key.FieldDataType)
keys = []*telemetrytypes.TelemetryFieldKey{stampColumnType(column, columns)}
} else {

View File

@@ -22,7 +22,7 @@ var (
)
func readSQL(ctx context.Context, storage qbtypes.Storage, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (string, error) {
read, err := storage.Read(ctx, qbtypes.QueryInfo{StartNs: startNs, EndNs: endNs}, key)
read, err := storage.Read(ctx, qbtypes.QueryInfo{StartNs: startNs, EndNs: endNs, TraceAttrsJSONOn: true}, key)
return read.SQL, err
}
@@ -96,6 +96,40 @@ func TestFieldForAttributeNoEvolutionParity(t *testing.T) {
}
}
// TestAttributeJSONFlagOffParity proves the evolution entry alone does not switch reads to the
// JSON column: with use_trace_attributes_json off, reads and conditions stay on the Map for every window.
func TestAttributeJSONFlagOffParity(t *testing.T) {
ctx := context.Background()
storage := NewStorage()
evo := MockAttributeEvolutionData(attrJSONRelease)
testCases := []struct {
name string
window [2]uint64
}{
{"BeforeRelease", attrWindowBefore},
{"AfterRelease", attrWindowAfter},
{"StraddlingRelease", attrWindowStraddle},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeNumber, evo)
q := qbtypes.QueryInfo{StartNs: testCase.window[0], EndNs: testCase.window[1]}
read, err := storage.Read(ctx, q, &key)
require.NoError(t, err)
assert.Equal(t, "attributes_number['user.id']", read.SQL)
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, q, storage, &key, qbtypes.FilterOperatorNotEqual, float64(1), map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
require.Len(t, conds, 1)
assert.NotContains(t, conds[0], "attributes.`user.id`")
})
}
}
// TestConditionForAttributeJSON asserts the emitted WHERE fragment per operator against the JSON
// column (window fully after release). Positive operators carry the raw-path existence guard;
// numeric comparisons keep numeric semantics; existence never tests the ::String cast.
@@ -170,7 +204,7 @@ func TestConditionForAttributeJSON(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &tc.key, tc.operator, tc.value, map[string][]*telemetrytypes.TelemetryFieldKey{tc.key.Name: {&tc.key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &tc.key, tc.operator, tc.value, map[string][]*telemetrytypes.TelemetryFieldKey{tc.key.Name: {&tc.key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -189,7 +223,7 @@ func TestConditionForAttributeJSONNotExistsDualRead(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1]}, storage, &key, qbtypes.FilterOperatorNotExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorNotExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -209,14 +243,14 @@ func TestColumnExpressionForAttributeJSON(t *testing.T) {
t.Run("group by string", func(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &key, telemetrytypes.FieldDataTypeString, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &key, telemetrytypes.FieldDataTypeString, nil)
require.NoError(t, err)
assert.Equal(t, "multiIf(attributes.`user.id` IS NOT NULL, attributes.`user.id`::String, mapContains(attributes_string, 'attribute.user.id'), attributes_string['attribute.user.id'], NULL)", got)
})
t.Run("aggregation numeric", func(t *testing.T) {
key := attrKey("latency", telemetrytypes.FieldDataTypeNumber, evo)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &key, telemetrytypes.FieldDataTypeFloat64, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &key, telemetrytypes.FieldDataTypeFloat64, nil)
require.NoError(t, err)
assert.Equal(t, "multiIf(if(dynamicType(attributes.`latency`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`latency`, 'Float64'), NULL) IS NOT NULL, toFloat64(if(dynamicType(attributes.`latency`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`latency`, 'Float64'), NULL)), mapContains(attributes_number, 'attribute.latency'), toFloat64(attributes_number['attribute.latency']), NULL)", got)
})
@@ -232,7 +266,7 @@ func TestAttributeJSONNoAmbiguityWarning(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
sb := sqlbuilder.NewSelectBuilder()
_, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &key, qbtypes.FilterOperatorEqual, "x", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
_, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorEqual, "x", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
assert.Empty(t, warnings, "a plain attribute filter must not emit an ambiguity warning")
}
@@ -254,7 +288,7 @@ func TestConditionForAttributeJSONTypeCollision(t *testing.T) {
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
sb := sqlbuilder.NewSelectBuilder()
conds, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
conds, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
require.NoError(t, err)
require.Len(t, conds, 2, "a colliding name must build one condition per data type")
@@ -283,7 +317,7 @@ func TestColumnExpressionForAttributeJSONTypeCollision(t *testing.T) {
}
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &ref, telemetrytypes.FieldDataTypeString, fieldKeys)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &ref, telemetrytypes.FieldDataTypeString, fieldKeys)
require.NoError(t, err)
assert.Equal(t,
"multiIf(attributes.`http.status_code` IS NOT NULL, attributes.`http.status_code`::String, if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL) IS NOT NULL, toString(if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL)), NULL)",
@@ -307,7 +341,7 @@ func TestColumnExpressionForAttributeJSONTypeCollisionNumericAgg(t *testing.T) {
}
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &ref, telemetrytypes.FieldDataTypeFloat64, fieldKeys)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &ref, telemetrytypes.FieldDataTypeFloat64, fieldKeys)
require.NoError(t, err)
assert.Equal(t,
"multiIf(if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL) IS NOT NULL, toFloat64(if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL)), attributes.`http.status_code` IS NOT NULL, toFloat64OrNull(attributes.`http.status_code`::String), NULL)",
@@ -330,7 +364,7 @@ func TestConditionForAttributeMapTypeCollisionParity(t *testing.T) {
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowBefore[0], EndNs: attrWindowBefore[1]}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowBefore[0], EndNs: attrWindowBefore[1], TraceAttrsJSONOn: true}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
require.NoError(t, err)
require.Len(t, conds, 2, "a colliding name must build one condition per data type")
@@ -351,7 +385,7 @@ func TestColumnForUnspecifiedAttributeNoBranchFlip(t *testing.T) {
evo := MockAttributeEvolutionData(attrJSONRelease)
key := attrKey("user.id", telemetrytypes.FieldDataTypeUnspecified, evo)
_, err := (&storage{}).getColumn(ctx, attrWindowAfter[0], attrWindowAfter[1], &key)
_, err := (&storage{}).getColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, &key)
assert.ErrorIs(t, err, qbtypes.ErrColumnNotFound)
}
@@ -369,7 +403,7 @@ func TestConditionForAttributeJSONNegativeOperatorParity(t *testing.T) {
build := func(t *testing.T, key telemetrytypes.TelemetryFieldKey, window [2]uint64, op qbtypes.FilterOperator, value any) string {
t.Helper()
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: window[0], EndNs: window[1]}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: window[0], EndNs: window[1], TraceAttrsJSONOn: true}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -449,7 +483,7 @@ func TestConditionForAttributeJSONStraddleAbsentKeyExclusion(t *testing.T) {
build := func(t *testing.T, key telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) string {
t.Helper()
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1]}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1], TraceAttrsJSONOn: true}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)

View File

@@ -81,7 +81,7 @@ func TestConditionForAttributePromoted(t *testing.T) {
t.Run("equal reads promoted column only", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1]}, storage, &key, qbtypes.FilterOperatorEqual, "GET", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorEqual, "GET", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -91,7 +91,7 @@ func TestConditionForAttributePromoted(t *testing.T) {
t.Run("exists uses promoted raw path", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1]}, storage, &key, qbtypes.FilterOperatorExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)

View File

@@ -0,0 +1,131 @@
package promotetypes
import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// Target identifies a promotion domain: the column evolution record written
// per promoted path, the table per-path indexes are created on, and the API
// path rules.
type Target struct {
Entry telemetrytypes.EvolutionEntry // evolution row template; FieldName and ReleaseTime are set per write
DBName string // index DDL database, used only when IndexesSupported
LocalTableName string // index DDL local table, used only when IndexesSupported
BaseColumn string // column holding every path; indexes for unpromoted paths are created on it
RequiredPathPrefix string // prefix API paths must carry, stripped before storing; empty for bare names
IndexesSupported bool // whether per-path skip indexes can be created for this domain
}
func (t Target) PromotedColumn() string { return t.Entry.ColumnName }
func (t Target) BaseColumnPrefix() string { return t.BaseColumn + "." }
func (t Target) PromotedColumnPrefix() string { return t.PromotedColumn() + "." }
// NewTarget creates the Target for a promotion domain.
func NewTarget(entry telemetrytypes.EvolutionEntry, dbName, localTableName, baseColumn, requiredPathPrefix string, indexesSupported bool) Target {
return Target{
Entry: entry,
DBName: dbName,
LocalTableName: localTableName,
BaseColumn: baseColumn,
RequiredPathPrefix: requiredPathPrefix,
IndexesSupported: indexesSupported,
}
}
// NewLogsBodyTarget returns the domain for the logs body JSON column
// (body_v2 -> body_promoted), with per-path skip index support.
func NewLogsBodyTarget() Target {
return NewTarget(
telemetrytypes.EvolutionEntry{
Signal: telemetrytypes.SignalLogs,
ColumnName: logstelemetryschema.LogsV2BodyPromotedColumn,
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextBody,
},
logstelemetryschema.DBName,
logstelemetryschema.LogsV2LocalTableName,
logstelemetryschema.LogsV2BodyV2Column,
telemetrytypes.BodyJSONStringSearchPrefix,
true,
)
}
// NewTracesAttributesTarget returns the domain for the spans attributes JSON
// column (attributes -> attributes_promoted); promotion only for now.
func NewTracesAttributesTarget() Target {
return NewTarget(
telemetrytypes.EvolutionEntry{
Signal: telemetrytypes.SignalTraces,
ColumnName: tracestelemetryschema.SpanAttributesPromotedColumn,
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextAttribute,
},
tracestelemetryschema.DBName,
tracestelemetryschema.SpanIndexV3LocalTableName,
tracestelemetryschema.SpanAttributesColumn,
"",
false,
)
}
// NewTargetFromPath validates the {telemetry_signal} and {context} path
// variables and returns their promotion domain.
func NewTargetFromPath(signal, context string) (Target, error) {
params := &PathParams{Signal: signal, Context: context}
if err := params.Validate(); err != nil {
return Target{}, err
}
parsedSignal, _ := telemetrytypes.SignalFromText(params.Signal)
parsedContext, _ := telemetrytypes.FieldContextFromText(params.Context)
target, _ := TargetFor(parsedSignal, parsedContext)
return target, nil
}
// Targets returns every supported promotion domain.
func Targets() []Target {
return []Target{
NewLogsBodyTarget(),
NewTracesAttributesTarget(),
}
}
// TargetFor resolves the domain for a (signal, context) pair; ok is false
// when no domain exists for the pair.
func TargetFor(signal telemetrytypes.Signal, context telemetrytypes.FieldContext) (Target, bool) {
for _, target := range Targets() {
if target.Entry.Signal.StringValue() == signal.StringValue() &&
target.Entry.FieldContext.StringValue() == context.StringValue() {
return target, true
}
}
return Target{}, false
}
// PathParams carries the raw {telemetry_signal} and {context} path variables
// of the promote paths API.
type PathParams struct {
Signal string
Context string
}
// Validate ensures the path variables are known values naming a supported
// promotion domain.
func (p *PathParams) Validate() error {
signal, ok := telemetrytypes.SignalFromText(p.Signal)
if !ok {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid signal: %s", p.Signal)
}
context, ok := telemetrytypes.FieldContextFromText(p.Context)
if !ok {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid context: %s", p.Context)
}
if _, ok := TargetFor(signal, context); !ok {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "promotion is not supported for %s %s", signal.StringValue(), context.StringValue())
}
return nil
}

View File

@@ -3,7 +3,6 @@ package promotetypes
import (
"strings"
"github.com/SigNoz/signoz-otel-collector/constants"
"github.com/SigNoz/signoz-otel-collector/pkg/keycheck"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -23,7 +22,7 @@ type PromotePath struct {
Indexes []WrappedIndex `json:"indexes,omitempty"`
}
func (i *PromotePath) ValidateAndSetDefaults() error {
func (i *PromotePath) ValidateAndSetDefaults(target Target) error {
if i.Path == "" {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "path is required")
}
@@ -36,22 +35,27 @@ func (i *PromotePath) ValidateAndSetDefaults() error {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "array paths can not be promoted or indexed")
}
if strings.HasPrefix(i.Path, constants.BodyV2ColumnPrefix) || strings.HasPrefix(i.Path, constants.BodyPromotedColumnPrefix) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "`%s`, `%s` don't add these prefixes to the path", constants.BodyV2ColumnPrefix, constants.BodyPromotedColumnPrefix)
if strings.HasPrefix(i.Path, target.BaseColumnPrefix()) || strings.HasPrefix(i.Path, target.PromotedColumnPrefix()) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "`%s`, `%s` don't add these prefixes to the path", target.BaseColumnPrefix(), target.PromotedColumnPrefix())
}
if !strings.HasPrefix(i.Path, telemetrytypes.BodyJSONStringSearchPrefix) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "path must start with `body.`")
if target.RequiredPathPrefix != "" {
if !strings.HasPrefix(i.Path, target.RequiredPathPrefix) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "path must start with `%s`", target.RequiredPathPrefix)
}
// remove the required prefix from the path
i.Path = strings.TrimPrefix(i.Path, target.RequiredPathPrefix)
}
// remove the "body." prefix from the path
i.Path = strings.TrimPrefix(i.Path, telemetrytypes.BodyJSONStringSearchPrefix)
isCardinal := keycheck.IsCardinal(i.Path)
if isCardinal {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "cardinal paths can not be promoted or indexed")
}
if len(i.Indexes) > 0 && !target.IndexesSupported {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "indexes are not supported for %s %s", target.Entry.Signal.StringValue(), target.Entry.FieldContext.StringValue())
}
for idx, index := range i.Indexes {
if index.Type == "" {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "index type is required")

View File

@@ -0,0 +1,172 @@
package promotetypes
import (
"testing"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestValidateAndSetDefaultsLogsBody(t *testing.T) {
target := NewLogsBodyTarget()
testCases := []struct {
name string
path *PromotePath
wantErr bool
wantPath string
wantJSONDataType telemetrytypes.JSONDataType
}{
{
name: "ValidPath_BodyPrefixStripped",
path: &PromotePath{Path: "body.user.name", Promote: true},
wantPath: "user.name",
},
{
name: "PathWithoutBodyPrefix_Rejected",
path: &PromotePath{Path: "user.name", Promote: true},
wantErr: true,
},
{
name: "BodyV2PrefixedPath_Rejected",
path: &PromotePath{Path: "body_v2.user.name", Promote: true},
wantErr: true,
},
{
name: "BodyPromotedPrefixedPath_Rejected",
path: &PromotePath{Path: "body_promoted.user.name", Promote: true},
wantErr: true,
},
{
name: "EmptyPath_Rejected",
path: &PromotePath{Path: "", Promote: true},
wantErr: true,
},
{
name: "SpacedPath_Rejected",
path: &PromotePath{Path: "body.my path", Promote: true},
wantErr: true,
},
{
name: "ArrayIndexPath_Rejected",
path: &PromotePath{Path: "body.users[].id", Promote: true},
wantErr: true,
},
{
name: "ArrayWildcardPath_Rejected",
path: &PromotePath{Path: "body.users[*].id", Promote: true},
wantErr: true,
},
{
name: "CardinalPath_Rejected",
path: &PromotePath{Path: "body.request.550e8400-e29b-41d4-a716-446655440000", Promote: true},
wantErr: true,
},
{
name: "ValidIndex_JSONDataTypeDefaulted",
path: &PromotePath{
Path: "body.user.name",
Indexes: []WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantPath: "user.name",
wantJSONDataType: telemetrytypes.String,
},
{
name: "UnsupportedColumnTypeIndex_Rejected",
path: &PromotePath{
Path: "body.user.active",
Indexes: []WrappedIndex{{FieldDataType: telemetrytypes.FieldDataTypeBool, Type: "minmax", Granularity: 1}},
},
wantErr: true,
},
{
name: "IndexWithoutType_Rejected",
path: &PromotePath{
Path: "body.user.name",
Indexes: []WrappedIndex{{FieldDataType: telemetrytypes.FieldDataTypeString, Granularity: 1}},
},
wantErr: true,
},
{
name: "IndexWithoutGranularity_Rejected",
path: &PromotePath{
Path: "body.user.name",
Indexes: []WrappedIndex{{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "minmax"}},
},
wantErr: true,
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := testCase.path.ValidateAndSetDefaults(target)
if testCase.wantErr {
assert.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, testCase.wantPath, testCase.path.Path)
if testCase.wantJSONDataType != (telemetrytypes.JSONDataType{}) {
require.Len(t, testCase.path.Indexes, 1)
assert.Equal(t, testCase.wantJSONDataType, testCase.path.Indexes[0].JSONDataType)
}
})
}
}
func TestValidateAndSetDefaultsTracesAttributes(t *testing.T) {
target := NewTracesAttributesTarget()
testCases := []struct {
name string
path *PromotePath
wantErr bool
wantPath string
}{
{
name: "BareAttributeName_KeptAsIs",
path: &PromotePath{Path: "http.method", Promote: true},
wantPath: "http.method",
},
{
name: "AttributesPrefixedPath_Rejected",
path: &PromotePath{Path: "attributes.http.method", Promote: true},
wantErr: true,
},
{
name: "AttributesPromotedPrefixedPath_Rejected",
path: &PromotePath{Path: "attributes_promoted.http.method", Promote: true},
wantErr: true,
},
{
name: "EmptyPath_Rejected",
path: &PromotePath{Path: "", Promote: true},
wantErr: true,
},
{
name: "SpacedPath_Rejected",
path: &PromotePath{Path: "my attr", Promote: true},
wantErr: true,
},
{
name: "ArrayIndexPath_Rejected",
path: &PromotePath{Path: "tags[].id", Promote: true},
wantErr: true,
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := testCase.path.ValidateAndSetDefaults(target)
if testCase.wantErr {
assert.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, testCase.wantPath, testCase.path.Path)
})
}
}

View File

@@ -13,6 +13,7 @@ import (
// SelectEvolutionsForColumns selects the appropriate evolution entries for each column based on the time range.
// Logic:
// - Ignores evolutions of columns outside the candidate columns
// - Finds the latest base evolution (<= tsStartTime) across ALL columns
// - Rejects all evolutions before this latest base evolution
// - For duplicate evolutions it considers the oldest one (first in ReleaseTime)
@@ -23,6 +24,11 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
return columns, nil, nil
}
columnLookUpMap := make(map[string]*schema.Column, len(columns))
for _, column := range columns {
columnLookUpMap[column.Name] = column
}
// Derive the base column from the candidate columns.
seen := make(map[string]struct{}, len(evolutions))
for _, e := range evolutions {
@@ -64,6 +70,9 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
if evolution.ReleaseTime.After(tsStartTime) {
break
}
if _, exists := columnLookUpMap[evolution.ColumnName]; !exists {
continue
}
latestBaseEvolutionAcrossAll = evolution
}
@@ -72,11 +81,6 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
return nil, nil, errors.Newf(errors.TypeInternal, errors.CodeInternal, "no base evolution found for columns %v", columns)
}
columnLookUpMap := make(map[string]*schema.Column)
for _, column := range columns {
columnLookUpMap[column.Name] = column
}
// Collect column-evolution pairs
type colEvoPair struct {
column *schema.Column
@@ -95,7 +99,7 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
}
if _, exists := columnLookUpMap[evolution.ColumnName]; !exists {
return nil, nil, errors.Newf(errors.TypeInternal, errors.CodeInternal, "evolution column %s not found in columns %v", evolution.ColumnName, columns)
continue
}
pairs = append(pairs, colEvoPair{columnLookUpMap[evolution.ColumnName], evolution})

View File

@@ -208,7 +208,7 @@ func TestSelectEvolutionsForColumns(t *testing.T) {
expectedColumns: []string{},
expectedEvols: []string{},
expectedError: true,
errorStr: "column resources_string not found",
errorStr: "no base evolution found",
},
{
name: "Duplicate evolutions - should use first encountered (oldest if sorted)",
@@ -441,6 +441,26 @@ func TestSelectEvolutionsForColumns(t *testing.T) {
expectedColumns: []string{"attributes"},
expectedEvols: []string{"attributes"},
},
{
name: "Non-candidate evolution ignored - JSON released before window keeps the map",
columns: []*schema.Column{
attributes_string,
},
evolutions: []*telemetrytypes.EvolutionEntry{
{
Signal: telemetrytypes.SignalTraces,
ColumnName: "attributes",
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextAttribute,
FieldName: "__all__",
ReleaseTime: time.Date(2024, 2, 10, 0, 0, 0, 0, time.UTC),
},
},
tsStart: uint64(time.Date(2024, 2, 15, 0, 0, 0, 0, time.UTC).UnixNano()),
tsEnd: uint64(time.Date(2024, 2, 20, 0, 0, 0, 0, time.UTC).UnixNano()),
expectedColumns: []string{"attributes_string"},
expectedEvols: []string{"attributes_string"},
},
}
for _, tc := range testCases {

View File

@@ -39,14 +39,16 @@ var (
// QueryInfo is the query's context as one value. It holds the time range
// every read needs, the signal and queried metric that family admission
// needs, and the query-path flags evaluated one time per request. The
// generic flows read FamiliesOn. Only the logs storage reads BodyJSONOn.
// generic flows read FamiliesOn. Only the logs storage reads BodyJSONOn, and
// only the traces storage reads TraceAttrsJSONOn.
type QueryInfo struct {
StartNs uint64
EndNs uint64
Signal telemetrytypes.Signal
Metric *telemetrytypes.MetricContext
FamiliesOn bool
BodyJSONOn bool
StartNs uint64
EndNs uint64
Signal telemetrytypes.Signal
Metric *telemetrytypes.MetricContext
FamiliesOn bool
BodyJSONOn bool
TraceAttrsJSONOn bool
}
// Absent is how a field key reads for a row that does not carry it, with

View File

@@ -22,3 +22,18 @@ func (Signal) Enum() []any {
SignalUnspecified,
}
}
// SignalFromText resolves a signal word to its Signal; ok is false for an
// unknown word.
func SignalFromText(text string) (Signal, bool) {
s := Signal{valuer.NewString(text)}
switch s {
case SignalTraces:
return SignalTraces, true
case SignalLogs:
return SignalLogs, true
case SignalMetrics:
return SignalMetrics, true
}
return Signal{}, false
}

View File

@@ -37,11 +37,13 @@ type MetadataStore interface {
// ListLogsJSONIndexes lists the JSON indexes for the logs table.
ListLogsJSONIndexes(ctx context.Context, filters ...string) ([]TelemetryFieldKeySkipIndex, error)
// ListPromotedPaths lists the promoted paths.
GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error)
// GetPromotedPaths lists the promoted paths recorded in the column
// evolution table for the entry's signal, column and field context.
GetPromotedPaths(ctx context.Context, entry EvolutionEntry, paths ...string) (map[string]bool, error)
// PromotePaths promotes the paths.
PromotePaths(ctx context.Context, paths ...string) error
// PromotePaths records promoted paths in the column evolution table as
// rows templated by entry; FieldName and ReleaseTime are set per path.
PromotePaths(ctx context.Context, entry EvolutionEntry, paths ...string) error
// GetFirstSeenFromMetricMetadata gets the first seen timestamp for a metric metadata lookup key.
GetFirstSeenFromMetricMetadata(ctx context.Context, lookupKeys []MetricMetadataLookupKey) (map[MetricMetadataLookupKey]int64, error)

View File

@@ -361,7 +361,7 @@ func (m *MockMetadataStore) SetTemporality(metricName string, temporality metric
}
// PromotePaths promotes the paths.
func (m *MockMetadataStore) PromotePaths(ctx context.Context, paths ...string) error {
func (m *MockMetadataStore) PromotePaths(_ context.Context, _ telemetrytypes.EvolutionEntry, paths ...string) error {
for _, path := range paths {
m.PromotedPathsMap[path] = true
}
@@ -369,7 +369,7 @@ func (m *MockMetadataStore) PromotePaths(ctx context.Context, paths ...string) e
}
// GetPromotedPaths returns the promoted paths.
func (m *MockMetadataStore) GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error) {
func (m *MockMetadataStore) GetPromotedPaths(_ context.Context, _ telemetrytypes.EvolutionEntry, _ ...string) (map[string]bool, error) {
return m.PromotedPathsMap, nil
}

View File

@@ -108,23 +108,14 @@ def delete_all_rules(signoz: types.SigNoz, token: str) -> None:
def seed_alert_rules(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
notification_channel: types.TestContainerDocker,
create_notification_channel: Callable[[dict], str],
create_alert_rule: Callable[[dict], str],
) -> Callable[[str, list[dict]], None]:
) -> Callable[[dict, list[dict]], None]:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# create_notification_channel rather than create_webhook_notification_channel:
# only the former deletes on teardown, and callers reuse one channel name
# across tests, so a leaked channel fails the next create as a duplicate.
def _seed_alert_rules(channel_name: str, rules: list[dict]) -> None:
def _seed_alert_rules(channel_config: dict, rules: list[dict]) -> None:
delete_all_rules(signoz, admin_token)
create_notification_channel(
{
"name": channel_name,
"webhook_configs": [{"url": notification_channel.container_configs["8080"].get(f"/alert/{channel_name}"), "send_resolved": False}],
}
)
create_notification_channel(channel_config)
for rule in rules:
create_alert_rule(rule)

Some files were not shown because too many files have changed in this diff Show More