Compare commits

...

10 Commits

Author SHA1 Message Date
nityanandagohain
a196433de6 Merge remote-tracking branch 'origin/main' into issue_nerve_462 2026-11-16 13:57:32 +05:30
Aditya Singh
f7e78384ed feat(query-builder): move QueryBuilderSearchV2 off the v3 autocomplete APIs (#13207)
#### Description
- The funnel step filters fetched keys and values from the v3
autocomplete APIs. The backend plans to remove those APIs.
- `QueryBuilderSearchV2` now fetches keys and values from the v5 fields
APIs. The pills and the saved filter shape stay the same, so saved
funnels and pipelines load as before.
- The funnel step Service and Span name dropdowns fetch values from v5
too, scoped to the `resource` and `span` contexts. They share a hook
with the Celery filters, so those move to v5 as well.
- Celery "Worker Online" counts workers from a v5 `query_range` over the
selected time range. It used to count the worker dropdown options, which
had no time range and a cap of 50.
- Pills carry the v5 context as the key type. The pipelines filter
converts it back to the legacy type when the filter changes (`attribute`
→ `tag`, log fields → top-level), so the filter preview, the save and
the stored pipelines match main.
- Span and log fields are marked as columns, so a funnel `EXISTS` on a
field like `name` reads the column.

#### Issues closed by this PR

Closes https://github.com/SigNoz/events-pod/issues/146
Closes https://github.com/SigNoz/events-pod/issues/145

#### Screen recordings
Funnels, Exceptions and pipelines on v5.


https://github.com/user-attachments/assets/217a0a85-984b-4f94-bf9e-a5f21696c3a6


https://github.com/user-attachments/assets/19ba4648-7466-447e-9cf2-e50700022cf9


https://github.com/user-attachments/assets/6dd6e984-d2e6-4a03-b639-a82181618d40

#### Additional Information

- Part of https://github.com/SigNoz/events-pod/issues/156. The same
changes move these filters off v3:
- the exceptions filter, https://github.com/SigNoz/events-pod/issues/145
  - the pipelines filter, https://github.com/SigNoz/events-pod/issues/25
- the Celery dropdowns, https://github.com/SigNoz/events-pod/issues/157
- UI change: the key dropdown badge shows the v5 context, `attribute`
instead of `tag` and `span` for span fields. Neither has a badge colour.
Visible on the funnel step Where filter, the exceptions filter and the
pipelines filter.
- The pipelines filter no longer shows example queries. The v5 fields
API has no equivalent.
- These filters stay on `QueryBuilderSearchV2` with v5 lookups. The move
to `QuerySearch` is left to the page revamps.
- Stories: the Funnels, Celery and Exceptions page stories serve the v5
fields APIs, and the Celery story serves the Worker Online query.
- another PR will be raised to move query range to v5 for logs pipeline
2026-11-16 07:59:24 +00:00
nityanandagohain
15a47b96d8 Merge remote-tracking branch 'origin/main' into issue_nerve_462 2026-11-11 23:47:08 +05:30
nityanandagohain
431be1d334 fix: don't import from types 2026-11-11 16:25:19 +05:30
nityanandagohain
b021cdfcc2 fix: add group update checheck on mapper update 2026-11-11 16:13:52 +05:30
nityanandagohain
52ee3943ce fix: address comment 2026-11-11 16:01:03 +05:30
nityanandagohain
99e0cb7d8e Merge remote-tracking branch 'origin/main' into issue_nerve_462 2026-11-11 15:57:34 +05:30
nityanandagohain
20080ec988 Merge remote-tracking branch 'origin/main' into issue_nerve_462 2026-10-29 16:17:03 +05:30
nityanandagohain
fb33e31c4c Merge remote-tracking branch 'origin/main' into issue_nerve_462 2026-10-26 17:54:41 +05:30
nityanandagohain
0a0cd96e93 feat(spanmapper): add fga to span mapper group and mapper routes 2026-10-26 17:51:42 +05:30
40 changed files with 1630 additions and 435 deletions

View File

@@ -3961,6 +3961,8 @@ components:
- traces-field
- metrics-field
- llm-pricing-rule
- span-mapper-group
- span-mapper
type: string
CoretypesObject:
properties:
@@ -18129,9 +18131,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- span-mapper-group:list
- tokenizer:
- VIEWER
- span-mapper-group:list
summary: List span attribute mapping groups
tags:
- spanmapper
@@ -18192,9 +18194,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- ADMIN
- span-mapper-group:create
- tokenizer:
- ADMIN
- span-mapper-group:create
summary: Create a span attribute mapping group
tags:
- spanmapper
@@ -18239,9 +18241,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- ADMIN
- span-mapper-group:delete
- tokenizer:
- ADMIN
- span-mapper-group:delete
summary: Delete a span attribute mapping group
tags:
- spanmapper
@@ -18297,9 +18299,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- ADMIN
- span-mapper-group:update
- tokenizer:
- ADMIN
- span-mapper-group:update
summary: Update a span attribute mapping group
tags:
- spanmapper
@@ -18362,9 +18364,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- span-mapper-group:read
- tokenizer:
- VIEWER
- span-mapper-group:read
summary: List span mappers for a group
tags:
- spanmapper
@@ -18437,9 +18439,11 @@ paths:
description: Internal Server Error
security:
- api_key:
- ADMIN
- span-mapper:create
- span-mapper-group:attach
- tokenizer:
- ADMIN
- span-mapper:create
- span-mapper-group:attach
summary: Create a span mapper
tags:
- spanmapper
@@ -18489,9 +18493,11 @@ paths:
description: Internal Server Error
security:
- api_key:
- ADMIN
- span-mapper:delete
- span-mapper-group:detach
- tokenizer:
- ADMIN
- span-mapper:delete
- span-mapper-group:detach
summary: Delete a span mapper
tags:
- spanmapper
@@ -18552,9 +18558,11 @@ paths:
description: Internal Server Error
security:
- api_key:
- ADMIN
- span-mapper:update
- span-mapper-group:update
- tokenizer:
- ADMIN
- span-mapper:update
- span-mapper-group:update
summary: Update a span mapper
tags:
- spanmapper
@@ -18562,7 +18570,8 @@ paths:
/api/v1/span_mapper_groups/test:
post:
deprecated: false
description: Tests how span mappers would transform sample spans
description: Tests how span mappers would transform sample spans. Groups sent
without mappers are read from the store by name.
operationId: TestSpanMappers
requestBody:
content:
@@ -18616,9 +18625,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- span-mapper-group:read
- tokenizer:
- VIEWER
- span-mapper-group:read
summary: Test span mappers against sample spans
tags:
- spanmapper

View File

@@ -3605,6 +3605,8 @@ export enum CoretypesKindDTO {
'traces-field' = 'traces-field',
'metrics-field' = 'metrics-field',
'llm-pricing-rule' = 'llm-pricing-rule',
'span-mapper-group' = 'span-mapper-group',
'span-mapper' = 'span-mapper',
}
export enum CoretypesTypeDTO {
user = 'user',

View File

@@ -803,7 +803,7 @@ export const useUpdateSpanMapper = <
return useMutation(getUpdateSpanMapperMutationOptions(options));
};
/**
* Tests how span mappers would transform sample spans
* Tests how span mappers would transform sample spans. Groups sent without mappers are read from the store by name.
* @summary Test span mappers against sample spans
*/
export const testSpanMappers = (

View File

@@ -1,5 +1,6 @@
import { useCallback, useMemo, useState } from 'react';
import { Row, Select, Spin } from 'antd';
import { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import { navigate } from 'lib/router/navigation';
import { useAppLocation } from 'lib/router/useAppLocation';
import {
@@ -17,6 +18,7 @@ export interface SelectOptionConfig {
placeholder: string;
queryParam: QueryParams;
filterType: string | string[];
fieldContext?: TelemetrytypesFieldContextDTO;
shouldSetQueryParams?: boolean;
onChange?: (value: string | string[]) => void;
values?: string | string[];
@@ -27,13 +29,16 @@ export function FilterSelect({
placeholder,
queryParam,
filterType,
fieldContext,
values,
shouldSetQueryParams,
onChange,
isMultiple,
}: SelectOptionConfig): JSX.Element {
const { handleSearch, isFetching, options } =
useCeleryFilterOptions(filterType);
const { handleSearch, isFetching, options } = useCeleryFilterOptions(
filterType,
fieldContext,
);
const urlQuery = useUrlQuery();
const location = useAppLocation();
@@ -118,6 +123,7 @@ export function FilterSelect({
}
FilterSelect.defaultProps = {
fieldContext: undefined,
shouldSetQueryParams: true,
onChange: (): void => {},
values: [],
@@ -129,12 +135,14 @@ function CeleryOverviewConfigOptions(): JSX.Element {
{
placeholder: 'Service Name',
queryParam: QueryParams.service,
filterType: 'serviceName',
filterType: 'service.name',
fieldContext: TelemetrytypesFieldContextDTO.resource,
},
{
placeholder: 'Span Name',
queryParam: QueryParams.spanName,
filterType: 'name',
fieldContext: TelemetrytypesFieldContextDTO.span,
},
{
placeholder: 'Msg System',
@@ -162,6 +170,7 @@ function CeleryOverviewConfigOptions(): JSX.Element {
placeholder={config.placeholder}
queryParam={config.queryParam}
filterType={config.filterType}
fieldContext={config.fieldContext}
/>
))}
</Row>

View File

@@ -0,0 +1,37 @@
import type { GetFieldsValues200 } from 'api/generated/services/sigNoz.schemas';
import { rest, server } from 'mocks-server/server';
import { render, waitFor } from 'tests/test-utils';
import CeleryOverviewConfigOptions from '../CeleryOverviewConfigOptions';
const emptyValues: GetFieldsValues200 = {
status: 'success',
data: { complete: true, values: { stringValues: [] } },
};
describe('CeleryOverviewConfigOptions', () => {
it('scopes the service and span name lookups to their field context', async () => {
const contextByKey = new Map<string, string | null>();
server.use(
rest.get('*/api/v1/fields/values', (req, res, ctx) => {
contextByKey.set(
req.url.searchParams.get('name') ?? '',
req.url.searchParams.get('fieldContext'),
);
return res(ctx.status(200), ctx.json(emptyValues));
}),
);
render(<CeleryOverviewConfigOptions />);
await waitFor(() => expect(contextByKey.size).toBe(6));
expect(Object.fromEntries(contextByKey)).toStrictEqual({
'service.name': 'resource',
name: 'span',
'messaging.system': null,
'messaging.destination.name': null,
'messaging.destination': null,
kind_string: null,
});
});
});

View File

@@ -1,23 +1,21 @@
import { useQuery } from 'react-query';
// eslint-disable-next-line no-restricted-imports
import { useSelector } from 'react-redux';
import type { DefaultOptionType } from 'antd/es/select';
import { getAttributesValues } from 'api/queryBuilder/getAttributesValues';
import { DATA_TYPE_VS_ATTRIBUTE_VALUES_KEY } from 'constants/queryBuilder';
import type { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import { getFieldValueSuggestions } from 'api/querySuggestions/getFieldValueSuggestions';
import { FIELD_API_CACHE_TIME } from 'constants/queryCacheTime';
import { REACT_QUERY_KEY } from 'constants/reactQueryKeys';
import { AppState } from 'store/reducers';
import { toFieldValues } from 'hooks/querySuggestions/useFieldValuesSuggestion';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { DataSource } from 'types/common/queryBuilder';
import { GlobalReducer } from 'types/reducer/globalTime';
import { DATA_SOURCE_TO_SIGNAL, DataSource } from 'types/common/queryBuilder';
import { getValuesForDataType } from 'utils/fieldValues';
export interface Filters {
searchText: string;
attributeKey: string | string[];
aggregateOperator?: string;
dataSource?: DataSource;
aggregateAttribute?: string;
metricName?: string;
filterAttributeKeyDataType?: DataTypes;
tagType?: string;
fieldContext?: TelemetrytypesFieldContextDTO;
}
export interface GetAllFiltersResponse {
@@ -29,54 +27,41 @@ export function useGetAllFilters(props: Filters): GetAllFiltersResponse {
const {
searchText,
attributeKey,
aggregateOperator,
dataSource,
aggregateAttribute,
metricName,
filterAttributeKeyDataType,
tagType,
fieldContext,
} = props;
const { minTime, maxTime } = useSelector<AppState, GlobalReducer>(
(state) => state.globalTime,
);
const { data, isLoading } = useQuery(
[
REACT_QUERY_KEY.GET_ATTRIBUTE_VALUES,
dataSource,
metricName,
attributeKey,
fieldContext,
searchText,
minTime,
maxTime,
],
async () => {
const keys = Array.isArray(attributeKey) ? attributeKey : [attributeKey];
const responses = await Promise.all(
keys.map((key) =>
getAttributesValues({
aggregateOperator: aggregateOperator || 'noop',
dataSource: dataSource || DataSource.TRACES,
aggregateAttribute: aggregateAttribute || '',
attributeKey: key,
getFieldValueSuggestions({
signal: DATA_SOURCE_TO_SIGNAL[dataSource || DataSource.TRACES],
name: key,
fieldContext,
searchText: searchText ?? '',
filterAttributeKeyDataType:
filterAttributeKeyDataType || DataTypes.String,
tagType: tagType || 'tag',
metricName,
}),
),
);
const uniqueValues = [
...new Set(
responses.flatMap(({ payload }) => {
if (!payload) {
return [];
}
const dataType = filterAttributeKeyDataType || DataTypes.String;
const key = DATA_TYPE_VS_ATTRIBUTE_VALUES_KEY[dataType];
return key ? payload[key] || [] : [];
}),
responses.flatMap((response) =>
getValuesForDataType(toFieldValues(response), filterAttributeKeyDataType),
),
),
];
@@ -85,6 +70,7 @@ export function useGetAllFilters(props: Filters): GetAllFiltersResponse {
value: val,
}));
},
{ staleTime: FIELD_API_CACHE_TIME },
);
return { options: data ?? [], isFetching: isLoading };

View File

@@ -8,12 +8,9 @@ import { CardContainer } from 'container/WidgetCard/styles';
import { useIsDarkMode } from 'hooks/useDarkMode';
import { ChevronDown, ChevronUp } from '@signozhq/icons';
import { AppState } from 'store/reducers';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { DataSource } from 'types/common/queryBuilder';
import { GlobalReducer } from 'types/reducer/globalTime';
import { CaptureDataProps } from '../CeleryTaskDetail/CeleryTaskDetail';
import { useCeleryFilterOptions } from '../useCeleryFilterOptions';
import CeleryTaskBar from './CeleryTaskBar';
import CeleryTaskGraph from './CeleryTaskGraph';
import {
@@ -23,6 +20,7 @@ import {
celeryTasksByWorkerWidgetData,
} from './CeleryTaskGraphUtils';
import CeleryTaskLatencyGraph from './CeleryTaskLatencyGraph';
import { useCeleryWorkerCount } from './useCeleryWorkerCount';
import './CeleryTaskGraph.style.scss';
@@ -71,14 +69,7 @@ export default function CeleryTaskGraphGrid({
'Latency by worker',
];
const { options } = useCeleryFilterOptions(
'worker',
'rate',
DataSource.METRICS,
'flower_task_runtime_seconds_sum',
DataTypes.String,
'tag',
);
const { data: workerCount } = useCeleryWorkerCount();
const isDarkMode = useIsDarkMode();
const [collapsedSections, setCollapsedSections] = useState<{
@@ -151,8 +142,11 @@ export default function CeleryTaskGraphGrid({
</Typography.Text>
</div>
<div className="worker-count-text-container">
<Typography.Text className="celery-task-graph-worker-count-text">
{options.filter((option) => option.value).length}
<Typography.Text
className="celery-task-graph-worker-count-text"
data-testid="celery-worker-count"
>
{workerCount ?? '-'}
</Typography.Text>
</div>
</Card>

View File

@@ -1,3 +1,12 @@
import {
MetrictypesSpaceAggregationDTO,
MetrictypesTimeAggregationDTO,
Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5MetricAggregationDTOSignal as MetricSignal,
Querybuildertypesv5QueryEnvelopeBuilderDTOType,
type Querybuildertypesv5QueryRangeRequestDTO,
Querybuildertypesv5ReduceToDTO,
Querybuildertypesv5RequestTypeDTO,
} from 'api/generated/services/sigNoz.schemas';
import { PANEL_TYPES } from 'constants/queryBuilder';
import { getWidgetQueryBuilder } from 'container/MetricsApplication/MetricsApplication.factory';
import { getWidgetQuery } from 'pages/MessagingQueues/MQDetails/MetricPage/MetricPageUtil';
@@ -1042,3 +1051,34 @@ export const celeryRetryStateCountWidgetData = getWidgetQueryBuilder(
],
}),
);
// One row per worker that reported in the range; the card counts the rows.
export const getCeleryWorkerCountRequest = (
startTime: number,
endTime: number,
): Querybuildertypesv5QueryRangeRequestDTO => ({
schemaVersion: 'v1',
start: Math.floor(startTime / 1e6),
end: Math.floor(endTime / 1e6),
requestType: Querybuildertypesv5RequestTypeDTO.scalar,
compositeQuery: {
queries: [
{
type: Querybuildertypesv5QueryEnvelopeBuilderDTOType.builder_query,
spec: {
name: 'A',
signal: MetricSignal.metrics,
aggregations: [
{
metricName: 'flower_task_runtime_seconds_sum',
timeAggregation: MetrictypesTimeAggregationDTO.count,
spaceAggregation: MetrictypesSpaceAggregationDTO.sum,
reduceTo: Querybuildertypesv5ReduceToDTO.sum,
},
],
groupBy: [{ name: 'worker' }],
},
},
],
},
});

View File

@@ -0,0 +1,93 @@
import {
Querybuildertypesv5ColumnTypeDTO,
type Querybuildertypesv5QueryRangeRequestDTO,
Querybuildertypesv5RequestTypeDTO,
type QueryRangeV5200,
} from 'api/generated/services/sigNoz.schemas';
import { rest, server } from 'mocks-server/server';
import store from 'store';
import { render, screen, waitFor } from 'tests/test-utils';
import CeleryTaskGraphGrid from '../CeleryTaskGraphGrid';
const workerCountResponse = (workers: string[]): QueryRangeV5200 => ({
status: 'success',
data: {
type: Querybuildertypesv5RequestTypeDTO.scalar,
data: {
results: [
{
columns: [
{
name: 'worker',
queryName: 'A',
columnType: Querybuildertypesv5ColumnTypeDTO.group,
},
{
name: '__result_0',
queryName: 'A',
aggregationIndex: 0,
columnType: Querybuildertypesv5ColumnTypeDTO.aggregation,
},
],
data: workers.map((worker) => [worker, 4]),
},
],
},
},
});
describe('CeleryTaskGraphGrid worker count', () => {
let requests: Querybuildertypesv5QueryRangeRequestDTO[];
beforeEach(() => {
requests = [];
server.use(
rest.post('*/api/v5/query_range', async (req, res, ctx) => {
requests.push(await req.json());
return res(
ctx.status(200),
ctx.json(workerCountResponse(['worker-a', 'worker-b', ''])),
);
}),
);
});
it('counts the workers that reported in the selected range', async () => {
render(<CeleryTaskGraphGrid onClick={jest.fn()} queryEnabled={false} />);
await waitFor(() =>
expect(screen.getByTestId('celery-worker-count')).toHaveTextContent('2'),
);
const { minTime, maxTime } = store.getState().globalTime;
expect(requests).toStrictEqual([
{
schemaVersion: 'v1',
start: Math.floor(minTime / 1e6),
end: Math.floor(maxTime / 1e6),
requestType: 'scalar',
compositeQuery: {
queries: [
{
type: 'builder_query',
spec: {
name: 'A',
signal: 'metrics',
aggregations: [
{
metricName: 'flower_task_runtime_seconds_sum',
timeAggregation: 'count',
spaceAggregation: 'sum',
reduceTo: 'sum',
},
],
groupBy: [{ name: 'worker' }],
},
},
],
},
},
]);
});
});

View File

@@ -0,0 +1,31 @@
import { useMemo } from 'react';
import { useQuery, UseQueryResult } from 'react-query';
// eslint-disable-next-line no-restricted-imports
import { useSelector } from 'react-redux';
import { queryRangeV5 } from 'api/generated/services/querier';
import type { QueryRangeV5200 } from 'api/generated/services/sigNoz.schemas';
import { REACT_QUERY_KEY } from 'constants/reactQueryKeys';
import { getScalarResults } from 'pages/DashboardPage/DashboardContainer/queryV5/v5ResponseData';
import { AppState } from 'store/reducers';
import { GlobalReducer } from 'types/reducer/globalTime';
import { getCeleryWorkerCountRequest } from './CeleryTaskGraphUtils';
export function useCeleryWorkerCount(): UseQueryResult<number, Error> {
const { minTime, maxTime } = useSelector<AppState, GlobalReducer>(
(state) => state.globalTime,
);
const request = useMemo(
() => getCeleryWorkerCountRequest(minTime, maxTime),
[minTime, maxTime],
);
return useQuery<QueryRangeV5200, Error, number>({
queryKey: [REACT_QUERY_KEY.GET_CELERY_WORKER_COUNT, request],
queryFn: ({ signal }) => queryRangeV5(request, signal),
select: (response) =>
(getScalarResults(response)[0]?.data ?? []).filter(([worker]) => worker)
.length,
});
}

View File

@@ -1,18 +1,13 @@
import { useState } from 'react';
import type { DefaultOptionType } from 'antd/es/select';
import type { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import useDebouncedFn from 'hooks/useDebouncedFunction';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { DataSource } from 'types/common/queryBuilder';
import { useGetAllFilters } from './CeleryTaskConfigOptions/useGetCeleryFilters';
export const useCeleryFilterOptions = (
type: string | string[],
aggregateOperator?: string,
dataSource?: DataSource,
aggregateAttribute?: string,
filterAttributeKeyDataType?: DataTypes,
tagType?: string,
fieldContext?: TelemetrytypesFieldContextDTO,
): {
searchText: string;
handleSearch: (value: string) => void;
@@ -23,11 +18,7 @@ export const useCeleryFilterOptions = (
const { isFetching, options } = useGetAllFilters({
attributeKey: type,
searchText,
aggregateOperator,
dataSource,
aggregateAttribute,
filterAttributeKeyDataType,
tagType,
fieldContext,
});
const handleDebouncedSearch = useDebouncedFn((searchText): void => {
setSearchText(searchText as string);

View File

@@ -1,5 +1,5 @@
import { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import { TelemetryFieldKey } from 'types/api/v5/queryRange';
import { fieldContextToAttributeType } from 'utils/fieldContext';
import { fieldDataTypeToDataType } from 'utils/fieldDataType';
import { FiltersType, IQuickFiltersConfig } from './types';
@@ -14,19 +14,6 @@ const FILTER_TYPE_MAP: Record<string, FiltersType> = {
duration_nano: FiltersType.DURATION,
};
// The map below exists only for the old v3 attribute-values fetch
// (useCheckboxFilterValues), the sole reader of attributeKey.dataType/type.
// Once the values fetch moves to fields/values, remove this and reduce
// attributeKey to { id, key }.
const FIELD_CONTEXT_TO_ATTRIBUTE_TYPE: Record<string, string> = {
[TelemetrytypesFieldContextDTO.attribute]: 'tag',
[TelemetrytypesFieldContextDTO.resource]: 'resource',
};
const mapFieldContext = (fieldContext?: string): string =>
(fieldContext && FIELD_CONTEXT_TO_ATTRIBUTE_TYPE[fieldContext]) || '';
const getFilterName = (str: string): string => {
if (FILTER_TITLE_MAP[str]) {
return FILTER_TITLE_MAP[str];
@@ -61,11 +48,15 @@ export const getFilterConfig = (
({
type: getFilterType(att),
title: getFilterName(att.name),
// The type mapping below exists only for the old v3 attribute-values fetch
// (useCheckboxFilterValues), the sole reader of attributeKey.dataType/type.
// Once the values fetch moves to fields/values, remove this and reduce
// attributeKey to { id, key }.
attributeKey: {
id: att.name,
key: att.name,
dataType: fieldDataTypeToDataType(att.fieldDataType),
type: mapFieldContext(att.fieldContext),
type: fieldContextToAttributeType(att.fieldContext),
},
defaultOpen: index < 2,
}) as IQuickFiltersConfig,

View File

@@ -109,6 +109,9 @@ export const REACT_QUERY_KEY = {
// Field Values Suggestion Query Keys
FIELD_VALUES_SUGGESTION: 'FIELD_VALUES_SUGGESTION',
// Celery Query Keys
GET_CELERY_WORKER_COUNT: 'GET_CELERY_WORKER_COUNT',
// AI Assistant Query Keys
AI_ASSISTANT_EMPTY_STATE_CHIPS: 'AI_ASSISTANT_EMPTY_STATE_CHIPS',
} as const;

View File

@@ -0,0 +1,70 @@
import { Form } from 'antd';
import QueryBuilderSearchV2 from 'container/QueryBuilder/filters/QueryBuilderSearchV2/QueryBuilderSearchV2';
import { rest, server } from 'mocks-server/server';
import { act, render, waitFor } from 'tests/test-utils';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import type { QueryRangePayload } from 'types/api/metrics/getQueryRange';
import type {
IBuilderQuery,
TagFilter,
} from 'types/api/queryBuilder/queryBuilderData';
import { pipelineFields } from '../../../../config';
import FilterInput from '..';
jest.mock(
'container/QueryBuilder/filters/QueryBuilderSearchV2/QueryBuilderSearchV2',
() => ({ __esModule: true, default: jest.fn(() => null) }),
);
const filterField = pipelineFields.find((field) => field.name === 'filter');
const v5Filter: TagFilter = {
op: 'AND',
items: [
{
id: 'attribute-pill',
key: { key: 'http.method', dataType: DataTypes.String, type: 'attribute' },
op: '=',
value: 'GET',
},
{
id: 'log-pill',
key: { key: 'severity_text', dataType: DataTypes.String, type: 'log' },
op: '=',
value: 'ERROR',
},
],
};
describe('Pipeline FilterInput', () => {
it('previews sample logs with the legacy key types the v3 logs builder reads', async () => {
const previewFilters: TagFilter[] = [];
server.use(
rest.post('*/api/v3/query_range', async (req, res, ctx) => {
const body: QueryRangePayload = await req.json();
const query = body.compositeQuery.builderQueries?.A as IBuilderQuery;
if (query.filters) {
previewFilters.push(query.filters);
}
return res(ctx.status(200), ctx.json({ data: { result: [] } }));
}),
);
render(<Form>{filterField && <FilterInput fieldData={filterField} />}</Form>);
const { onChange } = jest.mocked(QueryBuilderSearchV2).mock.calls[0][0];
act(() => onChange(v5Filter));
await waitFor(() => expect(previewFilters).not.toHaveLength(0));
expect(
previewFilters[previewFilters.length - 1].items.map((item) => [
item.key?.key,
item.key?.type,
]),
).toStrictEqual([
['http.method', 'tag'],
['severity_text', ''],
]);
});
});

View File

@@ -4,6 +4,7 @@ import { initialQueryBuilderFormValuesMap } from 'constants/queryBuilder';
import QueryBuilderSearchV2 from 'container/QueryBuilder/filters/QueryBuilderSearchV2/QueryBuilderSearchV2';
import isEqual from 'lodash-es/isEqual';
import { TagFilter } from 'types/api/queryBuilder/queryBuilderData';
import { toLegacyLogsFilter } from 'utils/fieldContext';
import { ProcessorFormField } from '../../../AddNewProcessor/config';
import { formValidationRules } from '../../../config';
@@ -22,9 +23,10 @@ function TagFilterInput({
}
const onQueryChange = (newValue: TagFilter): void => {
const legacyValue = toLegacyLogsFilter(newValue);
// Avoid unnecessary onChange calls
if (!isEqual(newValue, query.filters)) {
onChange(newValue);
if (!isEqual(legacyValue, query.filters)) {
onChange(legacyValue);
}
};

View File

@@ -10,7 +10,6 @@ import {
userEvent,
waitFor,
} from 'tests/test-utils';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { pipelineApiResponseMockData } from '../mocks/pipeline';
import PipelineListsView from '../PipelineListsView';
@@ -85,7 +84,7 @@ jest.mock('providers/preferences/sync/usePreferenceSync', () => ({
}));
const BASE_URL = ENVIRONMENT.baseURL;
const attributeKeysURL = `${BASE_URL}/api/v3/filter_suggestions`;
const fieldKeysURL = `${BASE_URL}/api/v1/fields/keys`;
describe('PipelinePage container test', () => {
beforeAll(() => {
@@ -339,42 +338,41 @@ describe('PipelinePage container test', () => {
expect(screen.queryAllByText('source = nginx')).toHaveLength(2);
server.use(
rest.get(attributeKeysURL, (_req, res, ctx) =>
rest.get(fieldKeysURL, (_req, res, ctx) =>
res(
ctx.status(200),
ctx.json({
status: 'success',
data: {
attributes: [
{
key: 'otelServiceName',
dataType: DataTypes.String,
type: 'tag',
isColumn: false,
isJSON: false,
},
{
key: 'service.name',
dataType: DataTypes.String,
type: 'resource',
isColumn: false,
isJSON: false,
},
{
key: 'service.instance.id',
dataType: DataTypes.String,
type: 'resource',
isColumn: false,
isJSON: false,
},
{
key: 'service.name',
dataType: DataTypes.String,
type: 'tag',
isColumn: false,
isJSON: false,
},
],
complete: true,
keys: {
otelServiceName: [
{
name: 'otelServiceName',
fieldContext: 'attribute',
fieldDataType: 'string',
},
],
'service.name': [
{
name: 'service.name',
fieldContext: 'resource',
fieldDataType: 'string',
},
{
name: 'service.name',
fieldContext: 'attribute',
fieldDataType: 'string',
},
],
'service.instance.id': [
{
name: 'service.instance.id',
fieldContext: 'resource',
fieldDataType: 'string',
},
],
},
},
}),
),

View File

@@ -10,9 +10,9 @@ import {
} from 'react';
import { type RefSelectProps, Select, Spin } from 'antd';
import { Pill } from '@signozhq/ui/pill';
import { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import cx from 'classnames';
import {
DATA_TYPE_VS_ATTRIBUTE_VALUES_KEY,
OperatorConfigKeys,
OPERATORS,
QUERY_BUILDER_OPERATORS_BY_TYPES,
@@ -22,23 +22,14 @@ import { DEBOUNCE_DELAY } from 'constants/queryBuilderFilterConfig';
import { LogsExplorerShortcuts } from 'constants/shortcuts/logsExplorerShortcuts';
import { useDynamicVariableSuggestions } from 'hooks/dashboard/useDynamicVariableSuggestions';
import { useKeyboardHotkeys } from 'hooks/hotkeys/useKeyboardHotkeys';
import { useGetAggregateKeys } from 'hooks/queryBuilder/useGetAggregateKeys';
import { useGetAggregateValues } from 'hooks/queryBuilder/useGetAggregateValues';
import { useGetAttributeSuggestions } from 'hooks/queryBuilder/useGetAttributeSuggestions';
import { validationMapper } from 'hooks/queryBuilder/useIsValidTag';
import { operatorTypeMapper } from 'hooks/queryBuilder/useOperatorType';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { useRunQueryWithFilterItems } from 'hooks/queryBuilder/useRunQueryWithFilterItems';
import { useFieldKeysSuggestion } from 'hooks/querySuggestions/useFieldKeysSuggestion';
import { useFieldValuesSuggestion } from 'hooks/querySuggestions/useFieldValuesSuggestion';
import useDebounceValue from 'hooks/useDebounce';
import {
cloneDeep,
isArray,
isEmpty,
isEqual,
isObject,
isUndefined,
unset,
} from 'lodash-es';
import { isArray, isEmpty, isEqual, isObject, isUndefined } from 'lodash-es';
import { ChevronDown, ChevronUp } from '@signozhq/icons';
import {
BaseAutocompleteData,
@@ -48,7 +39,13 @@ import {
IBuilderQuery,
TagFilter,
} from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
import {
DATA_SOURCE_TO_SIGNAL,
DataSource,
QUERY_SOURCE_TO_FIELD_SOURCE,
} from 'types/common/queryBuilder';
import { attributeTypeToFieldContext } from 'utils/fieldContext';
import { getValuesForDataType } from 'utils/fieldValues';
import { useSelectPopupContainer } from 'utils/selectPopupContainer';
import { v4 as uuid } from 'uuid';
@@ -61,7 +58,8 @@ import {
getTagToken,
isInNInOperator,
} from './utils';
import { filterByOperatorConfig } from '../utils';
import { toAttributeKey } from '../GroupByFilter/utils';
import { filterByOperatorConfig, getMetricNameForQuery } from '../utils';
import QueryBuilderSearchDropdown from './QueryBuilderSearchDropdown';
import SpanScopeSelector from './SpanScopeSelector';
import Suggestions from './Suggestions';
@@ -75,6 +73,8 @@ export interface ITag {
value: (string | number | boolean)[] | string | number | boolean;
}
const SPAN_SEARCH_SCOPE_KEYS = ['isRoot', 'isEntryPoint'];
interface CustomTagProps {
label: React.ReactNode;
value: string;
@@ -186,70 +186,15 @@ function QueryBuilderSearchV2(
[query.dataSource],
);
const memoizedSearchParams = useMemo(
() => [
searchValue,
query.dataSource,
query.aggregateOperator,
query.aggregateAttribute?.key,
],
[
searchValue,
query.dataSource,
query.aggregateOperator,
query.aggregateAttribute?.key,
],
);
const queryFiltersWithoutId = useMemo(
() => ({
...query.filters,
items: query.filters?.items.map((item) => {
const filterWithoutId = cloneDeep(item);
unset(filterWithoutId, 'id');
return filterWithoutId;
}),
}),
[query.filters],
);
const memoizedSuggestionsParams = useMemo(
() => [searchValue, query.dataSource, queryFiltersWithoutId],
[query.dataSource, queryFiltersWithoutId, searchValue],
);
const memoizedValueParams = useMemo(
() => [
query.aggregateOperator,
query.dataSource,
query.aggregateAttribute?.key,
currentFilterItem?.key?.key || '',
currentFilterItem?.key?.dataType,
currentFilterItem?.key?.type ?? '',
isArray(currentFilterItem?.value)
? currentFilterItem?.value?.[currentFilterItem.value.length - 1]
: currentFilterItem?.value,
],
[
query.aggregateOperator,
query.dataSource,
query.aggregateAttribute?.key,
currentFilterItem?.key?.key,
currentFilterItem?.key?.dataType,
currentFilterItem?.key?.type,
currentFilterItem?.value,
],
);
const searchParams = useDebounceValue(memoizedSearchParams, DEBOUNCE_DELAY);
const valueParams = useDebounceValue(memoizedValueParams, DEBOUNCE_DELAY);
const suggestionsParams = useDebounceValue(
memoizedSuggestionsParams,
const debouncedKeySearchText = useDebounceValue(
searchValue?.split(' ')[0] ?? '',
DEBOUNCE_DELAY,
);
const valueSearchText = isArray(currentFilterItem?.value)
? String(currentFilterItem?.value?.[currentFilterItem.value.length - 1] ?? '')
: currentFilterItem?.value?.toString() || '';
const isQueryEnabled = useMemo(() => {
if (currentState === DropdownState.ATTRIBUTE_KEY) {
return query.dataSource === DataSource.METRICS
@@ -262,53 +207,76 @@ function QueryBuilderSearchV2(
const dashboardDynamicVariables = useDynamicVariableSuggestions();
const { data, isFetching } = useGetAggregateKeys(
const signal = DATA_SOURCE_TO_SIGNAL[query.dataSource];
const source = QUERY_SOURCE_TO_FIELD_SOURCE[query.source ?? ''];
const metricName = getMetricNameForQuery(query);
const { data: fieldKeys, isFetching } = useFieldKeysSuggestion(
{
searchText: searchValue?.split(' ')[0],
dataSource: query.dataSource,
aggregateOperator: query.aggregateOperator || '',
aggregateAttribute: query.aggregateAttribute?.key || '',
tagType: query.aggregateAttribute?.type ?? null,
signal,
source,
metricName,
fieldContext: attributeTypeToFieldContext(query.aggregateAttribute?.type),
searchText: debouncedKeySearchText,
},
query.builderQueryType,
{
queryKey: [searchParams],
enabled: isQueryEnabled && !isLogsDataSource && !hardcodedAttributeKeys,
enabled: isQueryEnabled && !hardcodedAttributeKeys,
keepPreviousData: true,
},
);
const { data: suggestionsData, isFetching: isFetchingSuggestions } =
useGetAttributeSuggestions(
{
searchText: searchValue?.split(' ')[0],
dataSource: query.dataSource,
filters: query.filters || { items: [], op: 'AND' },
},
{
queryKey: [suggestionsParams],
enabled: isQueryEnabled && isLogsDataSource,
},
);
// The legacy traces builder checks `exists` on the column only for keys marked as columns.
// Body JSON keys come back as bare paths, which the legacy builders read as top-level fields,
// and the span search scope selectors are not columns the legacy filter builder can read.
const attributeKeys = useMemo(
() =>
(fieldKeys ?? [])
.filter(
(fieldKey) =>
fieldKey.fieldContext !== TelemetrytypesFieldContextDTO.body &&
!(
fieldKey.fieldContext === TelemetrytypesFieldContextDTO.span &&
SPAN_SEARCH_SCOPE_KEYS.includes(fieldKey.name)
),
)
.map((fieldKey) => ({
...toAttributeKey(fieldKey),
isColumn:
fieldKey.fieldContext === TelemetrytypesFieldContextDTO.span ||
fieldKey.fieldContext === TelemetrytypesFieldContextDTO.log,
})),
[fieldKeys],
);
const { data: attributeValues, isFetching: isFetchingAttributeValues } =
useGetAggregateValues(
{
aggregateOperator: query.aggregateOperator || '',
dataSource: query.dataSource,
aggregateAttribute: query.aggregateAttribute?.key || '',
attributeKey: currentFilterItem?.key?.key || '',
filterAttributeKeyDataType:
currentFilterItem?.key?.dataType ?? DataTypes.EMPTY,
tagType: currentFilterItem?.key?.type ?? '',
searchText: isArray(currentFilterItem?.value)
? String(currentFilterItem?.value?.[currentFilterItem.value.length - 1]) ||
''
: currentFilterItem?.value?.toString() || '',
},
{
enabled: currentState === DropdownState.ATTRIBUTE_VALUE,
queryKey: [valueParams],
},
);
const fieldValuesConfig = useMemo(
() => ({
signal,
source,
metricName,
name: currentFilterItem?.key?.key || '',
fieldContext: attributeTypeToFieldContext(currentFilterItem?.key?.type),
searchText: valueSearchText,
}),
[
signal,
source,
metricName,
currentFilterItem?.key?.key,
currentFilterItem?.key?.type,
valueSearchText,
],
);
const debouncedFieldValuesConfig = useDebounceValue(
fieldValuesConfig,
DEBOUNCE_DELAY,
);
const { data: fieldValues, isFetching: isFetchingAttributeValues } =
useFieldValuesSuggestion(debouncedFieldValuesConfig, query.builderQueryType, {
enabled: currentState === DropdownState.ATTRIBUTE_VALUE,
});
const handleDropdownSelect = useCallback(
(value: string) => {
@@ -533,15 +501,13 @@ function QueryBuilderSearchV2(
if (tagKey && isUndefined(currentFilterItem?.key)) {
let currentRunningAttributeKey;
const isSuggestedKeyInAutocomplete =
suggestionsData?.payload?.attributes?.some(
(value) => value.key === tagKey.split(' ')[0],
);
isLogsDataSource &&
attributeKeys.some((value) => value.key === tagKey.split(' ')[0]);
if (isSuggestedKeyInAutocomplete) {
const allAttributesMatchingTheKey =
suggestionsData?.payload?.attributes?.filter(
(value) => value.key === tagKey.split(' ')[0],
) || [];
const allAttributesMatchingTheKey = attributeKeys.filter(
(value) => value.key === tagKey.split(' ')[0],
);
if (allAttributesMatchingTheKey?.length === 1) {
[currentRunningAttributeKey] = allAttributesMatchingTheKey;
@@ -648,9 +614,10 @@ function QueryBuilderSearchV2(
currentFilterItem,
currentFilterItem?.key,
currentFilterItem?.op,
suggestionsData?.payload?.attributes,
attributeKeys,
isLogsDataSource,
searchValue,
isFetchingSuggestions,
isFetching,
currentState,
]);
@@ -662,9 +629,7 @@ function QueryBuilderSearchV2(
// add the user typed option in the dropdown to select that and move ahead irrespective of the matches and all
setDropdownOptions([
...(!isEmpty(tagKey) &&
!suggestionsData?.payload?.attributes?.some((val) =>
isEqual(val.key, tagKey),
)
!attributeKeys.some((val) => isEqual(val.key, tagKey))
? [
{
label: tagKey,
@@ -676,10 +641,10 @@ function QueryBuilderSearchV2(
},
]
: []),
...(suggestionsData?.payload?.attributes?.map((key) => ({
...attributeKeys.map((key) => ({
label: key.key,
value: key,
})) || []),
})),
]);
} else if (hardcodedAttributeKeys) {
const filteredKeys = hardcodedAttributeKeys.filter((key) =>
@@ -697,7 +662,7 @@ function QueryBuilderSearchV2(
setDropdownOptions([
// Add user typed option if it doesn't exist in the payload
...(tagKey.trim().length > 0 &&
!data?.payload?.attributeKeys?.some((val) => val.key === tagKey)
!attributeKeys.some((val) => val.key === tagKey)
? [
{
label: tagKey,
@@ -710,10 +675,10 @@ function QueryBuilderSearchV2(
]
: []),
// Map existing attribute keys from payload
...(data?.payload?.attributeKeys?.map((key) => ({
...attributeKeys.map((key) => ({
label: key.key,
value: key,
})) || []),
})),
]);
}
}
@@ -778,10 +743,10 @@ function QueryBuilderSearchV2(
values.push(tagValue);
}
if (attributeValues?.payload) {
const dataType = currentFilterItem?.key?.dataType || DataTypes.String;
const key = DATA_TYPE_VS_ATTRIBUTE_VALUES_KEY[dataType];
values.push(...(attributeValues?.payload?.[key] || []));
if (fieldValues) {
values.push(
...getValuesForDataType(fieldValues, currentFilterItem?.key?.dataType),
);
// here we want to suggest the variable name matching with the key here, we will go over the dynamic variables for the keys
const variableName = dashboardDynamicVariables.find(
@@ -805,13 +770,12 @@ function QueryBuilderSearchV2(
}
}, [
hardcodedAttributeKeys,
attributeValues?.payload,
fieldValues,
currentFilterItem?.key?.dataType,
currentState,
data?.payload?.attributeKeys,
attributeKeys,
isLogsDataSource,
searchValue,
suggestionsData?.payload?.attributes,
operatorConfigKey,
currentFilterItem?.key?.key,
dashboardDynamicVariables,
@@ -880,8 +844,8 @@ function QueryBuilderSearchV2(
}, [deregisterShortcut, isLastQuery, registerShortcut]);
const loading = useMemo(
() => isFetching || isFetchingAttributeValues || isFetchingSuggestions,
[isFetching, isFetchingAttributeValues, isFetchingSuggestions],
() => isFetching || isFetchingAttributeValues,
[isFetching, isFetchingAttributeValues],
);
const isMetricsDataSource = useMemo(
@@ -991,7 +955,7 @@ function QueryBuilderSearchV2(
setTags((prev) => [...prev, ...(val.items as ITag[])]);
}}
searchValue={searchValue}
exampleQueries={suggestionsData?.payload?.example_queries || []}
exampleQueries={[]}
tags={tags}
currentFilterItem={currentFilterItem}
isLogsDataSource={isLogsDataSource}

View File

@@ -1,13 +1,23 @@
import { QueryClient, QueryClientProvider } from 'react-query';
import { render, RenderResult, screen, within } from '@testing-library/react';
import {
render,
RenderResult,
screen,
waitFor,
within,
} from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import {
initialQueriesMap,
initialQueryBuilderFormValues,
} from 'constants/queryBuilder';
import { TelemetrytypesTelemetryFieldValuesDTO } from 'api/generated/services/sigNoz.schemas';
import { DynamicVariableSuggestion } from 'providers/Dashboard/store/dynamicVariableSuggestions';
import { useFieldKeysSuggestion } from 'hooks/querySuggestions/useFieldKeysSuggestion';
import { useFieldValuesSuggestion } from 'hooks/querySuggestions/useFieldValuesSuggestion';
import { QueryBuilderContext } from 'providers/QueryBuilder';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { FieldDataType, TelemetryFieldKey } from 'types/api/v5/queryRange';
import { DataSource } from 'types/common/queryBuilder';
import QueryBuilderSearchV2 from '../QueryBuilderSearchV2';
@@ -88,57 +98,34 @@ const renderWithContext = (props = {}): RenderResult => {
const getSearchCombobox = (): HTMLElement =>
within(screen.getByTestId('qb-search-select')).getByRole('combobox');
// Constants to fix linter errors
const TYPE_TAG = 'tag';
const IS_COLUMN_FALSE = false;
const IS_JSON_FALSE = false;
const mockAggregateKeysData = {
payload: {
attributeKeys: [
{
key: 'http.status',
dataType: DataTypes.String,
type: TYPE_TAG,
id: 'http.status--string--tag--false',
},
{
key: 'service.name',
dataType: DataTypes.String,
type: TYPE_TAG,
isColumn: IS_COLUMN_FALSE,
isJSON: IS_JSON_FALSE,
id: 'service.name--string--tag--false',
},
{
key: 'unmapped.attribute',
dataType: 'not-a-real-data-type' as unknown as DataTypes,
type: TYPE_TAG,
isColumn: IS_COLUMN_FALSE,
isJSON: IS_JSON_FALSE,
id: 'unmapped.attribute--String--tag--false',
},
],
const mockFieldKeys: TelemetryFieldKey[] = [
{ name: 'http.status', fieldContext: 'attribute', fieldDataType: 'string' },
{ name: 'service.name', fieldContext: 'attribute', fieldDataType: 'string' },
{ name: 'name', fieldContext: 'span', fieldDataType: 'string' },
{ name: 'user.id', fieldContext: 'body', fieldDataType: 'string' },
{ name: 'isRoot', fieldContext: 'span', fieldDataType: 'bool' },
{
name: 'unmapped.attribute',
fieldContext: 'attribute',
fieldDataType: 'not-a-real-data-type' as FieldDataType,
},
};
];
jest.mock('hooks/queryBuilder/useGetAggregateKeys', () => ({
useGetAggregateKeys: jest.fn(() => ({
data: mockAggregateKeysData,
jest.mock('hooks/querySuggestions/useFieldKeysSuggestion', () => ({
useFieldKeysSuggestion: jest.fn(() => ({
data: mockFieldKeys,
isFetching: false,
})),
}));
const mockAggregateValuesData = {
payload: {
stringAttributeValues: ['200', '404', '500'],
numberAttributeValues: [200, 404, 500],
},
const mockFieldValues: TelemetrytypesTelemetryFieldValuesDTO = {
stringValues: ['200', '404', '500'],
numberValues: [200, 404, 500],
};
jest.mock('hooks/queryBuilder/useGetAggregateValues', () => ({
useGetAggregateValues: jest.fn(() => ({
data: mockAggregateValuesData,
jest.mock('hooks/querySuggestions/useFieldValuesSuggestion', () => ({
useFieldValuesSuggestion: jest.fn(() => ({
data: mockFieldValues,
isFetching: false,
})),
}));
@@ -196,7 +183,11 @@ describe('Suggestion Key -> Operator -> Value Flow', () => {
expect.objectContaining({
items: expect.arrayContaining([
expect.objectContaining({
key: expect.objectContaining({ key: 'http.status' }),
key: expect.objectContaining({
key: 'http.status',
type: 'attribute',
dataType: 'string',
}),
op: '=',
value: '200',
}),
@@ -292,7 +283,7 @@ describe('Filter chips', () => {
key: {
key: 'http.status',
dataType: DataTypes.String,
type: TYPE_TAG,
type: 'tag',
id: 'http.status--string--tag--false',
},
op: '=',
@@ -332,3 +323,87 @@ describe('Filter chips', () => {
expect(getSearchCombobox()).toHaveDisplayValue(CHIP_LABEL);
});
});
describe('Key fetch context', () => {
it('limits keys to the aggregate attribute context, as the resource filter needs', () => {
renderWithContext({
query: {
...defaultProps.query,
aggregateAttribute: {
...defaultProps.query.aggregateAttribute,
type: 'resource',
},
},
});
expect(useFieldKeysSuggestion).toHaveBeenLastCalledWith(
expect.objectContaining({ fieldContext: 'resource' }),
undefined,
expect.anything(),
);
});
it('fetches keys from every context when the aggregate attribute has no type', () => {
renderWithContext();
expect(useFieldKeysSuggestion).toHaveBeenLastCalledWith(
expect.objectContaining({ fieldContext: undefined }),
undefined,
expect.anything(),
);
});
});
describe('Keys the legacy filter builders cannot read', () => {
it('does not suggest body JSON keys or span search scope selectors', async () => {
const user = userEvent.setup({ delay: null });
renderWithContext();
const combobox = getSearchCombobox();
await user.click(combobox);
await user.type(combobox, 'user');
await screen.findByRole('listbox');
await expect(screen.findByText('http.status')).resolves.toBeInTheDocument();
expect(screen.queryByText('user.id')).not.toBeInTheDocument();
expect(screen.queryByText('isRoot')).not.toBeInTheDocument();
});
});
describe('Span field keys', () => {
it('marks span fields as columns and looks up their values in the span context', async () => {
const user = userEvent.setup({ delay: null });
renderWithContext();
const combobox = getSearchCombobox();
await user.click(combobox);
await user.type(combobox, 'nam');
await screen.findByRole('listbox');
await user.click(await screen.findByText('name'));
await user.click(screen.getByText('='));
await waitFor(() =>
expect(jest.mocked(useFieldValuesSuggestion)).toHaveBeenLastCalledWith(
expect.objectContaining({ name: 'name', fieldContext: 'span' }),
undefined,
expect.anything(),
),
);
await user.click(screen.getByText('200'));
expect(mockOnChange).toHaveBeenLastCalledWith(
expect.objectContaining({
items: expect.arrayContaining([
expect.objectContaining({
key: expect.objectContaining({
key: 'name',
type: 'span',
isColumn: true,
}),
}),
]),
}),
);
});
});

View File

@@ -24,7 +24,6 @@ import { fieldValuesResponse } from '@/storybook/msw/__story_mockdata__/fields';
import {
EXCEPTION_CATALOGUE_SIZE,
EXCEPTION_QUICK_FILTER_CAP,
exceptionAttributeKeysResponse,
exceptionAttributeValuesResponse,
exceptionFieldKeysResponse,
exceptionQuickFiltersResponse,
@@ -58,7 +57,7 @@ export const exceptionsMocks = defineStoryMocks({
filterKeys: choiceControl<ResponseState>('Filter keys', {
group: FILTERS,
description:
'How `/autocomplete/attribute_keys` answers when the resource filter opens, apart from the page-wide Data control.',
'How `/fields/keys` answers when the resource filter opens, apart from the page-wide Data control.',
options: RESPONSE_STATES,
value: 'loaded',
}),
@@ -95,7 +94,7 @@ export const exceptionsMocks = defineStoryMocks({
rest.get(
'http://localhost/api/v1/fields/keys',
response.json((req) =>
respondWith(values.filterKeys, (req) =>
exceptionFieldKeysResponse(req.url.searchParams.get('searchText')),
),
),
@@ -109,23 +108,6 @@ export const exceptionsMocks = defineStoryMocks({
),
),
),
rest.get(
'http://localhost/api/v3/autocomplete/attribute_keys',
respondWith(values.filterKeys, (req) =>
exceptionAttributeKeysResponse(req.url.searchParams.get('searchText')),
),
),
rest.get(
'http://localhost/api/v3/autocomplete/attribute_values',
response.json((req) =>
exceptionAttributeValuesResponse(
req.url.searchParams.get('attributeKey'),
req.url.searchParams.get('searchText'),
),
),
),
],
effect: (values) => {
set(LOCALSTORAGE.SHOW_EXCEPTIONS_QUICK_FILTERS, String(values.filterPanel));

View File

@@ -17,7 +17,6 @@ import type {
import {
type BaseAutocompleteData,
DataTypes,
type IQueryAutocompleteResponse,
} from 'types/api/queryBuilder/queryAutocompleteResponse';
import type { Exception, Order, OrderBy } from 'types/api/errors/getAll';
import type { IAttributeValuesResponse } from 'types/api/queryBuilder/getAttributesValues';
@@ -311,21 +310,6 @@ const ATTRIBUTE_KEYS: BaseAutocompleteData[] = [
{ key: 'cloud.region', dataType: DataTypes.String, type: 'resource' },
];
export const exceptionAttributeKeysResponse = (
searchText: string | null,
): { status: string; data: IQueryAutocompleteResponse } => {
const search = (searchText ?? '').toLowerCase();
return {
status: 'success',
data: {
attributeKeys: ATTRIBUTE_KEYS.filter(({ key }) =>
key.toLowerCase().includes(search),
),
},
};
};
/** The quick filter settings panel lists its "other filters" from `/fields/keys`. */
export const exceptionFieldKeysResponse = (
searchText: string | null,

View File

@@ -9,6 +9,8 @@ import type { QueryRangePayload } from 'types/api/metrics/getQueryRange';
import { choiceControl, countControl } from '@/storybook/controls/controls';
import { defineStoryMocks } from '@/storybook/controls/defineStoryMocks';
import { fieldValuesResponse } from '@/storybook/msw/__story_mockdata__/fields';
import { queryRangeV5ScalarTableResponse } from '@/storybook/msw/__story_mockdata__/queryRange';
import {
attributeValuesResponse,
@@ -75,30 +77,34 @@ export const celeryTaskMocks = defineStoryMocks({
return [
rest.post('http://localhost/api/v3/query_range', queryRange),
rest.post('http://localhost/api/v4/query_range', queryRange),
rest.post(
'http://localhost/api/v5/query_range',
response.json(() =>
queryRangeV5ScalarTableResponse({
groupBy: ['worker'],
aggregations: ['A'],
rows: (
attributeValuesResponse('worker', '').data.stringAttributeValues ?? []
)
.slice(0, values.workers)
.map((worker) => [worker, 12]),
}),
),
),
rest.get(
'http://localhost/api/v3/autocomplete/attribute_values',
'http://localhost/api/v1/fields/values',
response.json((req) => {
const attributeKey = req.url.searchParams.get('attributeKey') ?? '';
const values_ = attributeValuesResponse(
attributeKey,
req.url.searchParams.get('searchText') ?? '',
const name = req.url.searchParams.get('name') ?? '';
const found =
attributeValuesResponse(
name,
req.url.searchParams.get('searchText') ?? '',
).data.stringAttributeValues ?? [];
return fieldValuesResponse(
name === 'worker' ? found.slice(0, values.workers) : found,
);
if (attributeKey !== 'worker') {
return values_;
}
return {
...values_,
data: {
...values_.data,
stringAttributeValues: (values_.data.stringAttributeValues ?? []).slice(
0,
values.workers,
),
},
};
}),
),
];

View File

@@ -13,6 +13,7 @@ import {
toggleControl,
} from '@/storybook/controls/controls';
import { defineStoryMocks } from '@/storybook/controls/defineStoryMocks';
import { fieldValuesResponse } from '@/storybook/msw/__story_mockdata__/fields';
import {
attributeValuesResponse,
@@ -91,11 +92,13 @@ export const queueOverviewMocks = defineStoryMocks({
rest.post('http://localhost/api/v4/query_range', queryRange),
rest.get(
'http://localhost/api/v3/autocomplete/attribute_values',
'http://localhost/api/v1/fields/values',
response.json((req) =>
attributeValuesResponse(
req.url.searchParams.get('attributeKey') ?? '',
req.url.searchParams.get('searchText') ?? '',
fieldValuesResponse(
attributeValuesResponse(
req.url.searchParams.get('name') ?? '',
req.url.searchParams.get('searchText') ?? '',
).data.stringAttributeValues ?? [],
),
),
),

View File

@@ -82,15 +82,15 @@ const CELERY_TASKS = [
const CELERY_WORKERS = ['worker-a', 'worker-b', 'worker-c'];
/**
* Values per autocomplete attribute key. The kafka config selects and both
* celery filter rows read the same v3 endpoint, so one map answers for all of
* Values per attribute key. The kafka config selects read the v3 endpoint and
* the celery filter rows read `/fields/values`, so one map answers for all of
* them; a key nobody seeded answers empty rather than 501.
*/
const ATTRIBUTE_VALUES: Record<string, string[]> = {
group: CONSUMER_GROUPS,
topic: TOPICS,
partition: PARTITIONS,
serviceName: KAFKA_SERVICES,
'service.name': KAFKA_SERVICES,
name: [
'order-events send',
'order-events process',

View File

@@ -3,6 +3,7 @@ import { Button, Form, Space, Tooltip } from 'antd';
import { Dropdown, type DropdownItemType } from '@signozhq/ui/dropdown';
import { Divider } from '@signozhq/ui/divider';
import { Switch } from '@signozhq/ui/switch';
import { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import cx from 'classnames';
import { FilterSelect } from 'components/CeleryOverview/CeleryOverviewConfigOptions/CeleryOverviewConfigOptions';
import { QueryParams } from 'constants/query';
@@ -149,7 +150,8 @@ function FunnelStep({
<FilterSelect
placeholder="Select Service"
queryParam={QueryParams.service}
filterType="serviceName"
filterType="service.name"
fieldContext={TelemetrytypesFieldContextDTO.resource}
shouldSetQueryParams={false}
values={stepData.service_name}
isMultiple={false}
@@ -169,6 +171,7 @@ function FunnelStep({
placeholder="Select Span name"
queryParam={QueryParams.spanName}
filterType="name"
fieldContext={TelemetrytypesFieldContextDTO.span}
shouldSetQueryParams={false}
values={stepData.span_name}
isMultiple={false}

View File

@@ -3,15 +3,19 @@
* Do not hand-edit: regenerate instead.
*/
import {
TelemetrytypesFieldContextDTO,
TelemetrytypesSignalDTO,
} from 'api/generated/services/sigNoz.schemas';
import ROUTES from 'constants/routes';
import { rest } from 'msw';
import { choiceControl, countControl } from '@/storybook/controls/controls';
import { defineStoryMocks } from '@/storybook/controls/defineStoryMocks';
import {
attributeKeysResponse,
attributeValuesResponse,
} from '@/storybook/msw/__story_mockdata__/attributes';
fieldKeysResponse,
fieldValuesResponse,
} from '@/storybook/msw/__story_mockdata__/fields';
import {
FUNNEL_ID,
@@ -41,13 +45,9 @@ const CONVERSION: Record<DropOff, number> = {
const ENTERED = 24_000;
const STEP_FILTER_KEYS = [
'service.name',
'name',
'http.status_code',
'deployment.environment',
'hasError',
];
const STEP_RESOURCE_KEYS = ['service.name', 'deployment.environment'];
const STEP_ATTRIBUTE_KEYS = ['name', 'http.status_code', 'hasError'];
const STEP_FILTER_VALUES: Record<string, string[]> = {
'service.name': ['frontend', 'checkout', 'payment', 'shipping'],
@@ -115,20 +115,44 @@ export const tracesFunnelDetailsMocks = defineStoryMocks({
'http://localhost/api/v1/trace-funnels/analytics/error-traces',
response.json(() => funnelErrorTracesResponse(values.traces)),
),
// The `Where` box on every step is the trace query builder, which reads
// its suggestions as soon as the step renders.
// The `Where` box on every step reads its suggestions as soon as the step
// renders, and the service and span name selects read `service.name` and
// `name` from the same values endpoint.
rest.get(
'http://localhost/api/v3/autocomplete/attribute_keys',
response.json(() => attributeKeysResponse(STEP_FILTER_KEYS)),
'http://localhost/api/v1/fields/keys',
response.json(() => {
const signal = TelemetrytypesSignalDTO.traces;
const resource = fieldKeysResponse(STEP_RESOURCE_KEYS, {
signal,
fieldContext: TelemetrytypesFieldContextDTO.resource,
});
const attribute = fieldKeysResponse(STEP_ATTRIBUTE_KEYS, {
signal,
fieldContext: TelemetrytypesFieldContextDTO.attribute,
});
return {
...resource,
data: {
...resource.data,
keys: { ...resource.data.keys, ...attribute.data.keys },
},
};
}),
),
rest.get(
'http://localhost/api/v3/autocomplete/attribute_values',
response.json((req) =>
attributeValuesResponse(
STEP_FILTER_VALUES[req.url.searchParams.get('attributeKey') ?? ''] ?? [],
req.url.searchParams.get('searchText') ?? '',
),
),
'http://localhost/api/v1/fields/values',
response.json((req) => {
const searchText = (
req.url.searchParams.get('searchText') ?? ''
).toLowerCase();
return fieldValuesResponse(
(STEP_FILTER_VALUES[req.url.searchParams.get('name') ?? ''] ?? []).filter(
(value) => value.toLowerCase().includes(searchText),
),
);
}),
),
];
},

View File

@@ -75,7 +75,9 @@
"logs-field",
"traces-field",
"metrics-field",
"llm-pricing-rule"
"llm-pricing-rule",
"span-mapper-group",
"span-mapper"
],
"type": "string"
},

View File

@@ -53,6 +53,7 @@ export type FieldContext =
| 'scope'
| 'attribute'
| 'event'
| 'body'
| '';
export type FieldDataType =

View File

@@ -0,0 +1,76 @@
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { TagFilter } from 'types/api/queryBuilder/queryBuilderData';
import {
attributeTypeToFieldContext,
toLegacyLogsFilter,
} from '../fieldContext';
const item = (type: string): TagFilter['items'][number] => ({
id: type,
key: { key: 'k', dataType: DataTypes.String, type },
op: '=',
value: 'v',
});
describe('toLegacyLogsFilter', () => {
it('sends attribute keys as tag and log fields as top-level fields', () => {
const filter: TagFilter = {
op: 'AND',
items: [
item('attribute'),
item('resource'),
item('scope'),
item('log'),
item('body'),
item('tag'),
item(''),
],
};
expect(
toLegacyLogsFilter(filter).items.map((filterItem) => filterItem.key?.type),
).toStrictEqual(['tag', 'resource', 'scope', '', '', 'tag', '']);
});
it('sends the scope name and version as top-level fields only from the scope context', () => {
const filter: TagFilter = {
op: 'AND',
items: [
['scope_name', 'scope'],
['scope_version', 'scope'],
['scope.attr', 'scope'],
['scope_name', 'attribute'],
['scope_name', 'tag'],
['scope_version', 'resource'],
].map(([key, type]) => ({
id: `${key}-${type}`,
key: { key, dataType: DataTypes.String, type },
op: '=',
value: 'v',
})),
};
expect(
toLegacyLogsFilter(filter).items.map((filterItem) => filterItem.key?.type),
).toStrictEqual(['', '', 'scope', 'tag', 'tag', 'resource']);
});
});
describe('attributeTypeToFieldContext', () => {
it('maps legacy tag to attribute and passes v5 contexts through', () => {
expect(
['tag', 'attribute', 'resource', 'scope', 'span', 'log', ''].map(
attributeTypeToFieldContext,
),
).toStrictEqual([
'attribute',
'attribute',
'resource',
'scope',
'span',
'log',
undefined,
]);
});
});

View File

@@ -0,0 +1,55 @@
import { TelemetrytypesFieldContextDTO } from 'api/generated/services/sigNoz.schemas';
import type { TagFilter } from 'types/api/queryBuilder/queryBuilderData';
const FIELD_CONTEXT_TO_ATTRIBUTE_TYPE: Record<string, string> = {
[TelemetrytypesFieldContextDTO.attribute]: 'tag',
[TelemetrytypesFieldContextDTO.resource]: 'resource',
};
const ATTRIBUTE_TYPE_TO_FIELD_CONTEXT: Record<
string,
TelemetrytypesFieldContextDTO
> = {
tag: TelemetrytypesFieldContextDTO.attribute,
attribute: TelemetrytypesFieldContextDTO.attribute,
resource: TelemetrytypesFieldContextDTO.resource,
scope: TelemetrytypesFieldContextDTO.scope,
span: TelemetrytypesFieldContextDTO.span,
log: TelemetrytypesFieldContextDTO.log,
};
// The legacy logs builders read `tag` as attributes and an empty type as a top-level field;
// any other type falls through to resources.
const LEGACY_LOGS_KEY_TYPE: Record<string, string> = {
[TelemetrytypesFieldContextDTO.attribute]: 'tag',
tag: 'tag',
[TelemetrytypesFieldContextDTO.resource]: 'resource',
[TelemetrytypesFieldContextDTO.scope]: 'scope',
};
// Top-level log columns that v5 files under the scope context.
const LEGACY_LOGS_TOP_LEVEL_SCOPE_FIELDS = ['scope_name', 'scope_version'];
export const fieldContextToAttributeType = (fieldContext?: string): string =>
(fieldContext && FIELD_CONTEXT_TO_ATTRIBUTE_TYPE[fieldContext]) || '';
export const attributeTypeToFieldContext = (
type?: string | null,
): TelemetrytypesFieldContextDTO | undefined =>
type ? ATTRIBUTE_TYPE_TO_FIELD_CONTEXT[type] : undefined;
export const toLegacyLogsFilter = (filter: TagFilter): TagFilter =>
filter && {
...filter,
items: (filter.items ?? []).map((item) => ({
...item,
key: item.key && {
...item.key,
type:
item.key.type === TelemetrytypesFieldContextDTO.scope &&
LEGACY_LOGS_TOP_LEVEL_SCOPE_FIELDS.includes(item.key.key)
? ''
: (LEGACY_LOGS_KEY_TYPE[item.key.type ?? ''] ?? ''),
},
})),
};

View File

@@ -0,0 +1,21 @@
import { TelemetrytypesTelemetryFieldValuesDTO } from 'api/generated/services/sigNoz.schemas';
import { DataTypes } from 'types/api/queryBuilder/queryAutocompleteResponse';
// Numbers and bools stay raw so tags keep their value type.
export const getValuesForDataType = (
values: TelemetrytypesTelemetryFieldValuesDTO | undefined,
dataType: DataTypes | undefined,
): string[] => {
switch (dataType) {
case DataTypes.Int64:
case DataTypes.Float64:
case DataTypes.ArrayInt64:
case DataTypes.ArrayFloat64:
return (values?.numberValues ?? []) as unknown as string[];
case DataTypes.bool:
case DataTypes.ArrayBool:
return (values?.boolValues ?? []) as unknown as string[];
default:
return values?.stringValues ?? [];
}
};

View File

@@ -3,15 +3,17 @@ package signozapiserver
import (
"net/http"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/spantypes"
"github.com/gorilla/mux"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/coretypes"
"github.com/SigNoz/signoz/pkg/types/spantypes"
)
func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
if err := router.Handle("/api/v1/span_mapper_groups", handler.New(
provider.authzMiddleware.ViewAccess(provider.spanMapperHandler.ListGroups),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.ListGroups, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "ListSpanMapperGroups",
Tags: []string{"spanmapper"},
@@ -25,14 +27,20 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbList)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapperGroup,
Verb: coretypes.VerbList,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}),
)).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups", handler.New(
provider.authzMiddleware.AdminAccess(provider.spanMapperHandler.CreateGroup),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.CreateGroup, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "CreateSpanMapperGroup",
Tags: []string{"spanmapper"},
@@ -45,19 +53,26 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusCreated,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusConflict},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbCreate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapperGroup,
Verb: coretypes.VerbCreate,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.ResponseJSONPath("data.id"),
Selector: coretypes.WildcardSelector,
}),
)).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups/test", handler.New(
provider.authzMiddleware.ViewAccess(provider.spanMapperHandler.TestMappers),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.TestMappers, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "TestSpanMappers",
Tags: []string{"spanmapper"},
Summary: "Test span mappers against sample spans",
Description: "Tests how span mappers would transform sample spans",
Description: "Tests how span mappers would transform sample spans. Groups sent without mappers are read from the store by name.",
Request: new(spantypes.PostableSpanMapperTest),
RequestContentType: "application/json",
Response: new(spantypes.GettableSpanMapperTest),
@@ -65,14 +80,20 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbRead)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapperGroup,
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}),
)).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups/{groupId}", handler.New(
provider.authzMiddleware.AdminAccess(provider.spanMapperHandler.UpdateGroup),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.UpdateGroup, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "UpdateSpanMapperGroup",
Tags: []string{"spanmapper"},
@@ -83,14 +104,21 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbUpdate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapperGroup,
Verb: coretypes.VerbUpdate,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.PathParam("groupId"),
Selector: coretypes.IDSelector,
}),
)).Methods(http.MethodPatch).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups/{groupId}", handler.New(
provider.authzMiddleware.AdminAccess(provider.spanMapperHandler.DeleteGroup),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.DeleteGroup, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "DeleteSpanMapperGroup",
Tags: []string{"spanmapper"},
@@ -103,14 +131,21 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbDelete)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapperGroup,
Verb: coretypes.VerbDelete,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.PathParam("groupId"),
Selector: coretypes.IDSelector,
}),
)).Methods(http.MethodDelete).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups/{groupId}/span_mappers", handler.New(
provider.authzMiddleware.ViewAccess(provider.spanMapperHandler.ListMappers),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.ListMappers, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "ListSpanMappers",
Tags: []string{"spanmapper"},
@@ -123,14 +158,21 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbRead)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapperGroup,
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
ID: coretypes.PathParam("groupId"),
Selector: coretypes.IDSelector,
}),
)).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups/{groupId}/span_mappers", handler.New(
provider.authzMiddleware.AdminAccess(provider.spanMapperHandler.CreateMapper),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.CreateMapper, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "CreateSpanMapper",
Tags: []string{"spanmapper"},
@@ -143,14 +185,32 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusCreated,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound, http.StatusConflict},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapper.Scope(coretypes.VerbCreate), coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbAttach)}),
},
handler.WithResourceDefs(
handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapper,
Verb: coretypes.VerbCreate,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.ResponseJSONPath("data.id"),
Selector: coretypes.WildcardSelector,
},
handler.AttachDetachParentChildResourceDef{
Verb: coretypes.VerbAttach,
Category: coretypes.ActionCategoryConfigurationChange,
ParentResource: coretypes.ResourceMetaResourceSpanMapperGroup,
ParentID: coretypes.PathParam("groupId"),
ParentSelector: coretypes.IDSelector,
ChildResource: coretypes.ResourceMetaResourceSpanMapper,
ChildIDs: coretypes.OneID(coretypes.ResponseJSONPath("data.id")),
},
),
)).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups/{groupId}/span_mappers/{mapperId}", handler.New(
provider.authzMiddleware.AdminAccess(provider.spanMapperHandler.UpdateMapper),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.UpdateMapper, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "UpdateSpanMapper",
Tags: []string{"spanmapper"},
@@ -161,14 +221,30 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapper.Scope(coretypes.VerbUpdate), coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbUpdate)}),
},
handler.WithResourceDefs(
handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapper,
Verb: coretypes.VerbUpdate,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.PathParam("mapperId"),
Selector: coretypes.IDSelector,
},
handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapperGroup,
Verb: coretypes.VerbUpdate,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.PathParam("groupId"),
Selector: coretypes.IDSelector,
},
),
)).Methods(http.MethodPatch).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/span_mapper_groups/{groupId}/span_mappers/{mapperId}", handler.New(
provider.authzMiddleware.AdminAccess(provider.spanMapperHandler.DeleteMapper),
provider.authzMiddleware.CheckResources(provider.spanMapperHandler.DeleteMapper, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "DeleteSpanMapper",
Tags: []string{"spanmapper"},
@@ -181,8 +257,26 @@ func (provider *provider) addSpanMapperRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusNotFound},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSpanMapper.Scope(coretypes.VerbDelete), coretypes.ResourceMetaResourceSpanMapperGroup.Scope(coretypes.VerbDetach)}),
},
handler.WithResourceDefs(
handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceSpanMapper,
Verb: coretypes.VerbDelete,
Category: coretypes.ActionCategoryConfigurationChange,
ID: coretypes.PathParam("mapperId"),
Selector: coretypes.IDSelector,
},
handler.AttachDetachParentChildResourceDef{
Verb: coretypes.VerbDetach,
Category: coretypes.ActionCategoryConfigurationChange,
ParentResource: coretypes.ResourceMetaResourceSpanMapperGroup,
ParentID: coretypes.PathParam("groupId"),
ParentSelector: coretypes.IDSelector,
ChildResource: coretypes.ResourceMetaResourceSpanMapper,
ChildIDs: coretypes.OneID(coretypes.PathParam("mapperId")),
},
),
)).Methods(http.MethodDelete).GetError(); err != nil {
return err
}

View File

@@ -275,6 +275,7 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewMigrateRulesV1ToV2Alpha1Factory(sqlstore),
sqlmigration.NewAddLLMPricingRuleTuplesFactory(sqlstore),
sqlmigration.NewRemoveUnsupportedQuickFiltersFactory(),
sqlmigration.NewAddSpanMapperTuplesFactory(sqlstore),
)
}

View File

@@ -6,14 +6,15 @@ import (
"encoding/json"
"time"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/coretypes"
"github.com/oklog/ulid/v2"
"github.com/uptrace/bun"
"github.com/uptrace/bun/dialect"
"github.com/uptrace/bun/migrate"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/coretypes"
)
type addLLMPricingRuleTuples struct {

View File

@@ -0,0 +1,229 @@
package sqlmigration
import (
"context"
"database/sql"
"encoding/json"
"slices"
"time"
"github.com/oklog/ulid/v2"
"github.com/uptrace/bun"
"github.com/uptrace/bun/dialect"
"github.com/uptrace/bun/migrate"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
)
type addSpanMapperTuples struct {
sqlstore sqlstore.SQLStore
}
func NewAddSpanMapperTuplesFactory(sqlstore sqlstore.SQLStore) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(factory.MustNewName("add_span_mapper_tuples"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &addSpanMapperTuples{sqlstore: sqlstore}, nil
})
}
func (migration *addSpanMapperTuples) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
func (migration *addSpanMapperTuples) Up(ctx context.Context, db *bun.DB) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var storeID string
err = tx.QueryRowContext(ctx, `SELECT id FROM store WHERE name = ? LIMIT 1`, "signoz").Scan(&storeID)
if err != nil {
return err
}
var orgIDs []string
err = tx.NewSelect().
Table("organizations").
Column("id").
Scan(ctx, &orgIDs)
if err != nil && err != sql.ErrNoRows {
return err
}
isPG := migration.sqlstore.BunDB().Dialect().Name() == dialect.PG
tuples := []migrationTuple{
{"signoz-admin", "metaresource", "span-mapper-group", "read"},
{"signoz-admin", "metaresource", "span-mapper-group", "update"},
{"signoz-admin", "metaresource", "span-mapper-group", "delete"},
{"signoz-admin", "metaresource", "span-mapper-group", "create"},
{"signoz-admin", "metaresource", "span-mapper-group", "list"},
{"signoz-admin", "metaresource", "span-mapper-group", "attach"},
{"signoz-admin", "metaresource", "span-mapper-group", "detach"},
{"signoz-admin", "metaresource", "span-mapper", "update"},
{"signoz-admin", "metaresource", "span-mapper", "delete"},
{"signoz-admin", "metaresource", "span-mapper", "create"},
{"signoz-editor", "metaresource", "span-mapper-group", "read"},
{"signoz-editor", "metaresource", "span-mapper-group", "list"},
{"signoz-viewer", "metaresource", "span-mapper-group", "read"},
{"signoz-viewer", "metaresource", "span-mapper-group", "list"},
}
for _, orgID := range orgIDs {
for _, tuple := range tuples {
entropy := ulid.DefaultEntropy()
now := time.Now().UTC()
tupleID := ulid.MustNew(ulid.Timestamp(now), entropy).String()
objectID := "organization/" + orgID + "/" + tuple.objectName + "/*"
roleSubject := "organization/" + orgID + "/role/" + tuple.roleName
if isPG {
user := "role:" + roleSubject + "#assignee"
result, err := tx.ExecContext(ctx, `
INSERT INTO tuple (store, object_type, object_id, relation, _user, user_type, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, object_type, object_id, relation, _user) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, user, "userset", tupleID, now,
)
if err != nil {
return err
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
continue
}
_, err = tx.ExecContext(ctx, `
INSERT INTO changelog (store, object_type, object_id, relation, _user, operation, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, user, 0, tupleID, now,
)
if err != nil {
return err
}
} else {
result, err := tx.ExecContext(ctx, `
INSERT INTO tuple (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, user_type, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", "userset", tupleID, now,
)
if err != nil {
return err
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
continue
}
_, err = tx.ExecContext(ctx, `
INSERT INTO changelog (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, operation, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", 0, tupleID, now,
)
if err != nil {
return err
}
}
}
}
for _, orgID := range orgIDs {
if err := addManagedRoleTransactionGroups(ctx, tx, orgID, tuples); err != nil {
return err
}
}
return tx.Commit()
}
func (migration *addSpanMapperTuples) Down(context.Context, *bun.DB) error {
return nil
}
type migrationTransactionGroup struct {
Relation string `json:"relation"`
ObjectGroup migrationObjectGroup `json:"objectGroup"`
}
type migrationObjectGroup struct {
Resource migrationObjectResource `json:"resource"`
Selectors []string `json:"selectors"`
}
type migrationObjectResource struct {
Type string `json:"type"`
Kind string `json:"kind"`
}
// addManagedRoleTransactionGroups merges each tuple as a wildcard grant into its managed role's stored transaction groups.
func addManagedRoleTransactionGroups(ctx context.Context, tx bun.Tx, orgID string, tuples []migrationTuple) error {
tuplesByRole := make(map[string][]migrationTuple)
for _, tuple := range tuples {
tuplesByRole[tuple.roleName] = append(tuplesByRole[tuple.roleName], tuple)
}
for roleName, roleTuples := range tuplesByRole {
var data sql.NullString
err := tx.NewSelect().
Table("role").
Column("transaction_groups").
Where("org_id = ?", orgID).
Where("type = ?", "managed").
Where("name = ?", roleName).
Scan(ctx, &data)
if err == sql.ErrNoRows || (err == nil && !data.Valid) {
continue
}
if err != nil {
return err
}
var groups []*migrationTransactionGroup
if err := json.Unmarshal([]byte(data.String), &groups); err != nil {
return err
}
for _, tuple := range roleTuples {
index := slices.IndexFunc(groups, func(group *migrationTransactionGroup) bool {
return group.Relation == tuple.relation && group.ObjectGroup.Resource.Type == tuple.objectType && group.ObjectGroup.Resource.Kind == tuple.objectName
})
if index == -1 {
groups = append(groups, &migrationTransactionGroup{
Relation: tuple.relation,
ObjectGroup: migrationObjectGroup{Resource: migrationObjectResource{Type: tuple.objectType, Kind: tuple.objectName}, Selectors: []string{"*"}},
})
continue
}
if !slices.Contains(groups[index].ObjectGroup.Selectors, "*") {
groups[index].ObjectGroup.Selectors = append(groups[index].ObjectGroup.Selectors, "*")
}
}
updated, err := json.Marshal(groups)
if err != nil {
return err
}
if _, err := tx.NewUpdate().
Table("role").
Set("transaction_groups = ?", string(updated)).
Where("org_id = ?", orgID).
Where("type = ?", "managed").
Where("name = ?", roleName).
Exec(ctx); err != nil {
return err
}
}
return nil
}

View File

@@ -81,6 +81,8 @@ func (name Kind) Enum() []any {
KindTracesField,
KindMetricsField,
KindLLMPricingRule,
KindSpanMapperGroup,
KindSpanMapper,
}
}

View File

@@ -41,6 +41,8 @@ var Kinds = []Kind{
KindTracesField,
KindMetricsField,
KindLLMPricingRule,
KindSpanMapperGroup,
KindSpanMapper,
}
var (
@@ -84,4 +86,6 @@ var (
KindTracesField = MustNewKind("traces-field")
KindMetricsField = MustNewKind("metrics-field")
KindLLMPricingRule = MustNewKind("llm-pricing-rule")
KindSpanMapperGroup = MustNewKind("span-mapper-group")
KindSpanMapper = MustNewKind("span-mapper")
)

View File

@@ -231,6 +231,17 @@ var ManagedRoleToTransactions = map[string][]Transaction{
{Verb: VerbUpdate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindLLMPricingRule}, WildCardSelectorString)},
{Verb: VerbDelete, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindLLMPricingRule}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindLLMPricingRule}, WildCardSelectorString)},
// span-mapper-group and span-mapper — admin manages, viewer reads
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbUpdate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbDelete, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbCreate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbAttach, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbDetach, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbUpdate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapper}, WildCardSelectorString)},
{Verb: VerbDelete, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapper}, WildCardSelectorString)},
{Verb: VerbCreate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapper}, WildCardSelectorString)},
},
SigNozEditorRoleName: {
// cloud-integration — editor can read accounts
@@ -339,6 +350,9 @@ var ManagedRoleToTransactions = map[string][]Transaction{
// llm-pricing-rule — read only (admin writes)
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindLLMPricingRule}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindLLMPricingRule}, WildCardSelectorString)},
// span-mapper-group — read only (admin manages)
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
},
SigNozViewerRoleName: {
// cloud-integration — viewer can read accounts
@@ -406,6 +420,9 @@ var ManagedRoleToTransactions = map[string][]Transaction{
// llm-pricing-rule — read only
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindLLMPricingRule}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindLLMPricingRule}, WildCardSelectorString)},
// span-mapper-group — read only
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindSpanMapperGroup}, WildCardSelectorString)},
},
SigNozAnonymousRoleName: {
// public-dashboard — anonymous read

View File

@@ -41,6 +41,8 @@ var Resources = []Resource{
ResourceMetaResourceTracesField,
ResourceMetaResourceMetricsField,
ResourceMetaResourceLLMPricingRule,
ResourceMetaResourceSpanMapperGroup,
ResourceMetaResourceSpanMapper,
}
var (
@@ -84,4 +86,6 @@ var (
ResourceMetaResourceTracesField = NewResourceMetaResource(KindTracesField)
ResourceMetaResourceMetricsField = NewResourceMetaResource(KindMetricsField, VerbList, VerbRead, VerbUpdate)
ResourceMetaResourceLLMPricingRule = NewResourceMetaResource(KindLLMPricingRule, VerbList, VerbRead, VerbUpdate, VerbDelete)
ResourceMetaResourceSpanMapperGroup = NewResourceMetaResource(KindSpanMapperGroup)
ResourceMetaResourceSpanMapper = NewResourceMetaResource(KindSpanMapper, VerbCreate, VerbUpdate, VerbDelete)
)

View File

@@ -315,6 +315,28 @@
"list"
]
},
{
"type": "metaresource",
"kind": "span-mapper-group",
"verbs": [
"read",
"update",
"delete",
"create",
"list",
"attach",
"detach"
]
},
{
"type": "metaresource",
"kind": "span-mapper",
"verbs": [
"update",
"delete",
"create"
]
},
{
"type": "metaresource",
"kind": "ttl-setting",
@@ -560,6 +582,14 @@
"list"
]
},
{
"type": "metaresource",
"kind": "span-mapper-group",
"verbs": [
"read",
"list"
]
},
{
"type": "metaresource",
"kind": "ttl-setting",
@@ -747,6 +777,14 @@
"list"
]
},
{
"type": "metaresource",
"kind": "span-mapper-group",
"verbs": [
"read",
"list"
]
},
{
"type": "metaresource",
"kind": "ttl-setting",

View File

@@ -0,0 +1,353 @@
from collections.abc import Callable
from http import HTTPStatus
import pytest
import requests
from wiremock.resources.mappings import Mapping
from fixtures.auth import (
USER_ADMIN_EMAIL,
USER_ADMIN_PASSWORD,
add_license,
change_user_role,
create_active_user,
find_user_by_email,
)
from fixtures.role import find_role_by_name, transaction_group
from fixtures.types import Operation, SigNoz, TestContainerDocker
GROUPS_PATH = "/api/v1/span_mapper_groups"
_EDITOR_EMAIL = "editor+spanmapperauthz@integration.test"
_VIEWER_EMAIL = "viewer+spanmapperauthz@integration.test"
_ACTOR_ROLE_NAME = "span-mapper-fga-actor"
_ACTOR_EMAIL = "customrole+spanmapperauthz@integration.test"
_PASSWORD = "password123Z$"
# Instance verbs are granted on _TARGET_A and its mapper only; _TARGET_B must stay forbidden.
_TARGET_A = "span-mapper-authz-target-a"
_TARGET_B = "span-mapper-authz-target-b"
_MAPPER = {
"name": "span_mapper_authz.target",
"fieldContext": "attribute",
"config": {"sources": [{"key": "authz.source", "context": "attribute", "operation": "copy", "priority": 1, "enabled": True}]},
"enabled": True,
}
_SIMULATE = {
"spans": [{"attributes": {"authz.source": "x"}, "resource": {}}],
"groups": [{"name": _TARGET_A, "condition": {"attributes": [{"value": "authz.a", "enabled": True}], "resource": []}, "enabled": True}],
}
def test_setup_users_and_targets(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
headers = {"Authorization": f"Bearer {admin_token}"}
# A --reuse stack may already hold part of this state; create only what is missing.
response = requests.get(signoz.self.host_configs["8080"].get("/api/v2/users"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
existing_emails = {user["email"] for user in response.json()["data"]}
for email, role in ((_EDITOR_EMAIL, "signoz-editor"), (_VIEWER_EMAIL, "signoz-viewer")):
if email not in existing_emails:
create_active_user(signoz, admin_token, email=email, role=role, password=_PASSWORD)
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
existing_groups = {group["name"] for group in response.json()["data"]["items"]}
for name, substring in ((_TARGET_A, "authz.a"), (_TARGET_B, "authz.b")):
if name in existing_groups:
continue
response = requests.post(
signoz.self.host_configs["8080"].get(GROUPS_PATH),
json={"name": name, "condition": {"attributes": [{"value": substring, "enabled": True}], "resource": []}, "enabled": True},
headers=headers,
timeout=5,
)
assert response.status_code == HTTPStatus.CREATED, response.text
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
ids = {group["name"]: group["id"] for group in response.json()["data"]["items"]}
for name in (_TARGET_A, _TARGET_B):
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[name]}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
if any(mapper["name"] == _MAPPER["name"] for mapper in response.json()["data"]["items"]):
continue
response = requests.post(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[name]}/span_mappers"), json=_MAPPER, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.CREATED, response.text
def test_mapper_is_only_reachable_through_its_own_group(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
# The authz check on the mapper routes trusts the group id in the path, so the
# store must refuse a mapper addressed through another group.
headers = {"Authorization": f"Bearer {get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)}"}
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
ids = {group["name"]: group["id"] for group in response.json()["data"]["items"]}
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
mapper_b = response.json()["data"]["items"][0]["id"]
response = requests.patch(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers/{mapper_b}"), json={"enabled": True}, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NOT_FOUND, f"update B's mapper via A: expected 404, got {response.status_code}: {response.text}"
response = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers/{mapper_b}"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NOT_FOUND, f"delete B's mapper via A: expected 404, got {response.status_code}: {response.text}"
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
assert {mapper["id"] for mapper in response.json()["data"]["items"]} == {mapper_b}
@pytest.mark.parametrize("email", [_VIEWER_EMAIL, _EDITOR_EMAIL], ids=["viewer", "editor"])
def test_managed_readers_allowed_on_reads_and_simulate(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
email: str,
):
headers = {"Authorization": f"Bearer {get_token(email, _PASSWORD)}"}
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
target_a = {group["name"]: group["id"] for group in response.json()["data"]["items"]}[_TARGET_A]
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{target_a}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
assert {mapper["name"] for mapper in response.json()["data"]["items"]} == {_MAPPER["name"]}
response = requests.post(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/test"), json=_SIMULATE, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
@pytest.mark.parametrize("email", [_VIEWER_EMAIL, _EDITOR_EMAIL], ids=["viewer", "editor"])
def test_managed_readers_forbidden_on_mutations(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
email: str,
):
headers = {"Authorization": f"Bearer {get_token(email, _PASSWORD)}"}
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
target_a = {group["name"]: group["id"] for group in response.json()["data"]["items"]}[_TARGET_A]
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{target_a}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
mapper_id = response.json()["data"]["items"][0]["id"]
for method, path, body in (
("POST", GROUPS_PATH, {"name": "span-mapper-authz-denied", "condition": {"attributes": [{"value": "authz.denied", "enabled": True}], "resource": []}, "enabled": True}),
("PATCH", f"{GROUPS_PATH}/{target_a}", {"enabled": True}),
("DELETE", f"{GROUPS_PATH}/{target_a}", None),
("POST", f"{GROUPS_PATH}/{target_a}/span_mappers", {**_MAPPER, "name": "span_mapper_authz.denied"}),
("PATCH", f"{GROUPS_PATH}/{target_a}/span_mappers/{mapper_id}", {"enabled": True}),
("DELETE", f"{GROUPS_PATH}/{target_a}/span_mappers/{mapper_id}", None),
):
response = requests.request(method, signoz.self.host_configs["8080"].get(path), json=body, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"{method} {path}: expected 403, got {response.status_code}: {response.text}"
def test_apply_license(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
) -> None:
add_license(signoz, make_http_mocks, get_token)
def test_setup_scoped_actor(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
create_role: Callable[..., str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
headers = {"Authorization": f"Bearer {admin_token}"}
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
target_a = {group["name"]: group["id"] for group in response.json()["data"]["items"]}[_TARGET_A]
# Adding a mapper needs create on the collection plus attach on the group;
# removing one needs delete on the mapper plus detach on the group; editing
# one needs update on both.
response = requests.get(signoz.self.host_configs["8080"].get("/api/v1/roles"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
if not any(role["name"] == _ACTOR_ROLE_NAME for role in response.json()["data"]):
create_role(
admin_token,
_ACTOR_ROLE_NAME,
[
transaction_group("list", "metaresource", "span-mapper-group", ["*"]),
transaction_group("read", "metaresource", "span-mapper-group", [target_a]),
transaction_group("update", "metaresource", "span-mapper-group", [target_a]),
transaction_group("attach", "metaresource", "span-mapper-group", [target_a]),
transaction_group("detach", "metaresource", "span-mapper-group", [target_a]),
transaction_group("create", "metaresource", "span-mapper", ["*"]),
transaction_group("delete", "metaresource", "span-mapper", ["*"]),
transaction_group("update", "metaresource", "span-mapper", ["*"]),
],
)
response = requests.get(signoz.self.host_configs["8080"].get("/api/v2/users"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
actor = next((user for user in response.json()["data"] if user["email"] == _ACTOR_EMAIL), None)
user_id = actor["id"] if actor else create_active_user(signoz, admin_token, email=_ACTOR_EMAIL, role="signoz-viewer", password=_PASSWORD)
response = requests.get(signoz.self.host_configs["8080"].get(f"/api/v2/users/{user_id}"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
if not any(ur["role"]["name"] == _ACTOR_ROLE_NAME for ur in response.json()["data"]["userRoles"]):
change_user_role(signoz, admin_token, user_id, "signoz-viewer", _ACTOR_ROLE_NAME)
def test_access_scoped_to_granted_group_and_mapper(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
headers = {"Authorization": f"Bearer {get_token(_ACTOR_EMAIL, _PASSWORD)}"}
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
ids = {group["name"]: group["id"] for group in response.json()["data"]["items"]}
assert {_TARGET_A, _TARGET_B} <= set(ids)
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, f"list granted group's mappers: {response.text}"
mapper_a = response.json()["data"]["items"][0]["id"]
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"list other group's mappers: expected 403, got {response.status_code}: {response.text}"
# Simulate reads saved groups by name, so it needs read on every group.
response = requests.post(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/test"), json=_SIMULATE, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"simulate: expected 403, got {response.status_code}: {response.text}"
response = requests.patch(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}"), json={"enabled": True}, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NO_CONTENT, f"update granted group: {response.text}"
response = requests.patch(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}"), json={"enabled": True}, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"update other group: expected 403, got {response.status_code}: {response.text}"
response = requests.post(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers"),
json={**_MAPPER, "name": "span_mapper_authz.actor"},
headers=headers,
timeout=5,
)
assert response.status_code == HTTPStatus.CREATED, f"create mapper in granted group: {response.text}"
created = response.json()["data"]["id"]
response = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers/{created}"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NO_CONTENT, f"delete mapper in granted group: {response.text}"
response = requests.post(
signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}/span_mappers"),
json={**_MAPPER, "name": "span_mapper_authz.actor"},
headers=headers,
timeout=5,
)
assert response.status_code == HTTPStatus.FORBIDDEN, f"create mapper in other group: expected 403, got {response.status_code}: {response.text}"
response = requests.patch(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers/{mapper_a}"), json={"enabled": True}, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NO_CONTENT, f"update granted mapper: {response.text}"
admin_headers = {"Authorization": f"Bearer {get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)}"}
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}/span_mappers"), headers=admin_headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
mapper_b = response.json()["data"]["items"][0]["id"]
response = requests.patch(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}/span_mappers/{mapper_b}"), json={"enabled": True}, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"update other mapper: expected 403, got {response.status_code}: {response.text}"
response = requests.patch(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers/{mapper_b}"), json={"enabled": True}, headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NOT_FOUND, f"update B's mapper via A: expected 404, got {response.status_code}: {response.text}"
response = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_B]}/span_mappers/{mapper_b}"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"delete other mapper: expected 403, got {response.status_code}: {response.text}"
# detach on A plus delete on "*" passes authz for B's mapper addressed through
# A's path; the store must still refuse it.
response = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}/span_mappers/{mapper_b}"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NOT_FOUND, f"delete B's mapper via A: expected 404, got {response.status_code}: {response.text}"
response = requests.post(
signoz.self.host_configs["8080"].get(GROUPS_PATH),
json={"name": "span-mapper-authz-actor", "condition": {"attributes": [{"value": "authz.actor", "enabled": True}], "resource": []}, "enabled": True},
headers=headers,
timeout=5,
)
assert response.status_code == HTTPStatus.FORBIDDEN, f"create group: expected 403, got {response.status_code}: {response.text}"
response = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[_TARGET_A]}"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"delete granted group: expected 403, got {response.status_code}: {response.text}"
def test_revoke_group_read(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
admin_headers = {"Authorization": f"Bearer {admin_token}"}
headers = {"Authorization": f"Bearer {get_token(_ACTOR_EMAIL, _PASSWORD)}"}
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=admin_headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
target_a = {group["name"]: group["id"] for group in response.json()["data"]["items"]}[_TARGET_A]
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{target_a}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, f"read before revoke: {response.text}"
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/roles/{find_role_by_name(signoz, admin_token, _ACTOR_ROLE_NAME)}"),
json={"description": "", "transactionGroups": [transaction_group("list", "metaresource", "span-mapper-group", ["*"])]},
headers=admin_headers,
timeout=5,
)
assert response.status_code == HTTPStatus.NO_CONTENT, response.text
response = requests.get(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{target_a}/span_mappers"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.FORBIDDEN, f"read after revoke: expected 403, got {response.status_code}: {response.text}"
def test_span_mapper_authz_cleanup(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
headers = {"Authorization": f"Bearer {admin_token}"}
actor = find_user_by_email(signoz, admin_token, _ACTOR_EMAIL)
# Back to viewer so a rerun on the same stack can set the actor up again.
change_user_role(signoz, admin_token, actor["id"], _ACTOR_ROLE_NAME, "signoz-viewer")
response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v1/roles/{find_role_by_name(signoz, admin_token, _ACTOR_ROLE_NAME)}"),
headers=headers,
timeout=5,
)
assert response.status_code == HTTPStatus.NO_CONTENT, f"delete {_ACTOR_ROLE_NAME}: {response.text}"
response = requests.get(signoz.self.host_configs["8080"].get(GROUPS_PATH), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.OK, response.text
ids = {group["name"]: group["id"] for group in response.json()["data"]["items"]}
for name in (_TARGET_A, _TARGET_B):
response = requests.delete(signoz.self.host_configs["8080"].get(f"{GROUPS_PATH}/{ids[name]}"), headers=headers, timeout=5)
assert response.status_code == HTTPStatus.NO_CONTENT, f"delete {name}: {response.text}"