Compare commits

...

20 Commits

Author SHA1 Message Date
Nikhil Soni
7e01566687 refactor(qb): skip non-candidate evolutions in place instead of filtering a copy
Assisted-by: Claude Opus 5.5
2026-09-24 22:40:06 +05:30
Nikhil Soni
1cbd6f3fe2 refactor(qb): ignore evolutions of non-candidate columns in evolution selection
The column mapper decides the candidate columns, so an evolution for a column it did not return is not an error.

Assisted-by: Claude Opus 5.5
2026-09-24 22:40:06 +05:30
Nikhil Soni
b1cce4d6a7 feat(traces-qb): gate JSON span attribute reads behind use_trace_attributes_json
The evolution entry is deployment-wide, so a per-deployment flag controls the rollout.

Assisted-by: Claude Opus 5.5
2026-09-24 22:40:06 +05:30
Nikhil Soni
00932bc04e refactor(promote): group target constructors; drop redundant and unsupported-feature tests
- move NewTargetFromPath next to the other target constructors
- drop TestNewTargetFromPath: thin glue over SignalFromText/FieldContextFromText/TargetFor
- drop traces index rejection cases: per-path indexes are simply not supported for traces yet
2026-09-24 19:57:13 +05:30
Nikhil Soni
5d5387f42d test(promote): cover the per-path skip index creation of the logs body domain 2026-09-24 19:57:13 +05:30
Nikhil Soni
3cb4802d84 refactor(promote): move the path resolution to types with a validate method, table-drive the tests 2026-09-24 19:57:13 +05:30
Nikhil Soni
291f1a49ac chore: regenerate openapi spec and api clients 2026-09-24 19:57:13 +05:30
Nikhil Soni
d8120f02f7 test: align the subtest names with the table format rule 2026-09-24 19:54:40 +05:30
Nikhil Soni
d5bf4ac6ff fix(promote): rename the signal path variable to telemetry_signal
orval generates an AbortSignal parameter named signal for every client
method, so a {signal} path variable produced a duplicate identifier in
the generated client (tsc error). The URL itself is unchanged in
behavior: /api/v1/promote_paths/{telemetry_signal}/{context}.
2026-09-24 19:54:39 +05:30
Nikhil Soni
e55f005805 refactor(promote): inline the promote and list helpers into their sole callers 2026-09-24 19:54:39 +05:30
Nikhil Soni
00c176e5bf refactor(promote)!: drop the legacy logs promote_paths routes
There are no consumers of /api/v1/logs/promote_paths, so no backward
compatibility is needed: the logs body domain is served by the generic
/api/v1/promote_paths/{signal}/{context} routes and the legacy routes
and handler methods are removed.
2026-09-24 19:54:39 +05:30
Nikhil Soni
48b047757c refactor(promote): move Target into target.go, enum-style SignalFromText, rename handler method
- Target type definition moves from types.go to target.go alongside its
  constructors, with inline comments
- SignalFromText follows the codebase enum pattern (switch over the
  declared values + Enum method) instead of a string-to-signal map
- generic route handler method renamed HandlePromotePaths -> PromotePaths
2026-09-24 19:54:39 +05:30
Nikhil Soni
5098cfb474 refactor(promote): centralize domain construction and generalize routes
- target construction moves to promotetypes: a generic NewTarget plus
  per-domain constructors (NewLogsBodyTarget, NewTracesAttributesTarget)
  and a TargetFor registry keyed by (signal, context); implpromote and
  telemetrymetadata no longer hand-roll domain literals
- routes generalize to /api/v1/promote_paths/{signal}/{context}: the
  legacy logs body route (/api/v1/logs/promote_paths) is kept for
  compatibility but the domain now travels in the path, so a future logs
  attribute domain does not collide with the logs body route; supersedes
  the /api/v1/traces/promote_paths routes
- add telemetrytypes.SignalFromText for parsing the signal path variable
2026-09-24 19:54:39 +05:30
Nikhil Soni
5a8cbb7347 refactor(promote): template the promotion record with EvolutionEntry
Target now carries an EvolutionEntry template (signal, promoted column
name and type, field context) instead of loose signal/context/column
fields, so the store write is exactly row template + field names +
release time and the hardcoded JSON() column type moves to the domain
definitions. DBName/LocalTableName stay on Target explicitly as index
DDL config, used only by targets with index support.
2026-09-24 19:54:39 +05:30
Nikhil Soni
c95c591e1f refactor(promote): collapse module interface to target-parameterized methods
The per-domain methods were pure delegates; the promotion domain now
travels as promotetypes.Target through Module.ListPromotedPaths /
Module.PromotePaths, with the handler methods (one per route) passing
their domain's target.
2026-09-24 19:54:39 +05:30
Nikhil Soni
5e11b1490f feat(promote): add traces attributes promotion API
Refactor the promote module into a target-parameterized core so the logs
body_v2 flow and future promotion domains share one implementation, and
add the spans attributes JSON column (attributes -> attributes_promoted)
as a second domain behind POST/GET /api/v1/traces/promote_paths.

- promotetypes.Target describes a promotion domain: signal, field
  context, db/table, base/promoted columns, path prefix rule and whether
  per-path skip indexes are supported
- index support is optional per target; traces starts promotion-only
  since the traces query builder does not consume per-path skip indexes
- metadata store GetPromotedPaths/PromotePaths take (signal, column,
  context) instead of being hardcoded to the logs body column
- fix the list response never attaching indexes to promoted entries
  (aggregated by unprefixed name but looked up by prefixed path) and
  reporting indexed+promoted paths twice
2026-09-24 19:54:39 +05:30
Nityananda Gohain
8371a70801 perf(querybuilder): compare materialized exists columns explicitly (#12978)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Materialized existence checks now render as an explicit comparison
instead of a bare bool column. Results are unchanged; only skip-index
usage improves.

  ```sql
  -- before
  WHERE `attribute_string_gen_ai$$request$$model_exists`
     OR `attribute_string_gen_ai$$provider$$name` = 'anthropic'

  -- after
  WHERE `attribute_string_gen_ai$$request$$model_exists` = true
     OR `attribute_string_gen_ai$$provider$$name` = 'anthropic'
  ```

  <details>
<summary>EXPLAIN indexes = 1 (trace-matching phase, 123M
spans)</summary>

  Before: bare `col_exists`
  ```
  Name: idx_gen_ai_span_exists
  Granules: 15193/15193
  Name: <Combined skip indexes>
  Granules: 15193/15193
  ```

  After: `col_exists = true`
  ```
  Name: idx_gen_ai_span_exists
  Granules: 15193/15193
  Name: <Combined skip indexes>
  Granules: 488/15193
  ```
  </details>

----
- ClickHouse can use a different skip index for each side of an OR and
union the results, but it can't when one side is a bare bool column.
Comparing with `= true` fixes that.
- This shape comes from the AI explorer trace list with a span filter: a
trace qualifies when it has a gen_ai span *and* a span matching the
filter (possibly different spans), so the WHERE is `(gen_ai gate) OR
<filter>` followed by a HAVING.
- Needs the gen_ai materialized columns and `idx_gen_ai_span_exists`
from SigNoz/signoz-otel-collector#929; without them there's no index to
combine.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Part of https://github.com/SigNoz/nerve-pod/issues/282

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
- Benchmarked the AI trace list filtered on `gen_ai.provider.name`
against a 123M-span table (direct I/O, caches off): from ~30M spans in
the window, latency drops 16–17% and CPU 35–38%, with ~25x fewer rows
read (123M spans: 510 → 427 ms, 1.5 → 0.9 sCPU). The saved time and CPU
keep growing with span count, so larger windows save more.
- Single-condition filters (`gen_ai.request.model EXISTS` in dashboard
panels, the AND-ed gate in AI aggregations) already pruned with the bare
form; no change there.
2026-09-24 13:15:32 +00:00
Naman Verma
9d9b0e194a chore: add ability to mark API stability as beta/alpha (#12957)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

If an API that is already deployed is currently being tested via UI
integration or any other means, we should mark such APIs as under
development so that other external clients know that these APIs aren't
fully stable. This is especially required if we are working on v2
versions of APIs for any entity.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Part of https://github.com/SigNoz/pulse-pod/issues/369

<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information

This PR adds the development flag on the v2 notification channel APIs

<!--Please delete paragraphs that you did not use before submitting.-->
2026-09-24 12:19:02 +00:00
Aditya Singh
2a7f4fd603 test(quick-filters): add settings-with-banner stories for every filters page (#12968)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- the quick filters settings panel is sized from the filters pane, not
the viewport. the case that broke was a banner shortening the layout,
which pushed the Save changes footer off screen.. so every page with
settings now has a story for exactly that.
- each new story opens settings and removes a filter first, that is what
puts the footer on screen. same play sequence as the existing dirty
story so the two are comparable side by side.
- external apis and cost meter had no settings story at all, they get
the plain and dirty ones too. cost meter's settings live on the explorer
tab so its stories start there.
- `banner` is already a global control on the app shell mocks, so no
mock changes anywhere.. the stories just turn it on.

#### Issues closed by this PR

Part of https://github.com/SigNoz/engineering-pod/issues/5978

#### Additional Information

- covers logs, traces, exceptions, ai observability, external apis and
cost meter.
- the play functions have not been run here, playwright's browser is not
installed on my machine. external apis and cost meter are the ones worth
checking first since they never had a settings story.
- cc. @H4ad
2026-09-24 08:54:54 +00:00
Nikhil Mantri
ee35fc351f feat(alerts): New list API for alert rules (powers filters, sorting, pagination) (#12780)
#### Description

- New `GET /api/v3/rules` list API for alert rules: filter query DSL,
`states` filter, sort, and offset pagination (design discussion:
SigNoz/pulse-pod#324).
- Based on #12806, which extracts the shared list filter SQL compiler;
this PR adds only the rules key-policy resolver
(`sqlrulestore/filterquery_resolver.go`) on top of it.
- Rule state lives only in the rule manager's memory, so state
filtering, total, sort and pagination run in code after the SQL fetch;
total always equals what is pageable.
- Sorting is deterministic on ties: equal rows break on name then id,
always ascending, so pages never overlap or drop rows between requests.
- Response rows carry only list-page fields, deliberately excluding
`condition`, `annotations` and `notificationSettings`. The envelope also
returns the org's distinct label pairs and the reserved filter keys for
suggestions.
- Also guards previously unlocked reads of the rules map
(`ListRuleStates`, `GetRule`, `TriggeredAlerts`).

**Filter keys and operators**

| Key | Operators | Notes |
|---|---|---|
| `name`, `created_by`, `updated_by` | `=`, `!=`, `CONTAINS`, `LIKE`,
`ILIKE`, `IN` and negations | string search |
| `labels.<key>` | string operators plus `EXISTS`, `NOT EXISTS` |
missing label evaluates as empty string; keys are case-sensitive |
| `severity` | same as `labels.<key>` | alias for `labels.severity` |
| `created_at`, `updated_at` | `=`, `!=`, `<`, `<=`, `>`, `>=`,
`BETWEEN`, `NOT BETWEEN` | quoted RFC3339 values |
| `alert_type` | `=`, `!=`, `IN`, `NOT IN` | enum: `METRIC_BASED_ALERT`,
`TRACES_BASED_ALERT`, `LOGS_BASED_ALERT`, `EXCEPTIONS_BASED_ALERT` |
| `rule_type` | `=`, `!=`, `IN`, `NOT IN` | enum: `threshold_rule`,
`promql_rule`, `anomaly_rule` |

- A bare word is free text: a case-insensitive substring match over
name, description and labels.
- `state` is not a DSL key. It is the repeated `states=` query param:
`firing`, `pending`, `recovering`, `inactive`, `nodata`, `disabled`.
- An unknown key or `REGEXP` returns a 400.

#### Issues closed by this PR

Closes SigNoz/pulse-pod#226

#### Additional Information

- A missing label evaluates as the empty string for every value
operator, one uniform rule instead of the querier's per-operator split
([`AddDefaultExistsFilter`](https://github.com/SigNoz/signoz/blob/e0da06f76d/pkg/types/querybuildertypes/querybuildertypesv5/builder_elements.go#L160));
presence is asked with `EXISTS` / `NOT EXISTS`.
- Integration tests
(`tests/integration/tests/alerts/06_list_rules_v3.py`) cover filters,
states, sorting, pagination, totals and the error contract, run against
both sqlite and postgres.
- Found while testing: the stock `create_notification_channel` fixture
teardown silently fails and leaks channels; follow-up fix needed.

---------

Co-authored-by: Naman Verma <naman.verma@signoz.io>
2026-09-24 07:16:12 +00:00
79 changed files with 4894 additions and 576 deletions

File diff suppressed because it is too large Load Diff

View File

@@ -179,6 +179,7 @@ The `handler.New` function ties the HTTP handler to OpenAPI metadata via `OpenAP
- **SuccessStatusCode**: The HTTP status for successful responses (for example, `http.StatusOK`, `http.StatusCreated`, `http.StatusNoContent`).
- **ErrorStatusCodes**: Additional error status codes beyond the standard ones automatically added by `handler.New`.
- **SecuritySchemes**: Auth mechanisms and scopes required by the operation.
- **Stability**: Maturity marker (`handler.StabilityDevelopment`, `handler.StabilityAlpha`, `handler.StabilityBeta`, `handler.StabilityStable`, the OpenTelemetry Collector levels) emitted as the `x-signoz-stability` extension on every operation. Unset is emitted as `alpha`.
The generic handler:

View File

@@ -23,6 +23,15 @@ func (f *formatter) JSONExtractString(column, path string) []byte {
return append(f.TextToJsonColumn(column), ops...)
}
func (f *formatter) JSONExtractMapValue(column, mapField, key string) []byte {
sql := f.TextToJsonColumn(column)
sql = append(sql, "->"...)
sql = schema.Append(f.bunf, sql, mapField)
sql = append(sql, "->>"...)
sql = schema.Append(f.bunf, sql, key)
return sql
}
func (f *formatter) JSONType(column, path string) []byte {
var sql []byte
sql = append(sql, "jsonb_typeof("...)

View File

@@ -55,6 +55,67 @@ func TestJSONExtractString(t *testing.T) {
}
}
func TestJSONExtractMapValue(t *testing.T) {
tests := []struct {
name string
column string
mapField string
key string
expected string
}{
{
name: "PlainKey",
column: "data",
mapField: "labels",
key: "team",
expected: `"data"::jsonb->'labels'->>'team'`,
},
{
name: "DottedKey_OneMapEntry",
column: "data",
mapField: "labels",
key: "k8s.cluster",
expected: `"data"::jsonb->'labels'->>'k8s.cluster'`,
},
{
name: "SingleQuoteInKey_Doubled",
column: "data",
mapField: "labels",
key: "o'brien",
expected: `"data"::jsonb->'labels'->>'o''brien'`,
},
{
name: "BackslashInKey_Literal",
column: "data",
mapField: "labels",
key: `a\b`,
expected: `"data"::jsonb->'labels'->>'a\b'`,
},
{
name: "DoubleQuoteInKey_Literal",
column: "data",
mapField: "labels",
key: `a"b`,
expected: `"data"::jsonb->'labels'->>'a"b'`,
},
{
name: "QualifiedColumn",
column: "rule.data",
mapField: "labels",
key: "severity",
expected: `"rule"."data"::jsonb->'labels'->>'severity'`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f := newFormatter(pgdialect.New())
got := string(f.JSONExtractMapValue(tt.column, tt.mapField, tt.key))
assert.Equal(t, tt.expected, got)
})
}
}
func TestJSONType(t *testing.T) {
tests := []struct {
name string

View File

@@ -4,23 +4,15 @@
* * regenerate with 'pnpm generate:api'
* SigNoz
*/
import { useMutation, useQuery } from 'react-query';
import { useMutation } from 'react-query';
import type {
InvalidateOptions,
MutationFunction,
QueryClient,
QueryFunction,
QueryKey,
UseMutationOptions,
UseMutationResult,
UseQueryOptions,
UseQueryResult,
} from 'react-query';
import type {
HandleExportRawDataPOSTParams,
ListPromotedAndIndexedPaths200,
PromotetypesPromotePathDTO,
Querybuildertypesv5QueryRangeRequestDTO,
RenderErrorResponseDTO,
} from '../sigNoz.schemas';
@@ -28,26 +20,6 @@ import type {
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
const withQueryKey = <T extends object, K>(
query: T,
queryKey: K,
): T & { queryKey: K } => {
const result = { queryKey } as T & { queryKey: K };
for (const key of Object.keys(query)) {
// The explicit queryKey always wins, matching the previous
// `{ ...query, queryKey }` spread where it was set last.
if (key === 'queryKey') {
continue;
}
Object.defineProperty(result, key, {
enumerable: true,
configurable: true,
get: () => (query as Record<string, unknown>)[key],
});
}
return result;
};
/**
* This endpoints allows complex query exporting raw data for traces and logs
* @summary Export raw data
@@ -149,175 +121,3 @@ export const useHandleExportRawDataPOST = <
> => {
return useMutation(getHandleExportRawDataPOSTMutationOptions(options));
};
/**
* This endpoints promotes and indexes paths
* @summary Promote and index paths
*/
export const listPromotedAndIndexedPaths = (signal?: AbortSignal) => {
return GeneratedAPIInstance<ListPromotedAndIndexedPaths200>({
url: `/api/v1/logs/promote_paths`,
method: 'GET',
signal,
});
};
export const getListPromotedAndIndexedPathsQueryKey = () => {
return [`/api/v1/logs/promote_paths`] as const;
};
export const getListPromotedAndIndexedPathsQueryOptions = <
TData = Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError,
TData
>;
}) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ?? getListPromotedAndIndexedPathsQueryKey();
const queryFn: QueryFunction<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>
> = ({ signal }) => listPromotedAndIndexedPaths(signal);
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListPromotedAndIndexedPathsQueryResult = NonNullable<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>
>;
export type ListPromotedAndIndexedPathsQueryError =
ErrorType<RenderErrorResponseDTO>;
/**
* @summary Promote and index paths
*/
export function useListPromotedAndIndexedPaths<
TData = Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedAndIndexedPaths>>,
TError,
TData
>;
}): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListPromotedAndIndexedPathsQueryOptions(options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return withQueryKey(query, queryOptions.queryKey);
}
/**
* @summary Promote and index paths
*/
export const invalidateListPromotedAndIndexedPaths = async (
queryClient: QueryClient,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListPromotedAndIndexedPathsQueryKey() },
options,
);
return queryClient;
};
/**
* This endpoints promotes and indexes paths
* @summary Promote and index paths
*/
export const handlePromoteAndIndexPaths = (
promotetypesPromotePathDTONull?: BodyType<
PromotetypesPromotePathDTO[] | null
> | null,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v1/logs/promote_paths`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: promotetypesPromotePathDTONull,
signal,
});
};
export const getHandlePromoteAndIndexPathsMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
> => {
const mutationKey = ['handlePromoteAndIndexPaths'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> }
> = (props) => {
const { data } = props ?? {};
return handlePromoteAndIndexPaths(data);
};
return { mutationFn, ...mutationOptions };
};
export type HandlePromoteAndIndexPathsMutationResult = NonNullable<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>
>;
export type HandlePromoteAndIndexPathsMutationBody =
| BodyType<PromotetypesPromotePathDTO[] | null>
| undefined;
export type HandlePromoteAndIndexPathsMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @summary Promote and index paths
*/
export const useHandlePromoteAndIndexPaths = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof handlePromoteAndIndexPaths>>,
TError,
{ data?: BodyType<PromotetypesPromotePathDTO[] | null> },
TContext
> => {
return useMutation(getHandlePromoteAndIndexPathsMutationOptions(options));
};

View File

@@ -0,0 +1,262 @@
/**
* ! Do not edit manually
* * The file has been auto-generated using Orval for SigNoz
* * regenerate with 'pnpm generate:api'
* SigNoz
*/
import { useMutation, useQuery } from 'react-query';
import type {
InvalidateOptions,
MutationFunction,
QueryClient,
QueryFunction,
QueryKey,
UseMutationOptions,
UseMutationResult,
UseQueryOptions,
UseQueryResult,
} from 'react-query';
import type {
ListPromotedPaths200,
ListPromotedPathsPathParameters,
PromotePathsPathParameters,
PromotetypesPromotePathDTO,
RenderErrorResponseDTO,
} from '../sigNoz.schemas';
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
const withQueryKey = <T extends object, K>(
query: T,
queryKey: K,
): T & { queryKey: K } => {
const result = { queryKey } as T & { queryKey: K };
for (const key of Object.keys(query)) {
// The explicit queryKey always wins, matching the previous
// `{ ...query, queryKey }` spread where it was set last.
if (key === 'queryKey') {
continue;
}
Object.defineProperty(result, key, {
enumerable: true,
configurable: true,
get: () => (query as Record<string, unknown>)[key],
});
}
return result;
};
/**
* This endpoint lists the promoted paths of a JSON column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.
* @summary List promoted paths
*/
export const listPromotedPaths = (
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<ListPromotedPaths200>({
url: `/api/v1/promote_paths/${telemetrySignal}/${context}`,
method: 'GET',
signal,
});
};
export const getListPromotedPathsQueryKey = ({
telemetrySignal,
context,
}: ListPromotedPathsPathParameters) => {
return [`/api/v1/promote_paths/${telemetrySignal}/${context}`] as const;
};
export const getListPromotedPathsQueryOptions = <
TData = Awaited<ReturnType<typeof listPromotedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedPaths>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ??
getListPromotedPathsQueryKey({ telemetrySignal, context });
const queryFn: QueryFunction<
Awaited<ReturnType<typeof listPromotedPaths>>
> = ({ signal }) => listPromotedPaths({ telemetrySignal, context }, signal);
return {
queryKey,
queryFn,
enabled:
telemetrySignal !== null &&
telemetrySignal !== undefined &&
context !== null &&
context !== undefined,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof listPromotedPaths>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListPromotedPathsQueryResult = NonNullable<
Awaited<ReturnType<typeof listPromotedPaths>>
>;
export type ListPromotedPathsQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary List promoted paths
*/
export function useListPromotedPaths<
TData = Awaited<ReturnType<typeof listPromotedPaths>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listPromotedPaths>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListPromotedPathsQueryOptions(
{ telemetrySignal, context },
options,
);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return withQueryKey(query, queryOptions.queryKey);
}
/**
* @summary List promoted paths
*/
export const invalidateListPromotedPaths = async (
queryClient: QueryClient,
{ telemetrySignal, context }: ListPromotedPathsPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListPromotedPathsQueryKey({ telemetrySignal, context }) },
options,
);
return queryClient;
};
/**
* This endpoint promotes paths of a JSON column to its promoted column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.
* @summary Promote paths
*/
export const promotePaths = (
{ telemetrySignal, context }: PromotePathsPathParameters,
promotetypesPromotePathDTONull?: BodyType<
PromotetypesPromotePathDTO[] | null
> | null,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v1/promote_paths/${telemetrySignal}/${context}`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: promotetypesPromotePathDTONull,
signal,
});
};
export const getPromotePathsMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
> => {
const mutationKey = ['promotePaths'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof promotePaths>>,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
}
> = (props) => {
const { pathParams, data } = props ?? {};
return promotePaths(pathParams, data);
};
return { mutationFn, ...mutationOptions };
};
export type PromotePathsMutationResult = NonNullable<
Awaited<ReturnType<typeof promotePaths>>
>;
export type PromotePathsMutationBody =
| BodyType<PromotetypesPromotePathDTO[] | null>
| undefined;
export type PromotePathsMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Promote paths
*/
export const usePromotePaths = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof promotePaths>>,
TError,
{
pathParams: PromotePathsPathParameters;
data?: BodyType<PromotetypesPromotePathDTO[] | null>;
},
TContext
> => {
return useMutation(getPromotePathsMutationOptions(options));
};

View File

@@ -41,6 +41,8 @@ import type {
GetRuleHistoryTopContributorsParams,
GetRuleHistoryTopContributorsPathParameters,
ListRules200,
ListRulesV3200,
ListRulesV3Params,
PatchRuleByID200,
PatchRuleByIDPathParameters,
RenderErrorResponseDTO,
@@ -73,7 +75,8 @@ const withQueryKey = <T extends object, K>(
};
/**
* This endpoint lists all alert rules with their current evaluation state
* This endpoint lists all alert rules with their current evaluation state. Deprecated: use ListRulesV3, which supports filtering, sorting and pagination.
* @deprecated
* @summary List alert rules
*/
export const listRules = (signal?: AbortSignal) => {
@@ -115,6 +118,7 @@ export type ListRulesQueryResult = NonNullable<
export type ListRulesQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary List alert rules
*/
@@ -134,6 +138,7 @@ export function useListRules<
}
/**
* @deprecated
* @summary List alert rules
*/
export const invalidateListRules = async (
@@ -1388,3 +1393,97 @@ export const useTestRule = <
> => {
return useMutation(getTestRuleMutationOptions(options));
};
/**
* Returns a page of alert rules with their current evaluation state, trimmed to the fields the list page renders. Supports a filter DSL (`query`), a repeated `states` filter applied after the state overlay, sort (`updated_at`/`created_at`/`name`/`state`/`severity`), order (`asc`/`desc`), and offset-based pagination (`limit`/`offset`). In the filter DSL, a non-reserved key is matched as a rule label directly (`team = infra`); a key that collides with a reserved keyword matches either interpretation (negative operators exclude both), and `labels.<key>` targets only the label. The response also carries the org's label pairs and the reserved filter keys for building filter suggestions.
* @summary List alert rules (v3)
*/
export const listRulesV3 = (
params?: ListRulesV3Params,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<ListRulesV3200>({
url: `/api/v3/rules`,
method: 'GET',
params,
signal,
});
};
export const getListRulesV3QueryKey = (params?: ListRulesV3Params) => {
return [`/api/v3/rules`, ...(params ? [params] : [])] as const;
};
export const getListRulesV3QueryOptions = <
TData = Awaited<ReturnType<typeof listRulesV3>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
params?: ListRulesV3Params,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listRulesV3>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey = queryOptions?.queryKey ?? getListRulesV3QueryKey(params);
const queryFn: QueryFunction<Awaited<ReturnType<typeof listRulesV3>>> = ({
signal,
}) => listRulesV3(params, signal);
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
Awaited<ReturnType<typeof listRulesV3>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListRulesV3QueryResult = NonNullable<
Awaited<ReturnType<typeof listRulesV3>>
>;
export type ListRulesV3QueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary List alert rules (v3)
*/
export function useListRulesV3<
TData = Awaited<ReturnType<typeof listRulesV3>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
params?: ListRulesV3Params,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listRulesV3>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListRulesV3QueryOptions(params, options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return withQueryKey(query, queryOptions.queryKey);
}
/**
* @summary List alert rules (v3)
*/
export const invalidateListRulesV3 = async (
queryClient: QueryClient,
params?: ListRulesV3Params,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListRulesV3QueryKey(params) },
options,
);
return queryClient;
};

View File

@@ -10188,6 +10188,99 @@ export interface RuletypesGettableTestRuleDTO {
message?: string;
}
export interface RuletypesLabelPairDTO {
/**
* @type string
*/
key: string;
/**
* @type string
*/
value: string;
}
export enum RuletypesListOrderDTO {
asc = 'asc',
desc = 'desc',
}
export enum RuletypesListSortDTO {
updated_at = 'updated_at',
created_at = 'created_at',
name = 'name',
state = 'state',
severity = 'severity',
}
export type RuletypesListableRuleDTOLabels = { [key: string]: string };
export enum RuletypesRuleTypeDTO {
threshold_rule = 'threshold_rule',
promql_rule = 'promql_rule',
anomaly_rule = 'anomaly_rule',
}
export interface RuletypesListableRuleDTO {
/**
* @type string
*/
alert: string;
alertType: RuletypesAlertTypeDTO;
/**
* @type string
* @format date-time
*/
createdAt?: string;
/**
* @type string
*/
createdBy?: string;
/**
* @type string
*/
description?: string;
/**
* @type boolean
*/
disabled?: boolean;
/**
* @type string
*/
id: string;
/**
* @type object
*/
labels?: RuletypesListableRuleDTOLabels;
ruleType: RuletypesRuleTypeDTO;
state: RuletypesAlertStateDTO;
/**
* @type string
* @format date-time
*/
updatedAt?: string;
/**
* @type string
*/
updatedBy?: string;
}
export interface RuletypesListableRulesDTO {
/**
* @type array
*/
labels: RuletypesLabelPairDTO[];
/**
* @type array
*/
reservedKeywords: string[];
/**
* @type array
*/
rules: RuletypesListableRuleDTO[];
/**
* @type integer
* @format int64
*/
total: number;
}
export interface RuletypesRenotifyDTO {
/**
* @type array,null
@@ -10284,11 +10377,6 @@ export interface RuletypesRuleConditionDTO {
thresholds?: RuletypesRuleThresholdDataDTO;
}
export enum RuletypesRuleTypeDTO {
threshold_rule = 'threshold_rule',
promql_rule = 'promql_rule',
anomaly_rule = 'anomaly_rule',
}
export interface RuletypesPostableRuleDTO {
/**
* @type string
@@ -12382,17 +12470,6 @@ export type ListUnmappedLLMModels200 = {
status: string;
};
export type ListPromotedAndIndexedPaths200 = {
/**
* @type array,null
*/
data: PromotetypesPromotePathDTO[] | null;
/**
* @type string
*/
status: string;
};
export type ListOrgPreferences200 = {
/**
* @type array
@@ -12418,6 +12495,25 @@ export type GetOrgPreference200 = {
export type UpdateOrgPreferencePathParameters = {
name: string;
};
export type ListPromotedPathsPathParameters = {
telemetrySignal: string;
context: string;
};
export type ListPromotedPaths200 = {
/**
* @type array,null
*/
data: PromotetypesPromotePathDTO[] | null;
/**
* @type string
*/
status: string;
};
export type PromotePathsPathParameters = {
telemetrySignal: string;
context: string;
};
export type ListRoles200 = {
/**
* @type array
@@ -14189,6 +14285,45 @@ export type GetMetricDashboardsV2200 = {
status: string;
};
export type ListRulesV3Params = {
/**
* @type string
* @description undefined
*/
query?: string;
/**
* @type array
* @description undefined
*/
states?: string[];
/**
* @description undefined
*/
sort?: RuletypesListSortDTO;
/**
* @description undefined
*/
order?: RuletypesListOrderDTO;
/**
* @type integer
* @description undefined
*/
limit?: number;
/**
* @type integer
* @description undefined
*/
offset?: number;
};
export type ListRulesV3200 = {
data: RuletypesListableRulesDTO;
/**
* @type string
*/
status: string;
};
export type GetFlamegraphPathParameters = {
traceID: string;
};

View File

@@ -98,17 +98,30 @@ export const QuickFiltersSettings: Story = {
play: openQuickFiltersSettings,
};
const dirtyQuickFiltersSettings = async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
};
/** Settings with an unsaved filter removal and the fixed action footer. */
export const QuickFiltersSettingsDirty: Story = {
play: async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
},
play: dirtyQuickFiltersSettings,
};
/**
* The same panel with a banner above the shell. The banner takes 48px off the
* layout, so this is the case where the footer used to be pushed off screen:
* the panel is sized from the filters pane rather than the viewport, which
* keeps Save changes reachable.
*/
export const QuickFiltersSettingsWithBanner: Story = {
args: { banner: 'trial-expiry' },
play: dirtyQuickFiltersSettings,
};

View File

@@ -1,5 +1,5 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, userEvent, waitFor, within } from 'storybook/test';
import { expect, screen, userEvent, waitFor, within } from 'storybook/test';
import { storyMocks } from '@/storybook/controls/defineStoryMocks';
import type { PageStoryArgs } from '@/storybook/runtime/resolveStory';
@@ -59,6 +59,35 @@ export const PortDomain: Story = {
/** The page fetches before it renders a filter, which outlasts the 1s default. */
const untilLoaded = { timeout: 15_000 };
const openQuickFiltersSettings = async (): Promise<void> => {
// The settings control renders disabled while its permission check is in
// flight and is swapped for the enabled one once the check answers, so it is
// looked up again on every attempt; a click on the disabled one is dropped in
// silence.
const control = await waitFor(() => {
const settings = screen.getByTestId('settings-icon-container');
expect(settings).toBeEnabled();
return settings;
}, untilLoaded);
await userEvent.click(control);
await screen.findByText('Edit quick filters', undefined, untilLoaded);
};
const dirtyQuickFiltersSettings = async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
};
/**
* The quick-filter panel has no test id of its own, and it only mounts once the
* workspace's filters have answered.
@@ -143,3 +172,24 @@ export const NoExternalCalls: Story = {
export const Loading: Story = {
args: { dataState: 'loading' },
};
/** The editable quick-filter settings panel. */
export const QuickFiltersSettings: Story = {
play: openQuickFiltersSettings,
};
/** Settings with an unsaved filter removal and the fixed action footer. */
export const QuickFiltersSettingsDirty: Story = {
play: dirtyQuickFiltersSettings,
};
/**
* The same panel with a banner above the shell. The banner takes 48px off the
* layout, so this is the case where the footer used to be pushed off screen:
* the panel is sized from the filters pane rather than the viewport, which
* keeps Save changes reachable.
*/
export const QuickFiltersSettingsWithBanner: Story = {
args: { banner: 'trial-expiry' },
play: dirtyQuickFiltersSettings,
};

View File

@@ -146,11 +146,39 @@ export const Failed: Story = {
parameters: { allowConsoleErrors: true },
};
const dirtyQuickFiltersSettings = async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
};
/** The editable quick-filter settings panel. */
export const QuickFiltersSettings: Story = {
play: openQuickFiltersSettings,
};
/** Settings with an unsaved filter removal and the fixed action footer. */
export const QuickFiltersSettingsDirty: Story = {
play: dirtyQuickFiltersSettings,
};
/**
* The same panel with a banner above the shell. The banner takes 48px off the
* layout, so this is the case where the footer used to be pushed off screen:
* the panel is sized from the filters pane rather than the viewport, which
* keeps Save changes reachable.
*/
export const QuickFiltersSettingsWithBanner: Story = {
args: { banner: 'trial-expiry' },
play: dirtyQuickFiltersSettings,
};
/** A quick-filter value selected against the LLM span query. */
export const QuickFilterSelected: Story = {
play: async ({ canvasElement }): Promise<void> => {

View File

@@ -166,18 +166,31 @@ export const QuickFiltersSettings: Story = {
play: openQuickFiltersSettings,
};
const dirtyQuickFiltersSettings = async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
};
/** Settings with an unsaved filter removal and the fixed action footer. */
export const QuickFiltersSettingsDirty: Story = {
play: async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
play: dirtyQuickFiltersSettings,
};
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
},
/**
* The same panel with a banner above the shell. The banner takes 48px off the
* layout, so this is the case where the footer used to be pushed off screen:
* the panel is sized from the filters pane rather than the viewport, which
* keeps Save changes reachable.
*/
export const QuickFiltersSettingsWithBanner: Story = {
args: { banner: 'trial-expiry' },
play: dirtyQuickFiltersSettings,
};
/**

View File

@@ -1,4 +1,5 @@
import type { Meta, StoryObj } from '@storybook/react-vite';
import { expect, screen, userEvent, waitFor } from 'storybook/test';
import { storyMocks } from '@/storybook/controls/defineStoryMocks';
import type { PageStoryArgs } from '@/storybook/runtime/resolveStory';
@@ -18,6 +19,7 @@ const pageStory = storyMocks(meterMocks, { layout: 'app' });
*/
const meta = {
title: 'Pages/Metering/Cost Meter',
tags: ['play'],
component: MeterExplorerPage,
...pageStory,
parameters: { ...pageStory.parameters },
@@ -27,6 +29,38 @@ export default meta;
type Story = StoryObj<MeterArgs>;
/** The page fetches before it renders its filters, which outlasts the 1s default. */
const untilLoaded = { timeout: 15_000 };
const openQuickFiltersSettings = async (): Promise<void> => {
// The settings control renders disabled while its permission check is in
// flight and is swapped for the enabled one once the check answers, so it is
// looked up again on every attempt; a click on the disabled one is dropped in
// silence.
const control = await waitFor(() => {
const settings = screen.getByTestId('settings-icon-container');
expect(settings).toBeEnabled();
return settings;
}, untilLoaded);
await userEvent.click(control);
await screen.findByText('Edit quick filters', undefined, untilLoaded);
};
const dirtyQuickFiltersSettings = async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
};
/**
* The Meter tab over the last day: what the workspace ingested in total, then
* the hourly count and size of log records, of spans, and the metric datapoints
@@ -88,3 +122,26 @@ export const ExplorerWithoutQuickFilters: Story = {
export const ViewsEmpty: Story = {
args: { tab: 'views', savedViews: 0 },
};
/** The editable quick-filter settings panel, which lives on the Explorer tab. */
export const QuickFiltersSettings: Story = {
args: { tab: 'explorer' },
play: openQuickFiltersSettings,
};
/** Settings with an unsaved filter removal and the fixed action footer. */
export const QuickFiltersSettingsDirty: Story = {
args: { tab: 'explorer' },
play: dirtyQuickFiltersSettings,
};
/**
* The same panel with a banner above the shell. The banner takes 48px off the
* layout, so this is the case where the footer used to be pushed off screen:
* the panel is sized from the filters pane rather than the viewport, which
* keeps Save changes reachable.
*/
export const QuickFiltersSettingsWithBanner: Story = {
args: { tab: 'explorer', banner: 'trial-expiry' },
play: dirtyQuickFiltersSettings,
};

View File

@@ -116,16 +116,29 @@ export const QuickFiltersSettings: Story = {
play: openQuickFiltersSettings,
};
const dirtyQuickFiltersSettings = async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
};
/** Settings with an unsaved filter removal and the fixed action footer. */
export const QuickFiltersSettingsDirty: Story = {
play: async (): Promise<void> => {
await openQuickFiltersSettings();
// One Remove per added filter; the first row's is the one clicked.
const [removeFilter] = await screen.findAllByRole('button', {
name: 'Remove',
});
await userEvent.click(removeFilter);
await screen.findByRole('button', { name: 'Save changes' });
},
play: dirtyQuickFiltersSettings,
};
/**
* The same panel with a banner above the shell. The banner takes 48px off the
* layout, so this is the case where the footer used to be pushed off screen:
* the panel is sized from the filters pane rather than the viewport, which
* keeps Save changes reachable.
*/
export const QuickFiltersSettingsWithBanner: Story = {
args: { banner: 'trial-expiry' },
play: dirtyQuickFiltersSettings,
};

View File

@@ -145,6 +145,7 @@ func (provider *provider) addAlertmanagerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusCreated,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusConflict},
Deprecated: false,
Stability: handler.StabilityDevelopment,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceNotificationChannel.Scope(coretypes.VerbCreate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
@@ -173,6 +174,7 @@ func (provider *provider) addAlertmanagerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
Stability: handler.StabilityDevelopment,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceNotificationChannel.Scope(coretypes.VerbList)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
@@ -199,6 +201,7 @@ func (provider *provider) addAlertmanagerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
Stability: handler.StabilityDevelopment,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceNotificationChannel.Scope(coretypes.VerbRead)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
@@ -226,6 +229,7 @@ func (provider *provider) addAlertmanagerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
Stability: handler.StabilityDevelopment,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceNotificationChannel.Scope(coretypes.VerbUpdate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
@@ -253,6 +257,7 @@ func (provider *provider) addAlertmanagerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
Stability: handler.StabilityDevelopment,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceNotificationChannel.Scope(coretypes.VerbDelete)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
@@ -281,6 +286,7 @@ func (provider *provider) addAlertmanagerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: false,
Stability: handler.StabilityDevelopment,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceNotificationChannel.Scope(coretypes.VerbUpdate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
@@ -308,6 +314,7 @@ func (provider *provider) addAlertmanagerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusBadRequest},
Deprecated: false,
Stability: handler.StabilityDevelopment,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceNotificationChannel.Scope(coretypes.VerbCreate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{

View File

@@ -10,11 +10,11 @@ import (
)
func (provider *provider) addPromoteRoutes(router *mux.Router) error {
if err := router.Handle("/api/v1/logs/promote_paths", handler.New(provider.authzMiddleware.EditAccess(provider.promoteHandler.HandlePromoteAndIndexPaths), handler.OpenAPIDef{
ID: "HandlePromoteAndIndexPaths",
Tags: []string{"logs"},
Summary: "Promote and index paths",
Description: "This endpoints promotes and indexes paths",
if err := router.Handle("/api/v1/promote_paths/{telemetry_signal}/{context}", handler.New(provider.authzMiddleware.EditAccess(provider.promoteHandler.PromotePaths), handler.OpenAPIDef{
ID: "PromotePaths",
Tags: []string{"promote"},
Summary: "Promote paths",
Description: "This endpoint promotes paths of a JSON column to its promoted column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.",
Request: new([]*promotetypes.PromotePath),
RequestContentType: "application/json",
Response: nil,
@@ -26,11 +26,11 @@ func (provider *provider) addPromoteRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/logs/promote_paths", handler.New(provider.authzMiddleware.ViewAccess(provider.promoteHandler.ListPromotedAndIndexedPaths), handler.OpenAPIDef{
ID: "ListPromotedAndIndexedPaths",
Tags: []string{"logs"},
Summary: "Promote and index paths",
Description: "This endpoints promotes and indexes paths",
if err := router.Handle("/api/v1/promote_paths/{telemetry_signal}/{context}", handler.New(provider.authzMiddleware.ViewAccess(provider.promoteHandler.ListPromotedPaths), handler.OpenAPIDef{
ID: "ListPromotedPaths",
Tags: []string{"promote"},
Summary: "List promoted paths",
Description: "This endpoint lists the promoted paths of a JSON column. The promotion domain is identified by the telemetry_signal and context path variables, e.g. traces/attribute.",
Request: nil,
RequestContentType: "",
Response: new([]*promotetypes.PromotePath),

View File

@@ -15,10 +15,26 @@ func (provider *provider) addRulerRoutes(router *mux.Router) error {
ID: "ListRules",
Tags: []string{"rules"},
Summary: "List alert rules",
Description: "This endpoint lists all alert rules with their current evaluation state",
Description: "This endpoint lists all alert rules with their current evaluation state. Deprecated: use ListRulesV3, which supports filtering, sorting and pagination.",
Response: make([]*ruletypes.Rule, 0),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v3/rules", handler.New(provider.authzMiddleware.ViewAccess(provider.rulerHandler.ListRulesV3), handler.OpenAPIDef{
ID: "ListRulesV3",
Tags: []string{"rules"},
Summary: "List alert rules (v3)",
Description: "Returns a page of alert rules with their current evaluation state, trimmed to the fields the list page renders. Supports a filter DSL (`query`), a repeated `states` filter applied after the state overlay, sort (`updated_at`/`created_at`/`name`/`state`/`severity`), order (`asc`/`desc`), and offset-based pagination (`limit`/`offset`). In the filter DSL, a non-reserved key is matched as a rule label directly (`team = infra`); a key that collides with a reserved keyword matches either interpretation (negative operators exclude both), and `labels.<key>` targets only the label. The response also carries the org's label pairs and the reserved filter keys for building filter suggestions.",
RequestQuery: new(ruletypes.ListRulesParams),
Response: new(ruletypes.ListableRules),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest},
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
return err

View File

@@ -11,6 +11,7 @@ var (
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureEnableMetricsReduction = featuretypes.MustNewName("enable_metrics_reduction")
FeatureResolveSemconvFamilies = featuretypes.MustNewName("resolve_semconv_families")
FeatureUseTraceAttributesJSON = featuretypes.MustNewName("use_trace_attributes_json")
)
func MustNewRegistry() featuretypes.Registry {
@@ -79,6 +80,14 @@ func MustNewRegistry() featuretypes.Registry {
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureUseTraceAttributesJSON,
Kind: featuretypes.KindBoolean,
Stage: featuretypes.StageExperimental,
Description: "Controls whether trace queries read span attributes from the JSON columns",
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
)
if err != nil {
panic(err)

View File

@@ -0,0 +1,75 @@
package handler
import (
"net/http"
"testing"
"github.com/gorilla/mux"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/swaggest/openapi-go"
"github.com/swaggest/openapi-go/openapi3"
)
type bespokeOpenAPIHandler struct{}
func (bespokeOpenAPIHandler) ServeHTTP(http.ResponseWriter, *http.Request) {}
func (bespokeOpenAPIHandler) ServeOpenAPI(opCtx openapi.OperationContext) {
opCtx.SetID("Bespoke")
opCtx.AddRespStructure(nil, openapi.WithHTTPStatus(http.StatusOK))
}
func (bespokeOpenAPIHandler) ResourceDefs() []ResourceDef { return nil }
func TestAttachStabilities(t *testing.T) {
router := mux.NewRouter()
router.Handle("/development", New(func(http.ResponseWriter, *http.Request) {}, OpenAPIDef{ID: "Development", SuccessStatusCode: http.StatusOK, Stability: StabilityDevelopment})).Methods(http.MethodGet)
router.Handle("/beta/{id}", New(func(http.ResponseWriter, *http.Request) {}, OpenAPIDef{ID: "Beta", SuccessStatusCode: http.StatusOK, Stability: StabilityBeta})).Methods(http.MethodPut)
router.Handle("/unset", New(func(http.ResponseWriter, *http.Request) {}, OpenAPIDef{ID: "Unset", SuccessStatusCode: http.StatusOK})).Methods(http.MethodGet)
router.Handle("/bespoke", bespokeOpenAPIHandler{}).Methods(http.MethodGet)
reflector := openapi3.NewReflector()
collector := NewOpenAPICollector(reflector)
require.NoError(t, router.Walk(collector.Walker))
collector.AttachStabilities(reflector.Spec)
testCases := []struct {
subtestName string
path string
method string
expectedExtensionValue any
}{
{
subtestName: "development handler",
path: "/development",
method: "get",
expectedExtensionValue: "development",
},
{
subtestName: "beta handler with path parameter",
path: "/beta/{id}",
method: "put",
expectedExtensionValue: "beta",
},
{
subtestName: "unset handler defaults to alpha",
path: "/unset",
method: "get",
expectedExtensionValue: "alpha",
},
{
subtestName: "handler built outside New defaults to alpha",
path: "/bespoke",
method: "get",
expectedExtensionValue: "alpha",
},
}
for _, testCase := range testCases {
t.Run(testCase.subtestName, func(t *testing.T) {
operation := reflector.Spec.Paths.MapOfPathItemValues[testCase.path].MapOfOperationValues[testCase.method]
assert.Equal(t, testCase.expectedExtensionValue, operation.MapOfAnything["x-signoz-stability"])
})
}
}

View File

@@ -1,14 +1,37 @@
package handler
import (
"net/http"
"reflect"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
"github.com/swaggest/jsonschema-go"
openapigo "github.com/swaggest/openapi-go"
"github.com/swaggest/openapi-go/openapi3"
"github.com/swaggest/rest/openapi"
)
const signozStabilityKey string = "x-signoz-stability"
var (
StabilityDevelopment = Stability{valuer.NewString("development")}
StabilityAlpha = Stability{valuer.NewString("alpha")}
StabilityBeta = Stability{valuer.NewString("beta")}
StabilityStable = Stability{valuer.NewString("stable")}
)
// Stability is emitted as the x-signoz-stability extension on every operation; unset means alpha.
type Stability struct{ valuer.String }
func (stability Stability) StringValue() string {
if stability.IsZero() {
return StabilityAlpha.String.StringValue()
}
return stability.String.StringValue()
}
// OpenAPIExample is a named example for an OpenAPI operation.
type OpenAPIExample struct {
Name string
@@ -32,6 +55,7 @@ type OpenAPIDef struct {
SuccessStatusCode int
ErrorStatusCodes []int
Deprecated bool
Stability Stability
SecuritySchemes []OpenAPISecurityScheme
}
@@ -42,14 +66,16 @@ type OpenAPISecurityScheme struct {
// OpenAPICollector is a collector for OpenAPI operations.
type OpenAPICollector struct {
collector *openapi.Collector
collector *openapi.Collector
stabilities map[operationKey]Stability
}
func NewOpenAPICollector(reflector openapigo.Reflector) *OpenAPICollector {
c := openapi.NewCollector(reflector)
return &OpenAPICollector{
collector: c,
collector: c,
stabilities: make(map[operationKey]Stability),
}
}
@@ -77,6 +103,9 @@ func (c *OpenAPICollector) Walker(route *mux.Route, _ *mux.Router, _ []*mux.Rout
if err := c.collector.CollectOperation(method, path, c.collect(method, path, handler.ServeOpenAPI)); err != nil {
return err
}
if err := c.recordStability(method, path, httpHandler); err != nil {
return err
}
}
return nil
}
@@ -84,6 +113,17 @@ func (c *OpenAPICollector) Walker(route *mux.Route, _ *mux.Router, _ []*mux.Rout
return nil
}
// AttachStabilities stamps every operation in spec, so handlers built outside New
// carry the unset stability rather than none.
func (c *OpenAPICollector) AttachStabilities(spec *openapi3.Spec) {
for path, pathItem := range spec.Paths.MapOfPathItemValues {
for method, operation := range pathItem.MapOfOperationValues {
operation.WithMapOfAnythingItem(signozStabilityKey, c.stabilities[operationKey{method: method, path: path}].StringValue())
pathItem.MapOfOperationValues[method] = operation
}
}
}
func (c *OpenAPICollector) collect(method string, path string, serveOpenAPIFunc ServeOpenAPIFunc) func(oc openapigo.OperationContext) error {
return func(oc openapigo.OperationContext) error {
// Serve the OpenAPI documentation for the handler
@@ -117,3 +157,23 @@ func (c *OpenAPICollector) collect(method string, path string, serveOpenAPIFunc
return nil
}
}
func (c *OpenAPICollector) recordStability(method string, path string, httpHandler http.Handler) error {
generic, ok := httpHandler.(*handler)
if !ok {
return nil
}
cleanMethod, cleanPath, _, err := openapigo.SanitizeMethodPath(method, path)
if err != nil {
return err
}
c.stabilities[operationKey{method: cleanMethod, path: cleanPath}] = generic.openAPIDef.Stability
return nil
}
type operationKey struct {
method string
path string
}

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/promotetypes"
"github.com/gorilla/mux"
)
type handler struct {
@@ -19,9 +20,16 @@ func NewHandler(module promote.Module) promote.Handler {
return &handler{module: module}
}
func (h *handler) HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Request) {
func (h *handler) PromotePaths(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
target, err := promotetypes.NewTargetFromPath(vars["telemetry_signal"], vars["context"])
if err != nil {
render.Error(w, err)
return
}
// TODO(Nitya): Use in multi tenant setup
_, err := authtypes.ClaimsFromContext(r.Context())
_, err = authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, errors.NewInternalf(errors.CodeInternal, "failed to get org id from context"))
return
@@ -33,7 +41,7 @@ func (h *handler) HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Requ
return
}
err = h.module.PromoteAndIndexPaths(r.Context(), req...)
err = h.module.PromotePaths(r.Context(), target, req...)
if err != nil {
render.Error(w, err)
return
@@ -42,15 +50,22 @@ func (h *handler) HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Requ
render.Success(w, http.StatusCreated, nil)
}
func (h *handler) ListPromotedAndIndexedPaths(w http.ResponseWriter, r *http.Request) {
func (h *handler) ListPromotedPaths(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
target, err := promotetypes.NewTargetFromPath(vars["telemetry_signal"], vars["context"])
if err != nil {
render.Error(w, err)
return
}
// TODO(Nitya): Use in multi tenant setup
_, err := authtypes.ClaimsFromContext(r.Context())
_, err = authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, errors.NewInternalf(errors.CodeInternal, "failed to get org id from context"))
return
}
paths, err := h.module.ListPromotedAndIndexedPaths(r.Context())
paths, err := h.module.ListPromotedPaths(r.Context(), target)
if err != nil {
render.Error(w, err)
return

View File

@@ -2,14 +2,11 @@ package implpromote
import (
"context"
"maps"
"slices"
"strings"
schemamigrator "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/modules/promote"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
@@ -31,45 +28,57 @@ func NewModule(metadataStore telemetrytypes.MetadataStore, telemetrystore teleme
return &module{metadataStore: metadataStore, telemetryStore: telemetrystore}
}
func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetypes.PromotePath, error) {
// ListPromotedPaths lists the promoted paths of the target JSON column,
// merged with per-path index metadata where the target supports indexes.
func (m *module) ListPromotedPaths(ctx context.Context, target promotetypes.Target) ([]promotetypes.PromotePath, error) {
promotedPaths, err := m.metadataStore.GetPromotedPaths(ctx, target.Entry)
if err != nil {
return nil, err
}
response := make([]promotetypes.PromotePath, 0, len(promotedPaths))
for path := range promotedPaths {
response = append(response, promotetypes.PromotePath{
Path: target.RequiredPathPrefix + path,
Promote: true,
})
}
// Index metadata is optional per target; merge it in only where supported.
if !target.IndexesSupported {
return response, nil
}
indexes, err := m.metadataStore.ListLogsJSONIndexes(ctx)
if err != nil {
return nil, err
}
// index.Name is the bare path and index.BaseColumn carries the column
// prefix, so the aggregate key is the full sub-column path.
aggr := map[string][]promotetypes.WrappedIndex{}
for _, index := range indexes {
aggr[index.Name] = append(aggr[index.Name], promotetypes.WrappedIndex{
fullPath := index.BaseColumn + index.Name
aggr[fullPath] = append(aggr[fullPath], promotetypes.WrappedIndex{
FieldDataType: index.FieldDataType,
Type: index.IndexType,
Granularity: index.Granularity,
})
}
promotedPaths, err := m.listPromotedPaths(ctx)
if err != nil {
return nil, err
}
response := []promotetypes.PromotePath{}
for _, path := range promotedPaths {
fullPath := logstelemetryschema.BodyPromotedColumnPrefix + path
path = telemetrytypes.BodyJSONStringSearchPrefix + path
item := promotetypes.PromotePath{
Path: path,
Promote: true,
}
indexes, ok := aggr[fullPath]
if ok {
item.Indexes = indexes
for i := range response {
fullPath := target.PromotedColumnPrefix() + strings.TrimPrefix(response[i].Path, target.RequiredPathPrefix)
if indexes, ok := aggr[fullPath]; ok {
response[i].Indexes = indexes
delete(aggr, fullPath)
}
response = append(response, item)
}
// add the paths that are not promoted but have indexes
for path, indexes := range aggr {
path := strings.TrimPrefix(path, logstelemetryschema.BodyV2ColumnPrefix)
path = telemetrytypes.BodyJSONStringSearchPrefix + path
for fullPath, indexes := range aggr {
path := strings.TrimPrefix(fullPath, target.BaseColumnPrefix())
path = strings.TrimPrefix(path, target.PromotedColumnPrefix())
path = target.RequiredPathPrefix + path
response = append(response, promotetypes.PromotePath{
Path: path,
Indexes: indexes,
@@ -78,54 +87,10 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
return response, nil
}
func (m *module) listPromotedPaths(ctx context.Context) ([]string, error) {
paths, err := m.metadataStore.GetPromotedPaths(ctx)
if err != nil {
return nil, err
}
return slices.Collect(maps.Keys(paths)), nil
}
// PromotePaths inserts provided JSON paths into the promoted paths table for logs queries.
func (m *module) PromotePaths(ctx context.Context, paths []string) error {
if len(paths) == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "paths cannot be empty")
}
return m.metadataStore.PromotePaths(ctx, paths...)
}
// createIndexes creates string ngram + token filter indexes on JSON path subcolumns for LIKE queries.
func (m *module) createIndexes(ctx context.Context, indexes []schemamigrator.Index) error {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalLogs.StringValue(),
instrumentationtypes.CodeNamespace: "promote",
instrumentationtypes.CodeFunctionName: "createIndexes",
})
if len(indexes) == 0 {
return nil
}
for _, index := range indexes {
alterStmt := schemamigrator.AlterTableAddIndex{
Database: logstelemetryschema.DBName,
Table: logstelemetryschema.LogsV2LocalTableName,
Index: index,
}
op := alterStmt.OnCluster(m.telemetryStore.Cluster())
if err := m.telemetryStore.ClickhouseDB().Exec(ctx, op.ToSQL()); err != nil {
return errors.WrapInternalf(err, CodeFailedToCreateIndex, "failed to create index")
}
}
return nil
}
// PromoteAndIndexPaths handles promoting paths and creating indexes in one call.
func (m *module) PromoteAndIndexPaths(
ctx context.Context,
paths ...*promotetypes.PromotePath,
) error {
// PromotePaths records new promotions of the target JSON column in the column
// evolution table and, for targets with index support, creates the requested
// per-path skip indexes.
func (m *module) PromotePaths(ctx context.Context, target promotetypes.Target, paths ...*promotetypes.PromotePath) error {
if len(paths) == 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "paths cannot be empty")
}
@@ -133,13 +98,13 @@ func (m *module) PromoteAndIndexPaths(
pathsStr := []string{}
// validate the paths
for _, path := range paths {
if err := path.ValidateAndSetDefaults(); err != nil {
if err := path.ValidateAndSetDefaults(target); err != nil {
return err
}
pathsStr = append(pathsStr, path.Path)
}
existingPromotedPaths, err := m.metadataStore.GetPromotedPaths(ctx, pathsStr...)
existingPromotedPaths, err := m.metadataStore.GetPromotedPaths(ctx, target.Entry, pathsStr...)
if err != nil {
return err
}
@@ -153,10 +118,10 @@ func (m *module) PromoteAndIndexPaths(
}
}
if len(it.Indexes) > 0 {
parentColumn := logstelemetryschema.LogsV2BodyV2Column
parentColumn := target.BaseColumn
// if the path is already promoted or is being promoted, add it to the promoted column
if _, promoted := existingPromotedPaths[it.Path]; promoted || it.Promote {
parentColumn = logstelemetryschema.LogsV2BodyPromotedColumn
parentColumn = target.PromotedColumn()
}
for _, index := range it.Indexes {
@@ -182,17 +147,43 @@ func (m *module) PromoteAndIndexPaths(
}
if len(toInsert) > 0 {
err := m.PromotePaths(ctx, toInsert)
err := m.metadataStore.PromotePaths(ctx, target.Entry, toInsert...)
if err != nil {
return err
}
}
if len(indexes) > 0 {
if err := m.createIndexes(ctx, indexes); err != nil {
if err := m.createIndexes(ctx, target, indexes); err != nil {
return err
}
}
return nil
}
// createIndexes creates string ngram + token filter indexes on JSON path subcolumns for LIKE queries.
func (m *module) createIndexes(ctx context.Context, target promotetypes.Target, indexes []schemamigrator.Index) error {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: target.Entry.Signal.StringValue(),
instrumentationtypes.CodeNamespace: "promote",
instrumentationtypes.CodeFunctionName: "createIndexes",
})
if len(indexes) == 0 {
return nil
}
for _, index := range indexes {
alterStmt := schemamigrator.AlterTableAddIndex{
Database: target.DBName,
Table: target.LocalTableName,
Index: index,
}
op := alterStmt.OnCluster(m.telemetryStore.Cluster())
if err := m.telemetryStore.ClickhouseDB().Exec(ctx, op.ToSQL()); err != nil {
return errors.WrapInternalf(err, CodeFailedToCreateIndex, "failed to create index")
}
}
return nil
}

View File

@@ -0,0 +1,234 @@
package implpromote
import (
"context"
"regexp"
"testing"
sqlmock "github.com/DATA-DOG/go-sqlmock"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
"github.com/SigNoz/signoz/pkg/types/promotetypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestPromotePaths(t *testing.T) {
ctx := context.Background()
testCases := []struct {
name string
target promotetypes.Target
paths []*promotetypes.PromotePath
promoteTwice bool
wantErr bool
wantPromoted []string
}{
{
name: "PromotesNewAttributes_Idempotent",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{
{Path: "http.method", Promote: true},
{Path: "span.operation", Promote: true},
},
promoteTwice: true,
wantPromoted: []string{"http.method", "span.operation"},
},
{
name: "NonPromoteEntries_NotRecorded",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{{Path: "http.method"}},
},
{
name: "ColumnPrefixedPath_Rejected",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{{Path: "attributes.http.method", Promote: true}},
wantErr: true,
},
{
name: "EmptyPath_Rejected",
target: promotetypes.NewTracesAttributesTarget(),
paths: []*promotetypes.PromotePath{{Path: "", Promote: true}},
wantErr: true,
},
{
name: "EmptyRequest_Rejected",
target: promotetypes.NewTracesAttributesTarget(),
wantErr: true,
},
{
name: "PromotesBodyPath_PrefixStripped",
target: promotetypes.NewLogsBodyTarget(),
paths: []*promotetypes.PromotePath{{Path: "body.user.name", Promote: true}},
wantPromoted: []string{"user.name"},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
store := telemetrytypestest.NewMockMetadataStore()
m := NewModule(store, nil)
err := m.PromotePaths(ctx, testCase.target, testCase.paths...)
if testCase.wantErr {
assert.Error(t, err)
assert.Empty(t, store.PromotedPathsMap)
return
}
require.NoError(t, err)
require.Len(t, store.PromotedPathsMap, len(testCase.wantPromoted))
for _, path := range testCase.wantPromoted {
assert.True(t, store.PromotedPathsMap[path], path)
}
if testCase.promoteTwice {
// promoting again must not fail
require.NoError(t, m.PromotePaths(ctx, testCase.target, testCase.paths...))
assert.Len(t, store.PromotedPathsMap, len(testCase.wantPromoted))
}
})
}
}
func TestPromotePathsCreatesIndexes(t *testing.T) {
ctx := context.Background()
testCases := []struct {
name string
promoted map[string]bool
path *promotetypes.PromotePath
wantDDLColumn string
}{
{
name: "NewPromotion_IndexesPromotedColumn",
path: &promotetypes.PromotePath{
Path: "body.user.name",
Promote: true,
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantDDLColumn: "dynamicElement(body_promoted.user.name",
},
{
name: "AlreadyPromoted_IndexesPromotedColumn",
promoted: map[string]bool{"user.name": true},
path: &promotetypes.PromotePath{
Path: "body.user.name",
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantDDLColumn: "dynamicElement(body_promoted.user.name",
},
{
name: "UnpromotedPath_IndexesBaseColumn",
path: &promotetypes.PromotePath{
Path: "body.user.name",
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantDDLColumn: "dynamicElement(body_v2.user.name",
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
ts := telemetrystoretest.New(telemetrystore.Config{}, sqlmock.QueryMatcherRegexp)
store := telemetrytypestest.NewMockMetadataStore()
if testCase.promoted != nil {
store.PromotedPathsMap = testCase.promoted
}
m := NewModule(store, ts)
ts.Mock().ExpectExec("ADD INDEX (.+)" + regexp.QuoteMeta(testCase.wantDDLColumn)).WillReturnError(nil)
require.NoError(t, m.PromotePaths(ctx, promotetypes.NewLogsBodyTarget(), testCase.path))
assert.NoError(t, ts.Mock().ExpectationsWereMet())
})
}
}
func TestListPromotedPaths(t *testing.T) {
ctx := context.Background()
testCases := []struct {
name string
target promotetypes.Target
promoted map[string]bool
indexes []telemetrytypes.TelemetryFieldKeySkipIndex
wantPaths []promotetypes.PromotePath
}{
{
name: "TracesAttributes_PromotedPaths",
target: promotetypes.NewTracesAttributesTarget(),
promoted: map[string]bool{"http.method": true},
wantPaths: []promotetypes.PromotePath{
{Path: "http.method", Promote: true},
},
},
{
name: "LogsBody_PromotedAndIndexedPaths",
target: promotetypes.NewLogsBodyTarget(),
promoted: map[string]bool{"user.name": true},
indexes: []telemetrytypes.TelemetryFieldKeySkipIndex{
{
Name: "user.name",
FieldContext: telemetrytypes.FieldContextBody,
FieldDataType: telemetrytypes.FieldDataTypeString,
BaseColumn: "body_promoted.",
IndexType: "ngrambf_v1(4, 1024, 2, 0)",
Granularity: 1,
},
{
Name: "request.duration",
FieldContext: telemetrytypes.FieldContextBody,
FieldDataType: telemetrytypes.FieldDataTypeFloat64,
BaseColumn: "body_v2.",
IndexType: "minmax",
Granularity: 1,
},
},
wantPaths: []promotetypes.PromotePath{
{
Path: "body.user.name",
Promote: true,
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
{
Path: "body.request.duration",
Indexes: []promotetypes.WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeFloat64, Type: "minmax", Granularity: 1},
},
},
},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
store := telemetrytypestest.NewMockMetadataStore()
store.PromotedPathsMap = testCase.promoted
store.LogsJSONIndexes = testCase.indexes
m := NewModule(store, nil)
paths, err := m.ListPromotedPaths(ctx, testCase.target)
require.NoError(t, err)
require.Len(t, paths, len(testCase.wantPaths))
byPath := map[string]promotetypes.PromotePath{}
for _, path := range paths {
byPath[path.Path] = path
}
for _, want := range testCase.wantPaths {
require.Contains(t, byPath, want.Path)
assert.Equal(t, want, byPath[want.Path])
}
})
}
}

View File

@@ -8,11 +8,11 @@ import (
)
type Module interface {
ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetypes.PromotePath, error)
PromoteAndIndexPaths(ctx context.Context, paths ...*promotetypes.PromotePath) error
ListPromotedPaths(ctx context.Context, target promotetypes.Target) ([]promotetypes.PromotePath, error)
PromotePaths(ctx context.Context, target promotetypes.Target, paths ...*promotetypes.PromotePath) error
}
type Handler interface {
HandlePromoteAndIndexPaths(w http.ResponseWriter, r *http.Request)
ListPromotedAndIndexedPaths(w http.ResponseWriter, r *http.Request)
PromotePaths(w http.ResponseWriter, r *http.Request)
ListPromotedPaths(w http.ResponseWriter, r *http.Request)
}

View File

@@ -0,0 +1,20 @@
package rules
import (
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/parser/filterquery/sqlcompiler"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
)
// Compile wraps compiler errors in the rules list filter error code.
func CompileListFilter(query string, formatter sqlstore.SQLFormatter) (*sqlcompiler.Compiled, error) {
compiled, errs := sqlcompiler.Compile(query, formatter, ruleFieldResolver{})
if len(errs) > 0 {
return nil, errors.NewInvalidInputf(ruletypes.ErrCodeRuleListFilterInvalid,
"invalid filter query: %s", strings.Join(errs, "; "))
}
return compiled, nil
}

View File

@@ -0,0 +1,196 @@
package rules
import (
"fmt"
"slices"
"strings"
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
"github.com/SigNoz/signoz/pkg/parser/filterquery/sqlcompiler"
qbtypesv5 "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
)
const (
ruleDataColumn = "rule.data"
ruleLabelsField = "labels"
nameJSONPath = "$.alert"
descriptionPath = "$.description"
labelsJSONPath = "$.labels"
alertTypePath = "$.alertType"
ruleTypePath = "$.ruleType"
)
// ruleFieldResolver maps rule list DSL keys; a non-reserved key is a case-sensitive label lookup.
type ruleFieldResolver struct{}
func (r ruleFieldResolver) ResolveComparison(v *sqlcompiler.Visitor, rawKey string, operation qbtypesv5.FilterOperator, ctx *grammar.ComparisonContext) string {
key := strings.ToLower(rawKey)
// labels.<key> is the explicit way to target only the label on a reserved-key collision.
if strings.HasPrefix(key, ruletypes.DSLLabelsKeyPrefix) {
labelKey := rawKey[len(ruletypes.DSLLabelsKeyPrefix):]
if labelKey == "" {
v.AddError("labels filter is missing a key, use labels.<key>")
return ""
}
if _, allowed := ruletypes.LabelsKeyOps[operation]; !allowed {
v.AddError("operator %s is not allowed on a labels.<key> filter", sqlcompiler.OperationName(operation))
return ""
}
return r.labelComparison(v, ctx, operation, labelKey)
}
allowedOperations, isReserved := ruletypes.ReservedOps[ruletypes.DSLKey(key)]
_, labelAllowed := ruletypes.LabelsKeyOps[operation]
if !isReserved {
if !labelAllowed {
v.AddError("operator %s is not allowed on the label filter %q", sqlcompiler.OperationName(operation), rawKey)
return ""
}
return r.labelComparison(v, ctx, operation, rawKey)
}
_, reservedAllowed := allowedOperations[operation]
// reserved severity is itself the severity-label lookup; an identical spelling would duplicate the predicate
if ruletypes.DSLKey(key) == ruletypes.DSLKeySeverity && rawKey == string(ruletypes.DSLKeySeverity) {
labelAllowed = false
}
switch {
case reservedAllowed && labelAllowed:
reservedPredicate := r.resolveReservedKey(v, ctx, operation, ruletypes.DSLKey(key))
labelPredicate := r.labelComparison(v, ctx, operation, rawKey)
if reservedPredicate == "" || labelPredicate == "" {
return ""
}
// the key matches both the reserved field and a same-named label; a negative term must exclude both
if operation.IsNegativeOperator() {
return v.Sb.And(reservedPredicate, labelPredicate)
}
return v.Sb.Or(reservedPredicate, labelPredicate)
case reservedAllowed:
return r.resolveReservedKey(v, ctx, operation, ruletypes.DSLKey(key))
case labelAllowed:
return r.labelComparison(v, ctx, operation, rawKey)
default:
v.AddError("operator %s is not allowed for key %q", sqlcompiler.OperationName(operation), key)
return ""
}
}
func (r ruleFieldResolver) resolveReservedKey(v *sqlcompiler.Visitor, ctx *grammar.ComparisonContext, operation qbtypesv5.FilterOperator, key ruletypes.DSLKey) string {
switch key {
case ruletypes.DSLKeyName:
columnExpression := string(v.Formatter.JSONExtractString(ruleDataColumn, nameJSONPath))
return v.BuildStringOperation(v.Sb, ctx, operation, columnExpression, string(key))
case ruletypes.DSLKeySeverity:
// severity is an alias for labels.severity, sharing its missing-label semantics.
return r.labelComparison(v, ctx, operation, "severity")
case ruletypes.DSLKeyCreatedBy:
return v.BuildStringOperation(v.Sb, ctx, operation, "rule.created_by", string(key))
case ruletypes.DSLKeyUpdatedBy:
return v.BuildStringOperation(v.Sb, ctx, operation, "rule.updated_by", string(key))
case ruletypes.DSLKeyCreatedAt:
return v.BuildTimestampComparison(ctx, operation, "rule.created_at")
case ruletypes.DSLKeyUpdatedAt:
return v.BuildTimestampComparison(ctx, operation, "rule.updated_at")
case ruletypes.DSLKeyAlertType:
return r.enumComparison(v, ctx, operation, key, alertTypePath, alertTypeValues)
case ruletypes.DSLKeyRuleType:
return r.enumComparison(v, ctx, operation, key, ruleTypePath, ruleTypeValues)
}
v.AddError("no handler for reserved key %q", key)
return ""
}
// A missing label evaluates as the empty string for every value operator; EXISTS/NOT EXISTS test the raw extraction.
func (ruleFieldResolver) labelComparison(v *sqlcompiler.Visitor, ctx *grammar.ComparisonContext, operation qbtypesv5.FilterOperator, labelKey string) string {
columnExpression := string(v.Formatter.JSONExtractMapValue(ruleDataColumn, ruleLabelsField, labelKey))
switch operation {
case qbtypesv5.FilterOperatorExists:
return fmt.Sprintf("%s IS NOT NULL", columnExpression)
case qbtypesv5.FilterOperatorNotExists:
return fmt.Sprintf("%s IS NULL", columnExpression)
}
keyForError := ruletypes.DSLLabelsKeyPrefix + labelKey
columnExpression = fmt.Sprintf("COALESCE(%s, '')", columnExpression)
return v.BuildStringOperation(v.Sb, ctx, operation, columnExpression, keyForError)
}
func (ruleFieldResolver) enumComparison(v *sqlcompiler.Visitor, ctx *grammar.ComparisonContext, operation qbtypesv5.FilterOperator, key ruletypes.DSLKey, jsonPath string, allowedValues []string) string {
columnExpression := string(v.Formatter.JSONExtractString(ruleDataColumn, jsonPath))
var values []string
switch operation {
case qbtypesv5.FilterOperatorEqual, qbtypesv5.FilterOperatorNotEqual:
value, ok := v.ExtractSingleStringValue(ctx, string(key))
if !ok {
return ""
}
values = []string{value}
case qbtypesv5.FilterOperatorIn, qbtypesv5.FilterOperatorNotIn:
list, ok := v.ExtractStringValueList(ctx, string(key))
if !ok {
return ""
}
values = list
default:
v.AddError("operator %s on %q is not implemented", sqlcompiler.OperationName(operation), key)
return ""
}
for _, value := range values {
if !slices.Contains(allowedValues, value) {
v.AddError("invalid value %q for %q, expected one of: %s", value, key, strings.Join(allowedValues, ", "))
return ""
}
}
arguments := make([]any, len(values))
for i, s := range values {
arguments[i] = s
}
switch operation {
case qbtypesv5.FilterOperatorEqual:
return v.Sb.Equal(columnExpression, arguments[0])
case qbtypesv5.FilterOperatorNotEqual:
return v.Sb.NotEqual(columnExpression, arguments[0])
case qbtypesv5.FilterOperatorNotIn:
return v.Sb.NotIn(columnExpression, arguments...)
default:
return v.Sb.In(columnExpression, arguments...)
}
}
// ResolveFreeText searches name, description and the raw labels JSON (which also matches label keys).
func (ruleFieldResolver) ResolveFreeText(v *sqlcompiler.Visitor, value string) string {
nameColumn := string(v.Formatter.JSONExtractString(ruleDataColumn, nameJSONPath))
descriptionColumn := string(v.Formatter.JSONExtractString(ruleDataColumn, descriptionPath))
labelsColumn := string(v.Formatter.JSONExtractString(ruleDataColumn, labelsJSONPath))
return v.Sb.Or(
v.BuildFreeTextContains(v.Sb, nameColumn, value),
v.BuildFreeTextContains(v.Sb, descriptionColumn, value),
v.BuildFreeTextContains(v.Sb, labelsColumn, value),
)
}
var alertTypeValues = func() []string {
values := make([]string, 0, 4)
for _, value := range (ruletypes.AlertType("")).Enum() {
values = append(values, string(value.(ruletypes.AlertType)))
}
return values
}()
var ruleTypeValues = func() []string {
values := make([]string, 0, 3)
for _, value := range (ruletypes.RuleType{}).Enum() {
values = append(values, value.(ruletypes.RuleType).StringValue())
}
return values
}()

View File

@@ -0,0 +1,488 @@
package rules
import (
"strings"
"testing"
"time"
"github.com/DATA-DOG/go-sqlmock"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstoretest"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
)
type compileCase struct {
subtestName string
dslQueryToCompile string
emptyQueryExpected bool
expectedSQL string
expectedArgs []any
expectedErrShouldContain string
}
func runCompileCases(t *testing.T, cases []compileCase) {
t.Helper()
for _, c := range cases {
t.Run(c.subtestName, func(t *testing.T) {
out, err := CompileListFilter(c.dslQueryToCompile, formatter(t))
if c.expectedErrShouldContain != "" {
require.Error(t, err)
assert.Contains(t, strings.ToLower(err.Error()), strings.ToLower(c.expectedErrShouldContain))
return
}
require.NoError(t, err)
if c.emptyQueryExpected {
assert.True(t, out.IsEmpty())
return
}
require.NotNil(t, out)
if c.expectedSQL != "" {
assert.Equal(t, normalizeSQL(c.expectedSQL), normalizeSQL(out.SQL))
}
if c.expectedArgs != nil {
require.Len(t, out.Args, len(c.expectedArgs))
for i, want := range c.expectedArgs {
// Equal instants can differ in *Location, so compare via .Equal() instead of DeepEqual.
if wantT, ok := want.(time.Time); ok {
gotT, ok := out.Args[i].(time.Time)
require.True(t, ok, "arg[%d]: want time.Time, got %T", i, out.Args[i])
assert.True(t, wantT.Equal(gotT), "arg[%d]: want %s, got %s", i, wantT, gotT)
continue
}
assert.Equal(t, want, out.Args[i], "arg[%d]", i)
}
}
})
}
}
func TestCompileEmpty(t *testing.T) {
runCompileCases(t, []compileCase{
{subtestName: "EmptyQuery_Nil", dslQueryToCompile: "", emptyQueryExpected: true},
{subtestName: "WhitespaceQuery_Nil", dslQueryToCompile: " ", emptyQueryExpected: true},
})
}
func TestCompileName(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "NameEquals_MatchesReservedOrLabel",
dslQueryToCompile: "name = 'payment latency'",
expectedSQL: `(json_extract("rule"."data", '$.alert') = ? OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') = ?)`,
expectedArgs: []any{"payment latency", "payment latency"},
},
{
subtestName: "NameContains_EscapesWildcardsBothSides",
dslQueryToCompile: "name CONTAINS '50%'",
expectedSQL: `(json_extract("rule"."data", '$.alert') LIKE ? ESCAPE '\' OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') LIKE ? ESCAPE '\')`,
expectedArgs: []any{`%50\%%`, `%50\%%`},
},
{
subtestName: "NameILike",
dslQueryToCompile: "name ILIKE 'Prod%'",
expectedSQL: `(lower(json_extract("rule"."data", '$.alert')) LIKE LOWER(?) ESCAPE '\' OR lower(COALESCE(json_extract("rule"."data", '$.labels."name"'), '')) LIKE LOWER(?) ESCAPE '\')`,
expectedArgs: []any{"Prod%", "Prod%"},
},
{
subtestName: "NameInList",
dslQueryToCompile: "name IN ['a', 'b']",
expectedSQL: `(json_extract("rule"."data", '$.alert') IN (?, ?) OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') IN (?, ?))`,
expectedArgs: []any{"a", "b", "a", "b"},
},
{
subtestName: "NameNotEquals_ExcludesBoth",
dslQueryToCompile: "name != 'x'",
expectedSQL: `(json_extract("rule"."data", '$.alert') <> ? AND COALESCE(json_extract("rule"."data", '$.labels."name"'), '') <> ?)`,
expectedArgs: []any{"x", "x"},
},
{
subtestName: "NameExists_LabelOnly",
dslQueryToCompile: "name EXISTS",
expectedSQL: `json_extract("rule"."data", '$.labels."name"') IS NOT NULL`,
},
{
subtestName: "RangeOperatorOnName_Rejected",
dslQueryToCompile: "name > 'x'",
expectedErrShouldContain: `operator > is not allowed for key "name"`,
},
{
subtestName: "RegexpOnName_Rejected",
dslQueryToCompile: "name REGEXP 'x.*'",
expectedErrShouldContain: `operator REGEXP is not allowed for key "name"`,
},
})
}
func TestCompileSeverityAndLabels(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "SeverityEquals_TargetsLabelsMap",
dslQueryToCompile: "severity = 'critical'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') = ?`,
expectedArgs: []any{"critical"},
},
{
subtestName: "SeverityNotEquals_MissingLabelAsEmptyString",
dslQueryToCompile: "severity != 'critical'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') <> ?`,
expectedArgs: []any{"critical"},
},
{
subtestName: "SeverityNotEqualsEmpty_ExcludesRulesWithoutSeverity",
dslQueryToCompile: "severity != ''",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') <> ?`,
expectedArgs: []any{""},
},
{
subtestName: "SeverityExists_ThroughAlias",
dslQueryToCompile: "severity EXISTS",
expectedSQL: `json_extract("rule"."data", '$.labels."severity"') IS NOT NULL`,
},
{
subtestName: "SeverityNotExists_ThroughAlias",
dslQueryToCompile: "severity NOT EXISTS",
expectedSQL: `json_extract("rule"."data", '$.labels."severity"') IS NULL`,
},
{
subtestName: "LabelEquals",
dslQueryToCompile: "labels.team = 'infra'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."team"'), '') = ?`,
expectedArgs: []any{"infra"},
},
{
subtestName: "DottedLabelKey_OneMapEntry",
dslQueryToCompile: "labels.k8s.cluster = 'prod-1'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."k8s.cluster"'), '') = ?`,
expectedArgs: []any{"prod-1"},
},
{
subtestName: "LabelKey_CaseSensitive",
dslQueryToCompile: "labels.Team = 'infra'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."Team"'), '') = ?`,
expectedArgs: []any{"infra"},
},
{
subtestName: "LabelExists",
dslQueryToCompile: "labels.team EXISTS",
expectedSQL: `json_extract("rule"."data", '$.labels."team"') IS NOT NULL`,
},
{
subtestName: "LabelNotExists",
dslQueryToCompile: "labels.team NOT EXISTS",
expectedSQL: `json_extract("rule"."data", '$.labels."team"') IS NULL`,
},
{
subtestName: "LabelNotContains_IncludesLabelLessRules",
dslQueryToCompile: "labels.team NOT CONTAINS 'infra'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."team"'), '') NOT LIKE ? ESCAPE '\'`,
expectedArgs: []any{"%infra%"},
},
{
subtestName: "LabelNotIn_IncludesLabelLessRules",
dslQueryToCompile: "labels.team NOT IN ['a', 'b']",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."team"'), '') NOT IN (?, ?)`,
expectedArgs: []any{"a", "b"},
},
})
}
func TestCompileEnums(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "AlertTypeEquals_MatchesEnumOrLabel",
dslQueryToCompile: "alert_type = 'LOGS_BASED_ALERT'",
expectedSQL: `(json_extract("rule"."data", '$.alertType') = ? OR COALESCE(json_extract("rule"."data", '$.labels."alert_type"'), '') = ?)`,
expectedArgs: []any{"LOGS_BASED_ALERT", "LOGS_BASED_ALERT"},
},
{
subtestName: "RuleTypeInList",
dslQueryToCompile: "rule_type IN ['threshold_rule', 'promql_rule']",
expectedSQL: `(json_extract("rule"."data", '$.ruleType') IN (?, ?) OR COALESCE(json_extract("rule"."data", '$.labels."rule_type"'), '') IN (?, ?))`,
expectedArgs: []any{"threshold_rule", "promql_rule", "threshold_rule", "promql_rule"},
},
{
subtestName: "InvalidAlertTypeValue_Rejected",
dslQueryToCompile: "alert_type = 'bogus'",
expectedErrShouldContain: `invalid value "bogus" for "alert_type"`,
},
{
subtestName: "ContainsOnRuleType_LabelOnly",
dslQueryToCompile: "rule_type CONTAINS 'thresh'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."rule_type"'), '') LIKE ? ESCAPE '\'`,
expectedArgs: []any{"%thresh%"},
},
})
}
func TestCompileAuditColumns(t *testing.T) {
createdAt, err := time.Parse(time.RFC3339, "2026-01-02T15:04:05Z")
require.NoError(t, err)
updatedFrom, err := time.Parse(time.RFC3339, "2026-02-01T00:00:00Z")
require.NoError(t, err)
updatedTo, err := time.Parse(time.RFC3339, "2026-03-01T00:00:00Z")
require.NoError(t, err)
runCompileCases(t, []compileCase{
{
subtestName: "CreatedByEquals_MatchesColumnOrLabel",
dslQueryToCompile: "created_by = 'nikhil@signoz.io'",
expectedSQL: `(rule.created_by = ? OR COALESCE(json_extract("rule"."data", '$.labels."created_by"'), '') = ?)`,
expectedArgs: []any{"nikhil@signoz.io", "nikhil@signoz.io"},
},
{
subtestName: "CreatedAtRange",
dslQueryToCompile: "created_at >= '2026-01-02T15:04:05Z'",
expectedSQL: `rule.created_at >= ?`,
expectedArgs: []any{createdAt},
},
{
subtestName: "UpdatedAtBetween",
dslQueryToCompile: "updated_at BETWEEN '2026-02-01T00:00:00Z' AND '2026-03-01T00:00:00Z'",
expectedSQL: `rule.updated_at BETWEEN ? AND ?`,
expectedArgs: []any{updatedFrom, updatedTo},
},
{
subtestName: "NonTimestampOnCreatedAt_Rejected",
dslQueryToCompile: "created_at >= 'yesterday'",
expectedErrShouldContain: "invalid RFC3339 timestamp",
},
})
}
func TestCompileFreeText(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "BareWord_SearchesNameDescriptionLabels",
dslQueryToCompile: "payment",
expectedSQL: `(lower(COALESCE(json_extract("rule"."data", '$.alert'), '')) LIKE LOWER(?) ESCAPE '\' ` +
`OR lower(COALESCE(json_extract("rule"."data", '$.description'), '')) LIKE LOWER(?) ESCAPE '\' ` +
`OR lower(COALESCE(json_extract("rule"."data", '$.labels'), '')) LIKE LOWER(?) ESCAPE '\')`,
expectedArgs: []any{"%payment%", "%payment%", "%payment%"},
},
})
}
func TestCompileComposition(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "AndOfLabelAndColumn",
dslQueryToCompile: "labels.team = 'infra' AND created_by = 'x'",
expectedSQL: `(COALESCE(json_extract("rule"."data", '$.labels."team"'), '') = ? ` +
`AND (rule.created_by = ? OR COALESCE(json_extract("rule"."data", '$.labels."created_by"'), '') = ?))`,
expectedArgs: []any{"infra", "x", "x"},
},
{
subtestName: "Not_WrapsInnerPredicate",
dslQueryToCompile: "NOT (name = 'x')",
expectedSQL: `NOT ((json_extract("rule"."data", '$.alert') = ? OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') = ?))`,
expectedArgs: []any{"x", "x"},
},
{
subtestName: "OrOfNameAndSeverity",
dslQueryToCompile: "name CONTAINS 'pay' OR severity = 'critical'",
expectedSQL: `((json_extract("rule"."data", '$.alert') LIKE ? ESCAPE '\' OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') LIKE ? ESCAPE '\') ` +
`OR COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') = ?)`,
expectedArgs: []any{"%pay%", "%pay%", "critical"},
},
})
}
func TestCompileComplexExamples(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "NameContains_LabelEquals_SeverityIn_CreatedByNotEquals",
dslQueryToCompile: `name CONTAINS 'latency' AND labels.team = 'payments' ` +
`AND severity IN ['critical', 'error'] AND created_by != 'ops@signoz.io'`,
expectedSQL: `((json_extract("rule"."data", '$.alert') LIKE ? ESCAPE '\' OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') LIKE ? ESCAPE '\') ` +
`AND COALESCE(json_extract("rule"."data", '$.labels."team"'), '') = ? ` +
`AND COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') IN (?, ?) ` +
`AND (rule.created_by <> ? AND COALESCE(json_extract("rule"."data", '$.labels."created_by"'), '') <> ?))`,
expectedArgs: []any{"%latency%", "%latency%", "payments", "critical", "error", "ops@signoz.io", "ops@signoz.io"},
},
{
subtestName: "NestedOrAnd_WithParens",
dslQueryToCompile: `(labels.env IN ['prod', 'staging'] OR name LIKE '%prod%') ` +
`AND (severity = 'critical' OR labels.team EXISTS)`,
expectedSQL: `((COALESCE(json_extract("rule"."data", '$.labels."env"'), '') IN (?, ?) ` +
`OR (json_extract("rule"."data", '$.alert') LIKE ? ESCAPE '\' OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') LIKE ? ESCAPE '\')) ` +
`AND (COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') = ? ` +
`OR json_extract("rule"."data", '$.labels."team"') IS NOT NULL))`,
expectedArgs: []any{"prod", "staging", "%prod%", "%prod%", "critical"},
},
{
subtestName: "NotOverGroup_AndedWithEnum",
dslQueryToCompile: `NOT (labels.team = 'infra' OR name CONTAINS 'cpu') AND alert_type = 'METRIC_BASED_ALERT'`,
expectedSQL: `(NOT ((COALESCE(json_extract("rule"."data", '$.labels."team"'), '') = ? ` +
`OR (json_extract("rule"."data", '$.alert') LIKE ? ESCAPE '\' OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') LIKE ? ESCAPE '\'))) ` +
`AND (json_extract("rule"."data", '$.alertType') = ? OR COALESCE(json_extract("rule"."data", '$.labels."alert_type"'), '') = ?))`,
expectedArgs: []any{"infra", "%cpu%", "%cpu%", "METRIC_BASED_ALERT", "METRIC_BASED_ALERT"},
},
{
subtestName: "FreeText_ThreeLevelNesting_Timestamp",
dslQueryToCompile: `prod AND (name ILIKE '%pay%' ` +
`OR (labels.team != 'infra' AND updated_at > '2026-01-02T15:04:05Z'))`,
expectedSQL: `((lower(COALESCE(json_extract("rule"."data", '$.alert'), '')) LIKE LOWER(?) ESCAPE '\' ` +
`OR lower(COALESCE(json_extract("rule"."data", '$.description'), '')) LIKE LOWER(?) ESCAPE '\' ` +
`OR lower(COALESCE(json_extract("rule"."data", '$.labels'), '')) LIKE LOWER(?) ESCAPE '\') ` +
`AND ((lower(json_extract("rule"."data", '$.alert')) LIKE LOWER(?) ESCAPE '\' ` +
`OR lower(COALESCE(json_extract("rule"."data", '$.labels."name"'), '')) LIKE LOWER(?) ESCAPE '\') ` +
`OR (COALESCE(json_extract("rule"."data", '$.labels."team"'), '') <> ? AND rule.updated_at > ?)))`,
expectedArgs: []any{"%prod%", "%prod%", "%prod%", "%pay%", "%pay%", "infra",
time.Date(2026, 1, 2, 15, 4, 5, 0, time.UTC)},
},
})
}
func TestCompileBareLabelKeys(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "BareKey_LabelMatch",
dslQueryToCompile: "team = 'infra'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."team"'), '') = ?`,
expectedArgs: []any{"infra"},
},
{
subtestName: "BareKey_CaseSensitive",
dslQueryToCompile: "Team CONTAINS 'inf'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."Team"'), '') LIKE ? ESCAPE '\'`,
expectedArgs: []any{"%inf%"},
},
{
subtestName: "BareKeyExists",
dslQueryToCompile: "env EXISTS",
expectedSQL: `json_extract("rule"."data", '$.labels."env"') IS NOT NULL`,
},
{
subtestName: "State_LabelLookupNotRuleState",
dslQueryToCompile: "state = 'firing'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."state"'), '') = ?`,
expectedArgs: []any{"firing"},
},
})
}
func TestCompileReservedLabelCollisions(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "UppercaseReservedKey_MatchesReservedOrExactCaseLabel",
dslQueryToCompile: "NAME = 'x'",
expectedSQL: `(json_extract("rule"."data", '$.alert') = ? OR COALESCE(json_extract("rule"."data", '$.labels."NAME"'), '') = ?)`,
expectedArgs: []any{"x", "x"},
},
{
subtestName: "SeverityExactSpelling_SinglePredicate",
dslQueryToCompile: "severity = 'critical'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') = ?`,
expectedArgs: []any{"critical"},
},
{
subtestName: "SeverityDifferentCase_MatchesBothLabelSpellings",
dslQueryToCompile: "Severity = 'critical'",
expectedSQL: `(COALESCE(json_extract("rule"."data", '$.labels."severity"'), '') = ? ` +
`OR COALESCE(json_extract("rule"."data", '$.labels."Severity"'), '') = ?)`,
expectedArgs: []any{"critical", "critical"},
},
{
subtestName: "RangeOperator_ReservedOnly",
dslQueryToCompile: "created_at >= '2026-01-02T15:04:05Z'",
expectedSQL: `rule.created_at >= ?`,
expectedArgs: []any{time.Date(2026, 1, 2, 15, 4, 5, 0, time.UTC)},
},
{
subtestName: "LabelsPrefix_LabelOnlyOnCollision",
dslQueryToCompile: "labels.name = 'x'",
expectedSQL: `COALESCE(json_extract("rule"."data", '$.labels."name"'), '') = ?`,
expectedArgs: []any{"x"},
},
{
subtestName: "NotIn_ExcludesBoth",
dslQueryToCompile: "created_by NOT IN ['a', 'b']",
expectedSQL: `(rule.created_by NOT IN (?, ?) ` +
`AND COALESCE(json_extract("rule"."data", '$.labels."created_by"'), '') NOT IN (?, ?))`,
expectedArgs: []any{"a", "b", "a", "b"},
},
})
}
func TestCompileErrors(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "RangeOperatorOnBareLabelKey_Rejected",
dslQueryToCompile: "team > 'infra'",
expectedErrShouldContain: `operator > is not allowed on the label filter "team"`,
},
{
subtestName: "SyntaxError_SurfacesPosition",
dslQueryToCompile: "created_by ==== (((",
expectedErrShouldContain: "syntax error",
},
{
subtestName: "LikeDanglingEscape_Rejected",
dslQueryToCompile: `name LIKE 'prod\\'`,
expectedErrShouldContain: "must not end with an unescaped backslash",
},
{
subtestName: "ILikeDanglingEscape_Rejected",
dslQueryToCompile: `name ILIKE '%\\'`,
expectedErrShouldContain: "must not end with an unescaped backslash",
},
{
subtestName: "LabelLikeDanglingEscape_Rejected",
dslQueryToCompile: `labels.team NOT LIKE 'infra\\'`,
expectedErrShouldContain: "must not end with an unescaped backslash",
},
})
}
func TestCompileTrailingLiteralBackslash(t *testing.T) {
runCompileCases(t, []compileCase{
{
subtestName: "EscapedTrailingBackslash_Compiles",
dslQueryToCompile: `name LIKE '%\\\\'`,
expectedSQL: `(json_extract("rule"."data", '$.alert') LIKE ? ESCAPE '\' OR COALESCE(json_extract("rule"."data", '$.labels."name"'), '') LIKE ? ESCAPE '\')`,
expectedArgs: []any{`%\\`, `%\\`},
},
})
}
// Guards that every ruletypes.ReservedOps key has a case in resolveReservedKey.
func TestCompileReservedKeysAllHandled(t *testing.T) {
sampleQueries := map[ruletypes.DSLKey]string{
ruletypes.DSLKeyName: "name = 'x'",
ruletypes.DSLKeySeverity: "severity = 'critical'",
ruletypes.DSLKeyCreatedBy: "created_by = 'x'",
ruletypes.DSLKeyUpdatedBy: "updated_by = 'x'",
ruletypes.DSLKeyCreatedAt: "created_at >= '2026-01-02T15:04:05Z'",
ruletypes.DSLKeyUpdatedAt: "updated_at >= '2026-01-02T15:04:05Z'",
ruletypes.DSLKeyAlertType: "alert_type = 'METRIC_BASED_ALERT'",
ruletypes.DSLKeyRuleType: "rule_type = 'threshold_rule'",
}
for key := range ruletypes.ReservedOps {
query, ok := sampleQueries[key]
require.True(t, ok, "no sample query for reserved key %q, add one", key)
out, err := CompileListFilter(query, formatter(t))
require.NoError(t, err, "reserved key %q failed to compile", key)
assert.False(t, out.IsEmpty(), "reserved key %q compiled to empty SQL", key)
}
}
func formatter(t *testing.T) sqlstore.SQLFormatter {
t.Helper()
p := sqlstoretest.New(sqlstore.Config{Provider: "sqlite"}, sqlmock.QueryMatcherEqual)
return p.Formatter()
}
func normalizeSQL(s string) string {
s = strings.Join(strings.Fields(s), " ")
s = strings.ReplaceAll(s, "( ", "(")
s = strings.ReplaceAll(s, " )", ")")
return s
}

View File

@@ -851,6 +851,8 @@ func (m *Manager) ListRuleStates(ctx context.Context) (*ruletypes.GettableRules,
// initiate response object
resp := make([]*ruletypes.GettableRule, 0)
stateByRuleID := m.snapshotRuleStates()
for _, s := range storedRules {
ruleResponse := ruletypes.GettableRule{}
@@ -863,11 +865,11 @@ func (m *Manager) ListRuleStates(ctx context.Context) (*ruletypes.GettableRules,
ruleResponse.Id = s.ID.StringValue()
// fetch state of rule from memory
if rm, ok := m.rules[ruleResponse.Id]; !ok {
if state, ok := stateByRuleID[ruleResponse.Id]; !ok {
ruleResponse.State = ruletypes.StateDisabled
ruleResponse.Disabled = true
} else {
ruleResponse.State = rm.State()
ruleResponse.State = state
}
ruleResponse.CreatedAt = s.CreatedAt
ruleResponse.CreatedBy = &s.CreatedBy
@@ -879,6 +881,71 @@ func (m *Manager) ListRuleStates(ctx context.Context) (*ruletypes.GettableRules,
return &ruletypes.GettableRules{Rules: resp}, nil
}
// ListRules' total counts what is pageable after corrupt-row drops and the states filter.
func (m *Manager) ListRules(ctx context.Context, params *ruletypes.ListRulesParams) (*ruletypes.ListableRules, error) {
// validated here too, not just in the handler: non-API callers reach the manager directly
if err := params.Validate(); err != nil {
return nil, err
}
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
return nil, err
}
states, err := params.GetAlertStates()
if err != nil {
return nil, err
}
stateFilter := make(map[ruletypes.AlertState]struct{}, len(states))
for _, state := range states {
stateFilter[state] = struct{}{}
}
compiled, err := CompileListFilter(params.Query, m.sqlstore.Formatter())
if err != nil {
return nil, err
}
storedRules, err := m.ruleStore.GetStoredRulesMatching(ctx, claims.OrgID, compiled.SQL, compiled.Args)
if err != nil {
return nil, err
}
stateByRuleID := m.snapshotRuleStates()
listableRules, errByRuleID := ruletypes.NewListableRulesFromStorableRules(storedRules, stateByRuleID, stateFilter)
for ruleID, err := range errByRuleID {
m.logger.ErrorContext(ctx, "failed to unmarshal rule from db", slog.String("rule.id", ruleID), errors.Attr(err))
}
total := int64(len(listableRules))
ruletypes.SortListableRules(listableRules, params.Sort, params.Order)
start := min(params.Offset, len(listableRules))
end := min(start+params.Limit, len(listableRules))
currentPageRules := listableRules[start:end]
rawLabels, err := m.ruleStore.GetStoredRuleLabels(ctx, claims.OrgID)
if err != nil {
return nil, err
}
labelPairs := ruletypes.NewLabelPairsFromRawJSON(rawLabels, ruletypes.MaxListLabelPairs)
return ruletypes.NewListableRules(currentPageRules, total, labelPairs), nil
}
func (m *Manager) snapshotRuleStates() map[string]ruletypes.AlertState {
m.mtx.RLock()
defer m.mtx.RUnlock()
states := make(map[string]ruletypes.AlertState, len(m.rules))
for id, rule := range m.rules {
states[id] = rule.State()
}
return states
}
func (m *Manager) GetRule(ctx context.Context, id valuer.UUID) (*ruletypes.GettableRule, error) {
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {

View File

@@ -20,6 +20,7 @@ import (
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
@@ -28,6 +29,17 @@ import (
cmock "github.com/SigNoz/clickhouse-go-mock"
)
func TestManager_ListRules_ValidatesParams(t *testing.T) {
m, err := NewManager(&ManagerOptions{})
require.NoError(t, err)
_, err = m.ListRules(context.Background(), &ruletypes.ListRulesParams{Limit: -1})
require.ErrorContains(t, err, "invalid limit")
_, err = m.ListRules(context.Background(), &ruletypes.ListRulesParams{States: []string{"bogus"}})
require.ErrorContains(t, err, `invalid state "bogus"`)
}
func TestManager_TestNotification_SendUnmatched_ThresholdRule(t *testing.T) {
target := 10.0
recovery := 5.0

View File

@@ -94,10 +94,10 @@ func ExistsExpression(columns []*schema.Column, key *telemetrytypes.TelemetryFie
switch valueType := column.Type.(schema.MapColumnType).ValueType; valueType.GetType() {
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumBool, schema.ColumnTypeEnumFloat64:
leftOperand := fmt.Sprintf("mapContains(%s, %s)", column.Name, clickhousesql.StringLiteral(key.Name))
if key.Materialized {
leftOperand = telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key)
return telemetrytypes.FieldKeyToMaterializedExistsCondition(key, exists), nil
}
leftOperand := fmt.Sprintf("mapContains(%s, %s)", column.Name, clickhousesql.StringLiteral(key.Name))
if exists {
return leftOperand, nil
}

View File

@@ -24,7 +24,9 @@ func NewQueryInfo(ctx context.Context, orgID valuer.UUID, fl flagger.Flagger, si
FamiliesOn: semconvFamiliesEnabled(ctx, orgID, fl),
}
if fl != nil {
q.BodyJSONOn = fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID))
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)
q.BodyJSONOn = fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, evalCtx)
q.TraceAttrsJSONOn = fl.BooleanOrEmpty(ctx, flagger.FeatureUseTraceAttributesJSON, evalCtx)
}
return q
}

View File

@@ -4,6 +4,7 @@ import "net/http"
type Handler interface {
ListRules(http.ResponseWriter, *http.Request)
ListRulesV3(http.ResponseWriter, *http.Request)
GetRuleByID(http.ResponseWriter, *http.Request)
CreateRule(http.ResponseWriter, *http.Request)
UpdateRuleByID(http.ResponseWriter, *http.Request)

View File

@@ -17,6 +17,9 @@ type Ruler interface {
// ListRuleStates returns all rules with their current evaluation state.
ListRuleStates(ctx context.Context) (*ruletypes.GettableRules, error)
// ListRules returns a filtered, sorted page of rules with state, plus label pairs and reserved filter keys.
ListRules(ctx context.Context, params *ruletypes.ListRulesParams) (*ruletypes.ListableRules, error)
// GetRule returns a single rule by ID.
GetRule(ctx context.Context, id valuer.UUID) (*ruletypes.GettableRule, error)

View File

@@ -64,6 +64,16 @@ func (m *MockSQLRuleStore) GetStoredRules(ctx context.Context, orgID string) ([]
return m.ruleStore.GetStoredRules(ctx, orgID)
}
// GetStoredRulesMatching implements ruletypes.RuleStore - delegates to underlying ruleStore to trigger SQL.
func (m *MockSQLRuleStore) GetStoredRulesMatching(ctx context.Context, orgID string, filterSQL string, filterArgs []any) ([]*ruletypes.StorableRule, error) {
return m.ruleStore.GetStoredRulesMatching(ctx, orgID, filterSQL, filterArgs)
}
// GetStoredRuleLabels implements ruletypes.RuleStore - delegates to underlying ruleStore to trigger SQL.
func (m *MockSQLRuleStore) GetStoredRuleLabels(ctx context.Context, orgID string) ([]string, error) {
return m.ruleStore.GetStoredRuleLabels(ctx, orgID)
}
// GetStoredRulesByMetricName implements ruletypes.RuleStore - delegates to underlying ruleStore.
func (m *MockSQLRuleStore) GetStoredRulesByMetricName(ctx context.Context, orgID string, metricName string) ([]ruletypes.RuleAlert, error) {
return m.ruleStore.GetStoredRulesByMetricName(ctx, orgID, metricName)

View File

@@ -3,6 +3,7 @@ package sqlrulestore
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"slices"
@@ -89,6 +90,41 @@ func (r *rule) DeleteRule(ctx context.Context, orgID valuer.UUID, id valuer.UUID
return nil
}
func (r *rule) GetStoredRulesMatching(ctx context.Context, orgID string, filterSQL string, filterArgs []any) ([]*ruletypes.StorableRule, error) {
rules := make([]*ruletypes.StorableRule, 0)
q := r.sqlstore.
BunDB().
NewSelect().
Model(&rules).
Where("org_id = ?", orgID)
if filterSQL != "" {
q = q.Where(filterSQL, filterArgs...)
}
if err := q.Scan(ctx); err != nil {
return nil, err
}
return rules, nil
}
func (r *rule) GetStoredRuleLabels(ctx context.Context, orgID string) ([]string, error) {
labelsExpression := string(r.sqlstore.Formatter().JSONExtractString("rule.data", "$.labels"))
labels := make([]string, 0)
err := r.sqlstore.
BunDB().
NewSelect().
Model((*ruletypes.StorableRule)(nil)).
ColumnExpr(fmt.Sprintf("COALESCE(%s, '')", labelsExpression)).
Where("org_id = ?", orgID).
Scan(ctx, &labels)
if err != nil {
return nil, err
}
return labels, nil
}
func (r *rule) GetStoredRules(ctx context.Context, orgID string) ([]*ruletypes.StorableRule, error) {
rules := make([]*ruletypes.StorableRule, 0)
err := r.sqlstore.

View File

@@ -43,6 +43,29 @@ func (handler *handler) ListRules(rw http.ResponseWriter, req *http.Request) {
render.Success(rw, http.StatusOK, view)
}
func (handler *handler) ListRulesV3(rw http.ResponseWriter, req *http.Request) {
ctx, cancel := context.WithTimeout(req.Context(), 30*time.Second)
defer cancel()
params := new(ruletypes.ListRulesParams)
if err := binding.Query.BindQuery(req.URL.Query(), params); err != nil {
render.Error(rw, err)
return
}
if err := params.Validate(); err != nil {
render.Error(rw, err)
return
}
listableRules, err := handler.ruler.ListRules(ctx, params)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, listableRules)
}
func (handler *handler) GetRuleByID(rw http.ResponseWriter, req *http.Request) {
ctx, cancel := context.WithTimeout(req.Context(), 30*time.Second)
defer cancel()

View File

@@ -116,6 +116,10 @@ func (provider *provider) ListRuleStates(ctx context.Context) (*ruletypes.Gettab
return provider.manager.ListRuleStates(ctx)
}
func (provider *provider) ListRules(ctx context.Context, params *ruletypes.ListRulesParams) (*ruletypes.ListableRules, error) {
return provider.manager.ListRules(ctx, params)
}
func (provider *provider) GetRule(ctx context.Context, id valuer.UUID) (*ruletypes.GettableRule, error) {
return provider.manager.GetRule(ctx, id)
}

View File

@@ -174,6 +174,7 @@ func (openapi *OpenAPI) CreateAndWrite(path string) error {
}
attachDiscriminators(openapi.reflector.Spec)
openapi.collector.AttachStabilities(openapi.reflector.Spec)
// The library's MarshalYAML does a JSON round-trip that converts all numbers
// to float64, causing large integers (e.g. epoch millisecond timestamps) to

View File

@@ -1,6 +1,7 @@
package sqlitesqlstore
import (
"fmt"
"strings"
"github.com/SigNoz/signoz/pkg/sqlstore"
@@ -25,6 +26,12 @@ func (f *formatter) JSONExtractString(column, path string) []byte {
return sql
}
func (f *formatter) JSONExtractMapValue(column, mapField, key string) []byte {
// Quote the key as one path segment; a double quote in it is inexpressible in sqlite JSON paths.
escapedKey := strings.NewReplacer(`\`, `\\`).Replace(key)
return f.JSONExtractString(column, fmt.Sprintf(`$.%s."%s"`, mapField, escapedKey))
}
func (f *formatter) JSONType(column, path string) []byte {
var sql []byte
sql = append(sql, "json_type("...)

View File

@@ -55,6 +55,60 @@ func TestJSONExtractString(t *testing.T) {
}
}
func TestJSONExtractMapValue(t *testing.T) {
tests := []struct {
name string
column string
mapField string
key string
expected string
}{
{
name: "PlainKey",
column: "data",
mapField: "labels",
key: "team",
expected: `json_extract("data", '$.labels."team"')`,
},
{
name: "DottedKey_OneMapEntry",
column: "data",
mapField: "labels",
key: "k8s.cluster",
expected: `json_extract("data", '$.labels."k8s.cluster"')`,
},
{
name: "BackslashInKey_Escaped",
column: "data",
mapField: "labels",
key: `a\b`,
expected: `json_extract("data", '$.labels."a\\b"')`,
},
{
name: "SingleQuoteInKey_Doubled",
column: "data",
mapField: "labels",
key: "o'brien",
expected: `json_extract("data", '$.labels."o''brien"')`,
},
{
name: "QualifiedColumn",
column: "rule.data",
mapField: "labels",
key: "severity",
expected: `json_extract("rule"."data", '$.labels."severity"')`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f := newFormatter(sqlitedialect.New())
got := string(f.JSONExtractMapValue(tt.column, tt.mapField, tt.key))
assert.Equal(t, tt.expected, got)
})
}
}
func TestJSONType(t *testing.T) {
tests := []struct {
name string

View File

@@ -114,6 +114,9 @@ type SQLFormatter interface {
// JSONKeys return extracted key from json as well as alias to be used for select and where clause
JSONKeys(column, path, alias string) ([]byte, []byte)
// JSONExtractMapValue extracts one key's value from a JSON object field; dots in the key are not path nesting.
JSONExtractMapValue(column, mapField, key string) []byte
// TextToJsonColumn converts a text column to JSON type
TextToJsonColumn(column string) []byte

View File

@@ -1,6 +1,7 @@
package sqlstoretest
import (
"fmt"
"strings"
"github.com/SigNoz/signoz/pkg/sqlstore"
@@ -25,6 +26,11 @@ func (f *formatter) JSONExtractString(column, path string) []byte {
return sql
}
func (f *formatter) JSONExtractMapValue(column, mapField, key string) []byte {
escapedKey := strings.NewReplacer(`\`, `\\`).Replace(key)
return f.JSONExtractString(column, fmt.Sprintf(`$.%s."%s"`, mapField, escapedKey))
}
func (f *formatter) JSONType(column, path string) []byte {
var sql []byte
sql = append(sql, "json_type("...)

View File

@@ -0,0 +1,62 @@
package sqlstoretest
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/uptrace/bun/dialect/sqlitedialect"
)
func TestJSONExtractMapValue(t *testing.T) {
tests := []struct {
name string
column string
mapField string
key string
expected string
}{
{
name: "PlainKey",
column: "data",
mapField: "labels",
key: "team",
expected: `json_extract("data", '$.labels."team"')`,
},
{
name: "DottedKey_OneMapEntry",
column: "data",
mapField: "labels",
key: "k8s.cluster",
expected: `json_extract("data", '$.labels."k8s.cluster"')`,
},
{
name: "BackslashInKey_Escaped",
column: "data",
mapField: "labels",
key: `a\b`,
expected: `json_extract("data", '$.labels."a\\b"')`,
},
{
name: "SingleQuoteInKey_Doubled",
column: "data",
mapField: "labels",
key: "o'brien",
expected: `json_extract("data", '$.labels."o''brien"')`,
},
{
name: "QualifiedColumn",
column: "rule.data",
mapField: "labels",
key: "severity",
expected: `json_extract("rule"."data", '$.labels."severity"')`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f := newFormatter(sqlitedialect.New())
got := string(f.JSONExtractMapValue(tt.column, tt.mapField, tt.key))
assert.Equal(t, tt.expected, got)
})
}
}

View File

@@ -237,13 +237,13 @@ func TestBuild_FullSQL_TraceList_MaterializedColumns(t *testing.T) {
assertSQLEqual(t, `
WITH matched AS (
SELECT trace_id,
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists = true OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time
FROM signoz_traces.distributed_signoz_index_v3
WHERE timestamp >= '1747947419000000000'
AND timestamp < '1747983448000000000'
AND ts_bucket_start >= 1747945619
AND ts_bucket_start <= 1747983448
AND ((attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
AND ((attribute_string_gen_ai$$request$$model_exists = true OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
GROUP BY trace_id
ORDER BY last_activity_time DESC, trace_id DESC
LIMIT 20
@@ -268,16 +268,16 @@ SELECT trace_id,
count() AS span_count,
anyIf(name, parent_span_id = '') AS root_span_name,
any(multiIf(resource.service.name IS NOT NULL, resource.service.name::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS service.name,
countIf(attribute_string_gen_ai$$request$$model_exists) AS llm_call_count,
countIf(attribute_string_gen_ai$$request$$model_exists = true) AS llm_call_count,
countIf(mapContains(attributes_string, 'gen_ai.tool.name')) AS tool_call_count,
uniqIf(multiIf(mapContains(attributes_string, 'gen_ai.tool.name'), attributes_string['gen_ai.tool.name'], NULL), mapContains(attributes_string, 'gen_ai.tool.name')) AS distinct_tool_count,
sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)) AS input_tokens,
sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists = true, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)) AS input_tokens,
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens,
coalesce(sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
coalesce(sum(multiIf(attribute_number_gen_ai$$usage$$input_tokens_exists = true, toFloat64(attribute_number_gen_ai$$usage$$input_tokens), NULL)), 0) + coalesce(sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)), 0) AS total_tokens,
sum(multiIf(mapContains(attributes_number, 'signoz.gen_ai.usage.tokens.cost'), toFloat64(attributes_number['signoz.gen_ai.usage.tokens.cost']), NULL)) AS estimated_total_cost,
maxIf(duration_nano, attribute_string_gen_ai$$request$$model_exists) AS max_llm_duration_nano,
maxIf(duration_nano, attribute_string_gen_ai$$request$$model_exists = true) AS max_llm_duration_nano,
countIf(has_error = true) AS error_count,
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
maxIf(timestamp, (attribute_string_gen_ai$$request$$model_exists = true OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name'))) AS last_activity_time,
argMinIf(multiIf(mapContains(attributes_string, 'gen_ai.input.messages'), attributes_string['gen_ai.input.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.input.messages')) AS input,
argMaxIf(multiIf(mapContains(attributes_string, 'gen_ai.output.messages'), attributes_string['gen_ai.output.messages'], NULL), timestamp, mapContains(attributes_string, 'gen_ai.output.messages')) AS output
FROM signoz_traces.distributed_signoz_index_v3

View File

@@ -92,7 +92,7 @@ func TestStatementBuilder(t *testing.T) {
Limit: 100,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$principal$$id` = ? AND `attribute_string_signoz$$audit$$principal$$id_exists`) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$principal$$id` = ? AND `attribute_string_signoz$$audit$$principal$$id_exists` = true) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"019a-1234-abcd-5678", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
@@ -109,7 +109,7 @@ func TestStatementBuilder(t *testing.T) {
Limit: 100,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists`) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = true) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"failure", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
@@ -143,7 +143,7 @@ func TestStatementBuilder(t *testing.T) {
Limit: 100,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_audit.distributed_logs_resource WHERE (simpleJSONExtractString(labels, 'signoz.audit.resource.kind') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND (`attribute_string_signoz$$audit$$action` = ? AND `attribute_string_signoz$$audit$$action_exists`) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_audit.distributed_logs_resource WHERE (simpleJSONExtractString(labels, 'signoz.audit.resource.kind') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND (`attribute_string_signoz$$audit$$action` = ? AND `attribute_string_signoz$$audit$$action_exists` = true) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"dashboard", "%signoz.audit.resource.kind%", "%signoz.audit.resource.kind\":\"dashboard%", uint64(1747945619), uint64(1747983448), "delete", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
@@ -160,7 +160,7 @@ func TestStatementBuilder(t *testing.T) {
Limit: 100,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$principal$$type` = ? AND `attribute_string_signoz$$audit$$principal$$type_exists`) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, event_name, attributes_string, attributes_number, attributes_bool, resource, scope_string FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$principal$$type` = ? AND `attribute_string_signoz$$audit$$principal$$type_exists` = true) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"service_account", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 100},
},
},
@@ -180,7 +180,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "SELECT count() AS __result_0 FROM signoz_audit.distributed_logs WHERE ((`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists`) AND (`attribute_string_signoz$$audit$$action` = ? AND `attribute_string_signoz$$audit$$action_exists`)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY __result_0 DESC",
Query: "SELECT count() AS __result_0 FROM signoz_audit.distributed_logs WHERE ((`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = true) AND (`attribute_string_signoz$$audit$$action` = ? AND `attribute_string_signoz$$audit$$action_exists` = true)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY __result_0 DESC",
Args: []any{"failure", "update", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448)},
},
},
@@ -204,7 +204,7 @@ func TestStatementBuilder(t *testing.T) {
Limit: 5,
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(`attribute_string_signoz$$audit$$principal$$email_exists`, `attribute_string_signoz$$audit$$principal$$email`, NULL)) AS `signoz.audit.principal.email`, count() AS __result_0 FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists`) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? GROUP BY `signoz.audit.principal.email` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 60 SECOND) AS ts, toString(multiIf(`attribute_string_signoz$$audit$$principal$$email_exists`, `attribute_string_signoz$$audit$$principal$$email`, NULL)) AS `signoz.audit.principal.email`, count() AS __result_0 FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists`) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? AND (`signoz.audit.principal.email`) GLOBAL IN (SELECT `signoz.audit.principal.email` FROM __limit_cte) GROUP BY ts, `signoz.audit.principal.email`",
Query: "WITH __limit_cte AS (SELECT toString(multiIf(`attribute_string_signoz$$audit$$principal$$email_exists` = true, `attribute_string_signoz$$audit$$principal$$email`, NULL)) AS `signoz.audit.principal.email`, count() AS __result_0 FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = true) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? GROUP BY `signoz.audit.principal.email` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 60 SECOND) AS ts, toString(multiIf(`attribute_string_signoz$$audit$$principal$$email_exists` = true, `attribute_string_signoz$$audit$$principal$$email`, NULL)) AS `signoz.audit.principal.email`, count() AS __result_0 FROM signoz_audit.distributed_logs WHERE (`attribute_string_signoz$$audit$$outcome` = ? AND `attribute_string_signoz$$audit$$outcome_exists` = true) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? AND (`signoz.audit.principal.email`) GLOBAL IN (SELECT `signoz.audit.principal.email` FROM __limit_cte) GROUP BY ts, `signoz.audit.principal.email`",
Args: []any{"failure", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 5, "failure", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448)},
},
},

View File

@@ -180,7 +180,7 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(`attribute_string_materialized$$key$$name_exists`, `attribute_string_materialized$$key$$name`, NULL)) AS `__GROUP_BY_KEY_0_materialized.key.name`, count() AS __result_0 FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_materialized.key.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 30 SECOND) AS ts, toString(multiIf(`attribute_string_materialized$$key$$name_exists`, `attribute_string_materialized$$key$$name`, NULL)) AS `__GROUP_BY_KEY_0_materialized.key.name`, count() AS __result_0 FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_materialized.key.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_materialized.key.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_materialized.key.name`",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(`attribute_string_materialized$$key$$name_exists` = true, `attribute_string_materialized$$key$$name`, NULL)) AS `__GROUP_BY_KEY_0_materialized.key.name`, count() AS __result_0 FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_materialized.key.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 30 SECOND) AS ts, toString(multiIf(`attribute_string_materialized$$key$$name_exists` = true, `attribute_string_materialized$$key$$name`, NULL)) AS `__GROUP_BY_KEY_0_materialized.key.name`, count() AS __result_0 FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_materialized.key.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_materialized.key.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_materialized.key.name`",
Args: []any{"cartservice", "%service.name%", "%service.name\":\"cartservice%", uint64(1705397400), uint64(1705485600), "1705399200000000000", uint64(1705397400), "1705485600000000000", uint64(1705485600), 10, "1705399200000000000", uint64(1705397400), "1705485600000000000", uint64(1705485600)},
},
},
@@ -203,7 +203,7 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 30 SECOND) AS ts, count() AS __result_0 FROM signoz_logs.distributed_logs_v2 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists`) OR (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists`)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? GROUP BY ts",
Query: "SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 30 SECOND) AS ts, count() AS __result_0 FROM signoz_logs.distributed_logs_v2 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = true) OR (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists` = true)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? GROUP BY ts",
Args: []any{"redis.*", "memcached", "1705399200000000000", uint64(1705397400), "1705485600000000000", uint64(1705485600)},
},
expectedErr: nil,
@@ -300,7 +300,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists`, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists` = true, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Args: []any{"cartservice", "%service.name%", "%service.name\":\"cartservice%", uint64(1747945619), uint64(1747983448), "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
expectedErr: nil,
@@ -328,7 +328,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists`) OR (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists`)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists`, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = true) OR (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists` = true)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists` = true, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Args: []any{"redis.*", "memcached", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
expectedErr: nil,
@@ -442,7 +442,7 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND match(LOWER(body), LOWER(?)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists`, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND match(LOWER(body), LOWER(?)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists` = true, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Args: []any{"cartservice", "%service.name%", "%service.name\":\"cartservice%", uint64(1747945619), uint64(1747983448), "hello", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
expectedErr: nil,
@@ -666,7 +666,7 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND LOWER(body) LIKE LOWER(?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists`, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_logs.distributed_logs_v2_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND LOWER(body) LIKE LOWER(?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? ORDER BY multiIf(`attribute_string_materialized$$key$$name_exists` = true, `attribute_string_materialized$$key$$name`, NULL) desc LIMIT ?",
Args: []any{"cartservice", "%service.name%", "%service.name\":\"cartservice%", uint64(1747945619), uint64(1747983448), "%error%", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
expectedErr: nil,

View File

@@ -7,6 +7,7 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
@@ -24,7 +25,7 @@ var jsonAttrColRe = regexp.MustCompile(`,\s*attributes\s*(,| FROM )`)
func newBulkTestBuilder(t *testing.T, releaseTime time.Time) *traceQueryStatementBuilder {
t.Helper()
fl := flaggertest.New(t)
fl := flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureUseTraceAttributesJSON.String(): true})
storage := tracestelemetryschema.NewStorage()
store := telemetrytypestest.NewMockMetadataStore()
store.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)

View File

@@ -129,7 +129,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists`) OR multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists`) OR multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Query: "WITH __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = true) OR multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = true) OR multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"redis-manual", "redis-manual", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "redis-manual", "redis-manual", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -268,7 +268,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists`, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists`, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = true, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY __result_0 DESC LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = true, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name`",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -307,7 +307,7 @@ func TestStatementBuilder(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists`, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY `__GROUP_BY_KEY_0_service.name` desc LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists`, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name` ORDER BY `__GROUP_BY_KEY_0_service.name` desc, ts desc",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint), __limit_cte AS (SELECT toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = true, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? GROUP BY `__GROUP_BY_KEY_0_service.name` ORDER BY `__GROUP_BY_KEY_0_service.name` desc LIMIT ?) SELECT toStartOfInterval(timestamp, INTERVAL 30 SECOND) AS ts, toString(multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL)) AS `__GROUP_BY_KEY_0_service.name`, sum(multiIf(`attribute_number_cart$$items_count_exists` = true, toFloat64(`attribute_number_cart$$items_count`), NULL)) AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? AND (`__GROUP_BY_KEY_0_service.name`) GLOBAL IN (SELECT `__GROUP_BY_KEY_0_service.name` FROM __limit_cte) GROUP BY ts, `__GROUP_BY_KEY_0_service.name` ORDER BY `__GROUP_BY_KEY_0_service.name` desc, ts desc",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10, "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
expectedErr: nil,
@@ -552,7 +552,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
},
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) AS `__SELECT_KEY_4_service.name`, duration_nano AS `__SELECT_KEY_5_duration_nano`, multiIf(`attribute_number_cart$$items_count_exists`, `attribute_number_cart$$items_count`, NULL) AS `__SELECT_KEY_6_cart.items_count` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) AS `__SELECT_KEY_4_service.name`, duration_nano AS `__SELECT_KEY_5_duration_nano`, multiIf(`attribute_number_cart$$items_count_exists` = true, `attribute_number_cart$$items_count`, NULL) AS `__SELECT_KEY_6_cart.items_count` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -669,7 +669,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, multiIf(`attribute_string_mixed$$materialization$$key_exists`, `attribute_string_mixed$$materialization$$key`, multiIf(resource.`mixed.materialization.key` IS NOT NULL, resource.`mixed.materialization.key`::String, mapContains(resources_string, 'mixed.materialization.key'), resources_string['mixed.materialization.key'], NULL) IS NOT NULL, multiIf(resource.`mixed.materialization.key` IS NOT NULL, resource.`mixed.materialization.key`::String, mapContains(resources_string, 'mixed.materialization.key'), resources_string['mixed.materialization.key'], NULL), NULL) AS `__SELECT_KEY_7_mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, multiIf(`attribute_string_mixed$$materialization$$key_exists` = true, `attribute_string_mixed$$materialization$$key`, multiIf(resource.`mixed.materialization.key` IS NOT NULL, resource.`mixed.materialization.key`::String, mapContains(resources_string, 'mixed.materialization.key'), resources_string['mixed.materialization.key'], NULL) IS NOT NULL, multiIf(resource.`mixed.materialization.key` IS NOT NULL, resource.`mixed.materialization.key`::String, mapContains(resources_string, 'mixed.materialization.key'), resources_string['mixed.materialization.key'], NULL), NULL) AS `__SELECT_KEY_7_mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -714,7 +714,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, multiIf(`attribute_string_mixed$$materialization$$key_exists`, `attribute_string_mixed$$materialization$$key`, NULL) AS `__SELECT_KEY_7_mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT timestamp AS `__SELECT_KEY_0_timestamp`, trace_id AS `__SELECT_KEY_1_trace_id`, span_id AS `__SELECT_KEY_2_span_id`, name AS `__SELECT_KEY_3_name`, resource_string_service$$name AS `__SELECT_KEY_4_serviceName`, duration_nano AS `__SELECT_KEY_5_durationNano`, http_method AS `__SELECT_KEY_6_httpMethod`, multiIf(`attribute_string_mixed$$materialization$$key_exists` = true, `attribute_string_mixed$$materialization$$key`, NULL) AS `__SELECT_KEY_7_mixed.materialization.key` FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"redis-manual", "%service.name%", "%service.name\":\"redis-manual%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -1178,7 +1178,7 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __toe AS (SELECT trace_id FROM signoz_traces.distributed_signoz_index_v3 WHERE (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists`) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __toe_duration_sorted AS (SELECT trace_id, duration_nano, resource_string_service$$name as `service.name`, name FROM signoz_traces.distributed_signoz_index_v3 WHERE parent_span_id = '' AND trace_id GLOBAL IN __toe AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY duration_nano DESC LIMIT 1 BY trace_id) SELECT __toe_duration_sorted.`service.name` AS `service.name`, __toe_duration_sorted.name AS `name`, count() AS span_count, __toe_duration_sorted.duration_nano AS `duration_nano`, __toe_duration_sorted.trace_id AS `trace_id` FROM __toe INNER JOIN __toe_duration_sorted ON __toe.trace_id = __toe_duration_sorted.trace_id GROUP BY trace_id, duration_nano, name, `service.name` ORDER BY duration_nano DESC LIMIT 1 BY trace_id LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH __toe AS (SELECT trace_id FROM signoz_traces.distributed_signoz_index_v3 WHERE (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists` = true) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __toe_duration_sorted AS (SELECT trace_id, duration_nano, resource_string_service$$name as `service.name`, name FROM signoz_traces.distributed_signoz_index_v3 WHERE parent_span_id = '' AND trace_id GLOBAL IN __toe AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY duration_nano DESC LIMIT 1 BY trace_id) SELECT __toe_duration_sorted.`service.name` AS `service.name`, __toe_duration_sorted.name AS `name`, count() AS span_count, __toe_duration_sorted.duration_nano AS `duration_nano`, __toe_duration_sorted.trace_id AS `trace_id` FROM __toe INNER JOIN __toe_duration_sorted ON __toe.trace_id = __toe_duration_sorted.trace_id GROUP BY trace_id, duration_nano, name, `service.name` ORDER BY duration_nano DESC LIMIT 1 BY trace_id LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"redis-manual", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -1194,7 +1194,7 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __toe AS (SELECT trace_id FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists`) OR multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __toe_duration_sorted AS (SELECT trace_id, duration_nano, resource_string_service$$name as `service.name`, name FROM signoz_traces.distributed_signoz_index_v3 WHERE parent_span_id = '' AND trace_id GLOBAL IN __toe AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY duration_nano DESC LIMIT 1 BY trace_id) SELECT __toe_duration_sorted.`service.name` AS `service.name`, __toe_duration_sorted.name AS `name`, count() AS span_count, __toe_duration_sorted.duration_nano AS `duration_nano`, __toe_duration_sorted.trace_id AS `trace_id` FROM __toe INNER JOIN __toe_duration_sorted ON __toe.trace_id = __toe_duration_sorted.trace_id GROUP BY trace_id, duration_nano, name, `service.name` ORDER BY duration_nano DESC LIMIT 1 BY trace_id LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH __toe AS (SELECT trace_id FROM signoz_traces.distributed_signoz_index_v3 WHERE ((match(`attribute_string_materialized$$key$$name`, ?) AND `attribute_string_materialized$$key$$name_exists` = true) OR multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, mapContains(resources_string, 'service.name'), resources_string['service.name'], NULL) = ?) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __toe_duration_sorted AS (SELECT trace_id, duration_nano, resource_string_service$$name as `service.name`, name FROM signoz_traces.distributed_signoz_index_v3 WHERE parent_span_id = '' AND trace_id GLOBAL IN __toe AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY duration_nano DESC LIMIT 1 BY trace_id) SELECT __toe_duration_sorted.`service.name` AS `service.name`, __toe_duration_sorted.name AS `name`, count() AS span_count, __toe_duration_sorted.duration_nano AS `duration_nano`, __toe_duration_sorted.trace_id AS `trace_id` FROM __toe INNER JOIN __toe_duration_sorted ON __toe.trace_id = __toe_duration_sorted.trace_id GROUP BY trace_id, duration_nano, name, `service.name` ORDER BY duration_nano DESC LIMIT 1 BY trace_id LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"redis-manual", "redis-manual", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,
@@ -1240,7 +1240,7 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "WITH __toe AS (SELECT trace_id FROM signoz_traces.distributed_signoz_index_v3 WHERE (((name, resource_string_service$$name) GLOBAL IN (SELECT DISTINCT name, serviceName from signoz_traces.distributed_top_level_operations WHERE time >= toDateTime(1747947419))) AND parent_span_id != '' OR (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists`)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __toe_duration_sorted AS (SELECT trace_id, duration_nano, resource_string_service$$name as `service.name`, name FROM signoz_traces.distributed_signoz_index_v3 WHERE parent_span_id = '' AND trace_id GLOBAL IN __toe AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY duration_nano DESC LIMIT 1 BY trace_id) SELECT __toe_duration_sorted.`service.name` AS `service.name`, __toe_duration_sorted.name AS `name`, count() AS span_count, __toe_duration_sorted.duration_nano AS `duration_nano`, __toe_duration_sorted.trace_id AS `trace_id` FROM __toe INNER JOIN __toe_duration_sorted ON __toe.trace_id = __toe_duration_sorted.trace_id GROUP BY trace_id, duration_nano, name, `service.name` ORDER BY duration_nano DESC LIMIT 1 BY trace_id LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Query: "WITH __toe AS (SELECT trace_id FROM signoz_traces.distributed_signoz_index_v3 WHERE (((name, resource_string_service$$name) GLOBAL IN (SELECT DISTINCT name, serviceName from signoz_traces.distributed_top_level_operations WHERE time >= toDateTime(1747947419))) AND parent_span_id != '' OR (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists` = true)) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ?), __toe_duration_sorted AS (SELECT trace_id, duration_nano, resource_string_service$$name as `service.name`, name FROM signoz_traces.distributed_signoz_index_v3 WHERE parent_span_id = '' AND trace_id GLOBAL IN __toe AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY duration_nano DESC LIMIT 1 BY trace_id) SELECT __toe_duration_sorted.`service.name` AS `service.name`, __toe_duration_sorted.name AS `name`, count() AS span_count, __toe_duration_sorted.duration_nano AS `duration_nano`, __toe_duration_sorted.trace_id AS `trace_id` FROM __toe INNER JOIN __toe_duration_sorted ON __toe.trace_id = __toe_duration_sorted.trace_id GROUP BY trace_id, duration_nano, name, `service.name` ORDER BY duration_nano DESC LIMIT 1 BY trace_id LIMIT ? SETTINGS distributed_product_mode='allow', max_memory_usage=10000000000",
Args: []any{"redis-manual", "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448), 10},
},
expectedErr: nil,

View File

@@ -16,6 +16,7 @@ import (
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/types/promotetypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/huandu/go-sqlbuilder"
)
@@ -33,6 +34,10 @@ var (
CodeFailedToAppendPath = errors.MustNewCode("failed_to_append_path_promoted_paths")
)
// logsBodyPromotedEntry templates the column evolution rows recorded for
// logs body promotions.
var logsBodyPromotedEntry = promotetypes.NewLogsBodyTarget().Entry
// enrichJSONKeys enriches body-context keys with promoted path info, indexes,
// and JSON access plans. parentTypeCache contains parent array types (ArrayJSON/ArrayDynamic)
// pre-fetched in the main UNION query.
@@ -67,7 +72,7 @@ func (t *telemetryMetaStore) enrichJSONKeys(ctx context.Context, selectors []*te
}
// fetch promoted paths
promoted, err := t.GetPromotedPaths(ctx, paths...)
promoted, err := t.GetPromotedPaths(ctx, logsBodyPromotedEntry, paths...)
if err != nil {
return err
}
@@ -157,7 +162,7 @@ func buildListLogsJSONIndexesQuery(cluster string, filters ...string) (string, [
}
func (t *telemetryMetaStore) ListLogsJSONIndexes(ctx context.Context, filters ...string) ([]telemetrytypes.TelemetryFieldKeySkipIndex, error) {
ctx = withTelemetryContext(ctx, "ListLogsJSONIndexes")
ctx = withTelemetryContext(ctx, telemetrytypes.SignalLogs, "ListLogsJSONIndexes")
query, args := buildListLogsJSONIndexesQuery(t.telemetrystore.Cluster(), filters...)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
@@ -215,14 +220,14 @@ func (t *telemetryMetaStore) ListLogsJSONIndexes(ctx context.Context, filters ..
// TODO(Piyush): Remove this if not used in future.
func (t *telemetryMetaStore) ListJSONValues(ctx context.Context, path string, limit int) (*telemetrytypes.TelemetryFieldValues, bool, error) {
ctx = withTelemetryContext(ctx, "ListJSONValues")
ctx = withTelemetryContext(ctx, telemetrytypes.SignalLogs, "ListJSONValues")
path = CleanPathPrefixes(path)
if strings.Contains(path, telemetrytypes.ArraySep) || strings.Contains(path, telemetrytypes.ArrayAnyIndex) {
return nil, false, errors.NewInvalidInputf(errors.CodeInvalidInput, "array paths are not supported")
}
promoted, err := t.IsPathPromoted(ctx, path)
promoted, err := t.isPathPromoted(ctx, logsBodyPromotedEntry, path)
if err != nil {
return nil, false, err
}
@@ -376,13 +381,13 @@ func derefValue(v any) any {
return val.Interface()
}
// IsPathPromoted checks if a specific path is promoted (Column Evolution table: field_name for logs body).
func (t *telemetryMetaStore) IsPathPromoted(ctx context.Context, path string) (bool, error) {
ctx = withTelemetryContext(ctx, "IsPathPromoted")
// isPathPromoted checks if a specific path is promoted (Column Evolution table: field_name for the entry's column).
func (t *telemetryMetaStore) isPathPromoted(ctx context.Context, entry telemetrytypes.EvolutionEntry, path string) (bool, error) {
ctx = withTelemetryContext(ctx, entry.Signal, "isPathPromoted")
split := strings.Split(path, telemetrytypes.ArraySep)
pathSegment := split[0]
query := fmt.Sprintf("SELECT 1 FROM %s.%s WHERE signal = ? AND column_name = ? AND field_context = ? AND field_name = ? LIMIT 1", DBName, PromotedPathsTableName)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, telemetrytypes.SignalLogs, logstelemetryschema.LogsV2BodyPromotedColumn, telemetrytypes.FieldContextBody, pathSegment)
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, entry.Signal, entry.ColumnName, entry.FieldContext, pathSegment)
if err != nil {
return false, errors.WrapInternalf(err, CodeFailCheckPathPromoted, "failed to check if path %s is promoted", path)
}
@@ -391,14 +396,14 @@ func (t *telemetryMetaStore) IsPathPromoted(ctx context.Context, path string) (b
return rows.Next(), nil
}
// GetPromotedPaths returns promoted paths from the Column Evolution table (field_name for logs body).
func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error) {
ctx = withTelemetryContext(ctx, "GetPromotedPaths")
// GetPromotedPaths returns promoted paths from the Column Evolution table (field_name for the entry's column).
func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, entry telemetrytypes.EvolutionEntry, paths ...string) (map[string]bool, error) {
ctx = withTelemetryContext(ctx, entry.Signal, "GetPromotedPaths")
sb := sqlbuilder.Select("field_name").From(fmt.Sprintf("%s.%s", DBName, PromotedPathsTableName))
conditions := []string{
sb.Equal("signal", telemetrytypes.SignalLogs),
sb.Equal("column_name", logstelemetryschema.LogsV2BodyPromotedColumn),
sb.Equal("field_context", telemetrytypes.FieldContextBody),
sb.Equal("signal", entry.Signal),
sb.Equal("column_name", entry.ColumnName),
sb.Equal("field_context", entry.FieldContext),
sb.NotEqual("field_name", "__all__"),
}
if len(paths) > 0 {
@@ -438,9 +443,10 @@ func CleanPathPrefixes(path string) string {
return path
}
// PromotePaths inserts promoted paths into the Column Evolution table (same schema as signoz-otel-collector metadata_migrations).
func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string) error {
ctx = withTelemetryContext(ctx, "PromotePaths")
// PromotePaths inserts promoted paths into the Column Evolution table as rows templated by entry
// (same schema as signoz-otel-collector metadata_migrations); FieldName and ReleaseTime are set per path.
func (t *telemetryMetaStore) PromotePaths(ctx context.Context, entry telemetrytypes.EvolutionEntry, paths ...string) error {
ctx = withTelemetryContext(ctx, entry.Signal, "PromotePaths")
batch, err := t.telemetrystore.ClickhouseDB().PrepareBatch(ctx,
fmt.Sprintf("INSERT INTO %s.%s (signal, column_name, column_type, field_context, field_name, version, release_time) VALUES", DBName,
PromotedPathsTableName))
@@ -454,7 +460,7 @@ func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string)
if trimmed == "" {
continue
}
if err := batch.Append(telemetrytypes.SignalLogs, logstelemetryschema.LogsV2BodyPromotedColumn, "JSON()", telemetrytypes.FieldContextBody, trimmed, 0, releaseTime); err != nil {
if err := batch.Append(entry.Signal, entry.ColumnName, entry.ColumnType, entry.FieldContext, trimmed, entry.Version, releaseTime); err != nil {
_ = batch.Abort()
return errors.WrapInternalf(err, CodeFailedToAppendPath, "failed to append path")
}
@@ -466,9 +472,9 @@ func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string)
return nil
}
func withTelemetryContext(ctx context.Context, functionName string) context.Context {
func withTelemetryContext(ctx context.Context, signal telemetrytypes.Signal, functionName string) context.Context {
return ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalLogs.StringValue(),
instrumentationtypes.TelemetrySignal: signal.StringValue(),
instrumentationtypes.CodeNamespace: "metadata",
instrumentationtypes.CodeFunctionName: functionName,
})

View File

@@ -461,7 +461,7 @@ func TestConditionFor(t *testing.T) {
evolutions: mockEvolution,
operator: qbtypes.FilterOperatorRegexp,
value: "frontend-.*",
expectedSQL: "WHERE (match(`resource_string_service$$name`, ?) AND `resource_string_service$$name_exists`)",
expectedSQL: "WHERE (match(`resource_string_service$$name`, ?) AND `resource_string_service$$name_exists` = true)",
expectedArgs: []any{"frontend-.*"},
expectedError: nil,
},

View File

@@ -1596,7 +1596,7 @@ func TestFilterExprLogs(t *testing.T) {
category: "Materialized key",
query: "materialized.key.name=\"test\"",
shouldPass: true,
expectedQuery: "WHERE (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists`)",
expectedQuery: "WHERE (`attribute_string_materialized$$key$$name` = ? AND `attribute_string_materialized$$key$$name_exists` = true)",
expectedArgs: []any{"test"},
expectedErrorContains: "",
},

View File

@@ -182,7 +182,7 @@ func (m *storage) read(_ context.Context, q qbtypes.QueryInfo, key *telemetrytyp
// a key could have been materialized, if so return the materialized column name
if key.Materialized {
exprs = append(exprs, telemetrytypes.FieldKeyToMaterializedColumnName(key))
existExpr = append(existExpr, telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key))
existExpr = append(existExpr, telemetrytypes.FieldKeyToMaterializedExistsCondition(key, true))
} else {
exprs = append(exprs, fmt.Sprintf("%s[%s]", columnName, clickhousesql.StringLiteral(key.Name)))
existExpr = append(existExpr, fmt.Sprintf("mapContains(%s, %s)", columnName, clickhousesql.StringLiteral(key.Name)))

View File

@@ -580,7 +580,7 @@ func TestFieldForWithMaterialized(t *testing.T) {
name: "Multi evolution - both columns (JSON + materialized)",
start: time.Date(2024, 2, 1, 0, 0, 0, 0, time.UTC),
end: time.Date(2024, 4, 2, 0, 0, 0, 0, time.UTC),
expectedResult: "multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, `resource_string_service$$name_exists`, `resource_string_service$$name`, NULL)",
expectedResult: "multiIf(resource.`service.name` IS NOT NULL, resource.`service.name`::String, `resource_string_service$$name_exists` = true, `resource_string_service$$name`, NULL)",
},
}

View File

@@ -172,7 +172,7 @@ func NewStorage() qbtypes.Storage {
func (m *storage) getColumn(
_ context.Context,
_, _ uint64,
q qbtypes.QueryInfo,
key *telemetrytypes.TelemetryFieldKey,
) ([]*schema.Column, error) {
switch key.FieldContext {
@@ -194,8 +194,8 @@ func (m *storage) getColumn(
default:
return nil, qbtypes.ErrColumnNotFound
}
// The `attributes` evolution entry is the rollout control.
if attributeColumnEvolutionRegistered(key, SpanAttributesColumn) {
// The use_trace_attributes_json flag and the `attributes` evolution entry are the rollout control.
if q.TraceAttrsJSONOn && attributeColumnEvolutionRegistered(key, SpanAttributesColumn) {
cols := make([]*schema.Column, 0, 3)
if attributeColumnEvolutionRegistered(key, SpanAttributesPromotedColumn) {
cols = append(cols, indexV3Columns["attributes_promoted"])
@@ -233,15 +233,15 @@ func (m *storage) getColumn(
// (after evolution selection); existExprs only carries guards for guardable column types.
func (m *storage) resolveColumnExprs(
ctx context.Context,
startNs, endNs uint64,
q qbtypes.QueryInfo,
key *telemetrytypes.TelemetryFieldKey,
) (exprs []string, existExprs []string, columns []*schema.Column, err error) {
columns, err = m.getColumn(ctx, startNs, endNs, key)
columns, err = m.getColumn(ctx, q, key)
if err != nil {
return nil, nil, nil, err
}
newColumns, evolutionsEntries, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
newColumns, evolutionsEntries, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, q.StartNs, q.EndNs)
if err != nil {
return nil, nil, nil, err
}
@@ -306,7 +306,7 @@ func (m *storage) resolveColumnExprs(
// a key could have been materialized, if so return the materialized column name
if key.Materialized {
exprs = append(exprs, telemetrytypes.FieldKeyToMaterializedColumnName(key))
existExprs = append(existExprs, telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key))
existExprs = append(existExprs, telemetrytypes.FieldKeyToMaterializedExistsCondition(key, true))
} else {
exprs = append(exprs, fmt.Sprintf("%s[%s]", columnName, clickhousesql.StringLiteral(key.Name)))
existExprs = append(existExprs, fmt.Sprintf("mapContains(%s, %s)", columnName, clickhousesql.StringLiteral(key.Name)))
@@ -355,12 +355,12 @@ func attributeJSONValueExpr(path string, dataType telemetrytypes.FieldDataType)
// columnIsTemporal reports whether key resolves to a single time column, after evolution
// selection. Multiple columns mean an attribute-map union, which is never temporal.
func (m *storage) columnIsTemporal(ctx context.Context, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (bool, error) {
columns, err := m.getColumn(ctx, startNs, endNs, key)
func (m *storage) columnIsTemporal(ctx context.Context, q qbtypes.QueryInfo, key *telemetrytypes.TelemetryFieldKey) (bool, error) {
columns, err := m.getColumn(ctx, q, key)
if err != nil {
return false, err
}
newColumns, _, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, startNs, endNs)
newColumns, _, err := qbtypes.SelectEvolutionsForColumns(columns, key.Evolutions, q.StartNs, q.EndNs)
if err != nil {
return false, err
}
@@ -394,7 +394,7 @@ func (m *storage) read(ctx context.Context, q qbtypes.QueryInfo, key *telemetryt
return key.Name, nil
}
exprs, existExpr, columns, err := m.resolveColumnExprs(ctx, q.StartNs, q.EndNs, key)
exprs, existExpr, columns, err := m.resolveColumnExprs(ctx, q, key)
if err != nil {
return "", err
}
@@ -457,7 +457,7 @@ func (m *storage) Read(ctx context.Context, q qbtypes.QueryInfo, key *telemetryt
if isSpanSearchScopeField(key.Name) {
return qbtypes.Read{SQL: key.Name, Presence: "true", Absence: "false", WhenAbsent: qbtypes.AlwaysPresent}, nil
}
exprs, existExprs, columns, err := m.resolveColumnExprs(ctx, q.StartNs, q.EndNs, key)
exprs, existExprs, columns, err := m.resolveColumnExprs(ctx, q, key)
if err != nil {
return qbtypes.Read{}, err
}
@@ -469,7 +469,7 @@ func (m *storage) Read(ctx context.Context, q qbtypes.QueryInfo, key *telemetryt
if err != nil {
return qbtypes.Read{}, err
}
temporal, err := m.columnIsTemporal(ctx, q.StartNs, q.EndNs, key)
temporal, err := m.columnIsTemporal(ctx, q, key)
if err != nil {
return qbtypes.Read{}, err
}
@@ -532,18 +532,18 @@ func foldAbsentJSONReadToTypeDefault(key *telemetrytypes.TelemetryFieldKey, oper
// and corrects to the attribute maps when it names no column. A strict
// context synthesizes its type variants under the stripped and the literal
// spelling.
func (m *storage) Fallback(ctx context.Context, _ qbtypes.QueryInfo, key *telemetrytypes.TelemetryFieldKey, _ qbtypes.FilterOperator, value any) ([]*telemetrytypes.LogicalField, error) {
func (m *storage) Fallback(ctx context.Context, q qbtypes.QueryInfo, key *telemetrytypes.TelemetryFieldKey, _ qbtypes.FilterOperator, value any) ([]*telemetrytypes.LogicalField, error) {
var keys []*telemetrytypes.TelemetryFieldKey
switch key.FieldContext {
case telemetrytypes.FieldContextUnspecified:
probe := telemetrytypes.NewTelemetryFieldKey(key.Name, telemetrytypes.FieldContextSpan, key.FieldDataType)
if columns, err := m.getColumn(ctx, 0, 0, probe); err == nil {
if columns, err := m.getColumn(ctx, q, probe); err == nil {
keys = []*telemetrytypes.TelemetryFieldKey{stampColumnType(probe, columns)}
} else {
keys = querybuilder.SynthesizeKeys(key, value)
}
case telemetrytypes.FieldContextSpan, telemetrytypes.FieldContextTrace:
if columns, err := m.getColumn(ctx, 0, 0, key); err == nil {
if columns, err := m.getColumn(ctx, q, key); err == nil {
column := telemetrytypes.NewTelemetryFieldKey(key.Name, key.FieldContext, key.FieldDataType)
keys = []*telemetrytypes.TelemetryFieldKey{stampColumnType(column, columns)}
} else {

View File

@@ -22,7 +22,7 @@ var (
)
func readSQL(ctx context.Context, storage qbtypes.Storage, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (string, error) {
read, err := storage.Read(ctx, qbtypes.QueryInfo{StartNs: startNs, EndNs: endNs}, key)
read, err := storage.Read(ctx, qbtypes.QueryInfo{StartNs: startNs, EndNs: endNs, TraceAttrsJSONOn: true}, key)
return read.SQL, err
}
@@ -96,6 +96,40 @@ func TestFieldForAttributeNoEvolutionParity(t *testing.T) {
}
}
// TestAttributeJSONFlagOffParity proves the evolution entry alone does not switch reads to the
// JSON column: with use_trace_attributes_json off, reads and conditions stay on the Map for every window.
func TestAttributeJSONFlagOffParity(t *testing.T) {
ctx := context.Background()
storage := NewStorage()
evo := MockAttributeEvolutionData(attrJSONRelease)
testCases := []struct {
name string
window [2]uint64
}{
{"BeforeRelease", attrWindowBefore},
{"AfterRelease", attrWindowAfter},
{"StraddlingRelease", attrWindowStraddle},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeNumber, evo)
q := qbtypes.QueryInfo{StartNs: testCase.window[0], EndNs: testCase.window[1]}
read, err := storage.Read(ctx, q, &key)
require.NoError(t, err)
assert.Equal(t, "attributes_number['user.id']", read.SQL)
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, q, storage, &key, qbtypes.FilterOperatorNotEqual, float64(1), map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
require.Len(t, conds, 1)
assert.NotContains(t, conds[0], "attributes.`user.id`")
})
}
}
// TestConditionForAttributeJSON asserts the emitted WHERE fragment per operator against the JSON
// column (window fully after release). Positive operators carry the raw-path existence guard;
// numeric comparisons keep numeric semantics; existence never tests the ::String cast.
@@ -170,7 +204,7 @@ func TestConditionForAttributeJSON(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &tc.key, tc.operator, tc.value, map[string][]*telemetrytypes.TelemetryFieldKey{tc.key.Name: {&tc.key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &tc.key, tc.operator, tc.value, map[string][]*telemetrytypes.TelemetryFieldKey{tc.key.Name: {&tc.key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -189,7 +223,7 @@ func TestConditionForAttributeJSONNotExistsDualRead(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1]}, storage, &key, qbtypes.FilterOperatorNotExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorNotExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -209,14 +243,14 @@ func TestColumnExpressionForAttributeJSON(t *testing.T) {
t.Run("group by string", func(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &key, telemetrytypes.FieldDataTypeString, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &key, telemetrytypes.FieldDataTypeString, nil)
require.NoError(t, err)
assert.Equal(t, "multiIf(attributes.`user.id` IS NOT NULL, attributes.`user.id`::String, mapContains(attributes_string, 'attribute.user.id'), attributes_string['attribute.user.id'], NULL)", got)
})
t.Run("aggregation numeric", func(t *testing.T) {
key := attrKey("latency", telemetrytypes.FieldDataTypeNumber, evo)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &key, telemetrytypes.FieldDataTypeFloat64, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &key, telemetrytypes.FieldDataTypeFloat64, nil)
require.NoError(t, err)
assert.Equal(t, "multiIf(if(dynamicType(attributes.`latency`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`latency`, 'Float64'), NULL) IS NOT NULL, toFloat64(if(dynamicType(attributes.`latency`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`latency`, 'Float64'), NULL)), mapContains(attributes_number, 'attribute.latency'), toFloat64(attributes_number['attribute.latency']), NULL)", got)
})
@@ -232,7 +266,7 @@ func TestAttributeJSONNoAmbiguityWarning(t *testing.T) {
key := attrKey("user.id", telemetrytypes.FieldDataTypeString, evo)
sb := sqlbuilder.NewSelectBuilder()
_, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &key, qbtypes.FilterOperatorEqual, "x", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
_, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorEqual, "x", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
assert.Empty(t, warnings, "a plain attribute filter must not emit an ambiguity warning")
}
@@ -254,7 +288,7 @@ func TestConditionForAttributeJSONTypeCollision(t *testing.T) {
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
sb := sqlbuilder.NewSelectBuilder()
conds, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
conds, warnings, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
require.NoError(t, err)
require.Len(t, conds, 2, "a colliding name must build one condition per data type")
@@ -283,7 +317,7 @@ func TestColumnExpressionForAttributeJSONTypeCollision(t *testing.T) {
}
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &ref, telemetrytypes.FieldDataTypeString, fieldKeys)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &ref, telemetrytypes.FieldDataTypeString, fieldKeys)
require.NoError(t, err)
assert.Equal(t,
"multiIf(attributes.`http.status_code` IS NOT NULL, attributes.`http.status_code`::String, if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL) IS NOT NULL, toString(if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL)), NULL)",
@@ -307,7 +341,7 @@ func TestColumnExpressionForAttributeJSONTypeCollisionNumericAgg(t *testing.T) {
}
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1]}, storage, &ref, telemetrytypes.FieldDataTypeFloat64, fieldKeys)
got, err := querybuilder.ResolveColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, storage, &ref, telemetrytypes.FieldDataTypeFloat64, fieldKeys)
require.NoError(t, err)
assert.Equal(t,
"multiIf(if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL) IS NOT NULL, toFloat64(if(dynamicType(attributes.`http.status_code`) IN ('Int64', 'UInt64', 'Float64'), accurateCastOrNull(attributes.`http.status_code`, 'Float64'), NULL)), attributes.`http.status_code` IS NOT NULL, toFloat64OrNull(attributes.`http.status_code`::String), NULL)",
@@ -330,7 +364,7 @@ func TestConditionForAttributeMapTypeCollisionParity(t *testing.T) {
ref := attrKey("http.status_code", telemetrytypes.FieldDataTypeUnspecified, nil)
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowBefore[0], EndNs: attrWindowBefore[1]}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowBefore[0], EndNs: attrWindowBefore[1], TraceAttrsJSONOn: true}, storage, &ref, qbtypes.FilterOperatorEqual, float64(200), fieldKeys, false, sb)
require.NoError(t, err)
require.Len(t, conds, 2, "a colliding name must build one condition per data type")
@@ -351,7 +385,7 @@ func TestColumnForUnspecifiedAttributeNoBranchFlip(t *testing.T) {
evo := MockAttributeEvolutionData(attrJSONRelease)
key := attrKey("user.id", telemetrytypes.FieldDataTypeUnspecified, evo)
_, err := (&storage{}).getColumn(ctx, attrWindowAfter[0], attrWindowAfter[1], &key)
_, err := (&storage{}).getColumn(ctx, qbtypes.QueryInfo{StartNs: attrWindowAfter[0], EndNs: attrWindowAfter[1], TraceAttrsJSONOn: true}, &key)
assert.ErrorIs(t, err, qbtypes.ErrColumnNotFound)
}
@@ -369,7 +403,7 @@ func TestConditionForAttributeJSONNegativeOperatorParity(t *testing.T) {
build := func(t *testing.T, key telemetrytypes.TelemetryFieldKey, window [2]uint64, op qbtypes.FilterOperator, value any) string {
t.Helper()
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: window[0], EndNs: window[1]}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: window[0], EndNs: window[1], TraceAttrsJSONOn: true}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -449,7 +483,7 @@ func TestConditionForAttributeJSONStraddleAbsentKeyExclusion(t *testing.T) {
build := func(t *testing.T, key telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) string {
t.Helper()
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1]}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: attrWindowStraddle[0], EndNs: attrWindowStraddle[1], TraceAttrsJSONOn: true}, storage, &key, op, value, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)

View File

@@ -81,7 +81,7 @@ func TestConditionForAttributePromoted(t *testing.T) {
t.Run("equal reads promoted column only", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1]}, storage, &key, qbtypes.FilterOperatorEqual, "GET", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorEqual, "GET", map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
@@ -91,7 +91,7 @@ func TestConditionForAttributePromoted(t *testing.T) {
t.Run("exists uses promoted raw path", func(t *testing.T) {
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1]}, storage, &key, qbtypes.FilterOperatorExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
conds, _, err := querybuilder.Conditions(ctx, qbtypes.QueryInfo{StartNs: afterPromo[0], EndNs: afterPromo[1], TraceAttrsJSONOn: true}, storage, &key, qbtypes.FilterOperatorExists, nil, map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, false, sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)

View File

@@ -80,7 +80,7 @@ func TestGetFieldKeyName(t *testing.T) {
Materialized: true,
Evolutions: mockEvolution,
},
expectedResult: "multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, `resource_string_deployment$$environment_exists`, `resource_string_deployment$$environment`, NULL)",
expectedResult: "multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, `resource_string_deployment$$environment_exists` = true, `resource_string_deployment$$environment`, NULL)",
expectedError: nil,
},
{
@@ -228,7 +228,7 @@ func TestFieldForResourceWithEvolution(t *testing.T) {
},
tsStart: uint64(time.Date(2024, 6, 1, 0, 0, 0, 0, time.UTC).UnixNano()),
tsEnd: uint64(time.Date(2025, 6, 1, 0, 0, 0, 0, time.UTC).UnixNano()),
expectedResult: "multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, `resource_string_deployment$$environment_exists`, `resource_string_deployment$$environment`, NULL)",
expectedResult: "multiIf(resource.`deployment.environment` IS NOT NULL, resource.`deployment.environment`::String, `resource_string_deployment$$environment_exists` = true, `resource_string_deployment$$environment`, NULL)",
},
}

View File

@@ -0,0 +1,131 @@
package promotetypes
import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// Target identifies a promotion domain: the column evolution record written
// per promoted path, the table per-path indexes are created on, and the API
// path rules.
type Target struct {
Entry telemetrytypes.EvolutionEntry // evolution row template; FieldName and ReleaseTime are set per write
DBName string // index DDL database, used only when IndexesSupported
LocalTableName string // index DDL local table, used only when IndexesSupported
BaseColumn string // column holding every path; indexes for unpromoted paths are created on it
RequiredPathPrefix string // prefix API paths must carry, stripped before storing; empty for bare names
IndexesSupported bool // whether per-path skip indexes can be created for this domain
}
func (t Target) PromotedColumn() string { return t.Entry.ColumnName }
func (t Target) BaseColumnPrefix() string { return t.BaseColumn + "." }
func (t Target) PromotedColumnPrefix() string { return t.PromotedColumn() + "." }
// NewTarget creates the Target for a promotion domain.
func NewTarget(entry telemetrytypes.EvolutionEntry, dbName, localTableName, baseColumn, requiredPathPrefix string, indexesSupported bool) Target {
return Target{
Entry: entry,
DBName: dbName,
LocalTableName: localTableName,
BaseColumn: baseColumn,
RequiredPathPrefix: requiredPathPrefix,
IndexesSupported: indexesSupported,
}
}
// NewLogsBodyTarget returns the domain for the logs body JSON column
// (body_v2 -> body_promoted), with per-path skip index support.
func NewLogsBodyTarget() Target {
return NewTarget(
telemetrytypes.EvolutionEntry{
Signal: telemetrytypes.SignalLogs,
ColumnName: logstelemetryschema.LogsV2BodyPromotedColumn,
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextBody,
},
logstelemetryschema.DBName,
logstelemetryschema.LogsV2LocalTableName,
logstelemetryschema.LogsV2BodyV2Column,
telemetrytypes.BodyJSONStringSearchPrefix,
true,
)
}
// NewTracesAttributesTarget returns the domain for the spans attributes JSON
// column (attributes -> attributes_promoted); promotion only for now.
func NewTracesAttributesTarget() Target {
return NewTarget(
telemetrytypes.EvolutionEntry{
Signal: telemetrytypes.SignalTraces,
ColumnName: tracestelemetryschema.SpanAttributesPromotedColumn,
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextAttribute,
},
tracestelemetryschema.DBName,
tracestelemetryschema.SpanIndexV3LocalTableName,
tracestelemetryschema.SpanAttributesColumn,
"",
false,
)
}
// NewTargetFromPath validates the {telemetry_signal} and {context} path
// variables and returns their promotion domain.
func NewTargetFromPath(signal, context string) (Target, error) {
params := &PathParams{Signal: signal, Context: context}
if err := params.Validate(); err != nil {
return Target{}, err
}
parsedSignal, _ := telemetrytypes.SignalFromText(params.Signal)
parsedContext, _ := telemetrytypes.FieldContextFromText(params.Context)
target, _ := TargetFor(parsedSignal, parsedContext)
return target, nil
}
// Targets returns every supported promotion domain.
func Targets() []Target {
return []Target{
NewLogsBodyTarget(),
NewTracesAttributesTarget(),
}
}
// TargetFor resolves the domain for a (signal, context) pair; ok is false
// when no domain exists for the pair.
func TargetFor(signal telemetrytypes.Signal, context telemetrytypes.FieldContext) (Target, bool) {
for _, target := range Targets() {
if target.Entry.Signal.StringValue() == signal.StringValue() &&
target.Entry.FieldContext.StringValue() == context.StringValue() {
return target, true
}
}
return Target{}, false
}
// PathParams carries the raw {telemetry_signal} and {context} path variables
// of the promote paths API.
type PathParams struct {
Signal string
Context string
}
// Validate ensures the path variables are known values naming a supported
// promotion domain.
func (p *PathParams) Validate() error {
signal, ok := telemetrytypes.SignalFromText(p.Signal)
if !ok {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid signal: %s", p.Signal)
}
context, ok := telemetrytypes.FieldContextFromText(p.Context)
if !ok {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid context: %s", p.Context)
}
if _, ok := TargetFor(signal, context); !ok {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "promotion is not supported for %s %s", signal.StringValue(), context.StringValue())
}
return nil
}

View File

@@ -3,7 +3,6 @@ package promotetypes
import (
"strings"
"github.com/SigNoz/signoz-otel-collector/constants"
"github.com/SigNoz/signoz-otel-collector/pkg/keycheck"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -23,7 +22,7 @@ type PromotePath struct {
Indexes []WrappedIndex `json:"indexes,omitempty"`
}
func (i *PromotePath) ValidateAndSetDefaults() error {
func (i *PromotePath) ValidateAndSetDefaults(target Target) error {
if i.Path == "" {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "path is required")
}
@@ -36,22 +35,27 @@ func (i *PromotePath) ValidateAndSetDefaults() error {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "array paths can not be promoted or indexed")
}
if strings.HasPrefix(i.Path, constants.BodyV2ColumnPrefix) || strings.HasPrefix(i.Path, constants.BodyPromotedColumnPrefix) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "`%s`, `%s` don't add these prefixes to the path", constants.BodyV2ColumnPrefix, constants.BodyPromotedColumnPrefix)
if strings.HasPrefix(i.Path, target.BaseColumnPrefix()) || strings.HasPrefix(i.Path, target.PromotedColumnPrefix()) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "`%s`, `%s` don't add these prefixes to the path", target.BaseColumnPrefix(), target.PromotedColumnPrefix())
}
if !strings.HasPrefix(i.Path, telemetrytypes.BodyJSONStringSearchPrefix) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "path must start with `body.`")
if target.RequiredPathPrefix != "" {
if !strings.HasPrefix(i.Path, target.RequiredPathPrefix) {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "path must start with `%s`", target.RequiredPathPrefix)
}
// remove the required prefix from the path
i.Path = strings.TrimPrefix(i.Path, target.RequiredPathPrefix)
}
// remove the "body." prefix from the path
i.Path = strings.TrimPrefix(i.Path, telemetrytypes.BodyJSONStringSearchPrefix)
isCardinal := keycheck.IsCardinal(i.Path)
if isCardinal {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "cardinal paths can not be promoted or indexed")
}
if len(i.Indexes) > 0 && !target.IndexesSupported {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "indexes are not supported for %s %s", target.Entry.Signal.StringValue(), target.Entry.FieldContext.StringValue())
}
for idx, index := range i.Indexes {
if index.Type == "" {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "index type is required")

View File

@@ -0,0 +1,172 @@
package promotetypes
import (
"testing"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestValidateAndSetDefaultsLogsBody(t *testing.T) {
target := NewLogsBodyTarget()
testCases := []struct {
name string
path *PromotePath
wantErr bool
wantPath string
wantJSONDataType telemetrytypes.JSONDataType
}{
{
name: "ValidPath_BodyPrefixStripped",
path: &PromotePath{Path: "body.user.name", Promote: true},
wantPath: "user.name",
},
{
name: "PathWithoutBodyPrefix_Rejected",
path: &PromotePath{Path: "user.name", Promote: true},
wantErr: true,
},
{
name: "BodyV2PrefixedPath_Rejected",
path: &PromotePath{Path: "body_v2.user.name", Promote: true},
wantErr: true,
},
{
name: "BodyPromotedPrefixedPath_Rejected",
path: &PromotePath{Path: "body_promoted.user.name", Promote: true},
wantErr: true,
},
{
name: "EmptyPath_Rejected",
path: &PromotePath{Path: "", Promote: true},
wantErr: true,
},
{
name: "SpacedPath_Rejected",
path: &PromotePath{Path: "body.my path", Promote: true},
wantErr: true,
},
{
name: "ArrayIndexPath_Rejected",
path: &PromotePath{Path: "body.users[].id", Promote: true},
wantErr: true,
},
{
name: "ArrayWildcardPath_Rejected",
path: &PromotePath{Path: "body.users[*].id", Promote: true},
wantErr: true,
},
{
name: "CardinalPath_Rejected",
path: &PromotePath{Path: "body.request.550e8400-e29b-41d4-a716-446655440000", Promote: true},
wantErr: true,
},
{
name: "ValidIndex_JSONDataTypeDefaulted",
path: &PromotePath{
Path: "body.user.name",
Indexes: []WrappedIndex{
{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "ngrambf_v1(4, 1024, 2, 0)", Granularity: 1},
},
},
wantPath: "user.name",
wantJSONDataType: telemetrytypes.String,
},
{
name: "UnsupportedColumnTypeIndex_Rejected",
path: &PromotePath{
Path: "body.user.active",
Indexes: []WrappedIndex{{FieldDataType: telemetrytypes.FieldDataTypeBool, Type: "minmax", Granularity: 1}},
},
wantErr: true,
},
{
name: "IndexWithoutType_Rejected",
path: &PromotePath{
Path: "body.user.name",
Indexes: []WrappedIndex{{FieldDataType: telemetrytypes.FieldDataTypeString, Granularity: 1}},
},
wantErr: true,
},
{
name: "IndexWithoutGranularity_Rejected",
path: &PromotePath{
Path: "body.user.name",
Indexes: []WrappedIndex{{FieldDataType: telemetrytypes.FieldDataTypeString, Type: "minmax"}},
},
wantErr: true,
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := testCase.path.ValidateAndSetDefaults(target)
if testCase.wantErr {
assert.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, testCase.wantPath, testCase.path.Path)
if testCase.wantJSONDataType != (telemetrytypes.JSONDataType{}) {
require.Len(t, testCase.path.Indexes, 1)
assert.Equal(t, testCase.wantJSONDataType, testCase.path.Indexes[0].JSONDataType)
}
})
}
}
func TestValidateAndSetDefaultsTracesAttributes(t *testing.T) {
target := NewTracesAttributesTarget()
testCases := []struct {
name string
path *PromotePath
wantErr bool
wantPath string
}{
{
name: "BareAttributeName_KeptAsIs",
path: &PromotePath{Path: "http.method", Promote: true},
wantPath: "http.method",
},
{
name: "AttributesPrefixedPath_Rejected",
path: &PromotePath{Path: "attributes.http.method", Promote: true},
wantErr: true,
},
{
name: "AttributesPromotedPrefixedPath_Rejected",
path: &PromotePath{Path: "attributes_promoted.http.method", Promote: true},
wantErr: true,
},
{
name: "EmptyPath_Rejected",
path: &PromotePath{Path: "", Promote: true},
wantErr: true,
},
{
name: "SpacedPath_Rejected",
path: &PromotePath{Path: "my attr", Promote: true},
wantErr: true,
},
{
name: "ArrayIndexPath_Rejected",
path: &PromotePath{Path: "tags[].id", Promote: true},
wantErr: true,
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := testCase.path.ValidateAndSetDefaults(target)
if testCase.wantErr {
assert.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, testCase.wantPath, testCase.path.Path)
})
}
}

View File

@@ -13,6 +13,7 @@ import (
// SelectEvolutionsForColumns selects the appropriate evolution entries for each column based on the time range.
// Logic:
// - Ignores evolutions of columns outside the candidate columns
// - Finds the latest base evolution (<= tsStartTime) across ALL columns
// - Rejects all evolutions before this latest base evolution
// - For duplicate evolutions it considers the oldest one (first in ReleaseTime)
@@ -23,6 +24,11 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
return columns, nil, nil
}
columnLookUpMap := make(map[string]*schema.Column, len(columns))
for _, column := range columns {
columnLookUpMap[column.Name] = column
}
// Derive the base column from the candidate columns.
seen := make(map[string]struct{}, len(evolutions))
for _, e := range evolutions {
@@ -64,6 +70,9 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
if evolution.ReleaseTime.After(tsStartTime) {
break
}
if _, exists := columnLookUpMap[evolution.ColumnName]; !exists {
continue
}
latestBaseEvolutionAcrossAll = evolution
}
@@ -72,11 +81,6 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
return nil, nil, errors.Newf(errors.TypeInternal, errors.CodeInternal, "no base evolution found for columns %v", columns)
}
columnLookUpMap := make(map[string]*schema.Column)
for _, column := range columns {
columnLookUpMap[column.Name] = column
}
// Collect column-evolution pairs
type colEvoPair struct {
column *schema.Column
@@ -95,7 +99,7 @@ func SelectEvolutionsForColumns(columns []*schema.Column, evolutions []*telemetr
}
if _, exists := columnLookUpMap[evolution.ColumnName]; !exists {
return nil, nil, errors.Newf(errors.TypeInternal, errors.CodeInternal, "evolution column %s not found in columns %v", evolution.ColumnName, columns)
continue
}
pairs = append(pairs, colEvoPair{columnLookUpMap[evolution.ColumnName], evolution})

View File

@@ -208,7 +208,7 @@ func TestSelectEvolutionsForColumns(t *testing.T) {
expectedColumns: []string{},
expectedEvols: []string{},
expectedError: true,
errorStr: "column resources_string not found",
errorStr: "no base evolution found",
},
{
name: "Duplicate evolutions - should use first encountered (oldest if sorted)",
@@ -441,6 +441,26 @@ func TestSelectEvolutionsForColumns(t *testing.T) {
expectedColumns: []string{"attributes"},
expectedEvols: []string{"attributes"},
},
{
name: "Non-candidate evolution ignored - JSON released before window keeps the map",
columns: []*schema.Column{
attributes_string,
},
evolutions: []*telemetrytypes.EvolutionEntry{
{
Signal: telemetrytypes.SignalTraces,
ColumnName: "attributes",
ColumnType: "JSON()",
FieldContext: telemetrytypes.FieldContextAttribute,
FieldName: "__all__",
ReleaseTime: time.Date(2024, 2, 10, 0, 0, 0, 0, time.UTC),
},
},
tsStart: uint64(time.Date(2024, 2, 15, 0, 0, 0, 0, time.UTC).UnixNano()),
tsEnd: uint64(time.Date(2024, 2, 20, 0, 0, 0, 0, time.UTC).UnixNano()),
expectedColumns: []string{"attributes_string"},
expectedEvols: []string{"attributes_string"},
},
}
for _, tc := range testCases {

View File

@@ -39,14 +39,16 @@ var (
// QueryInfo is the query's context as one value. It holds the time range
// every read needs, the signal and queried metric that family admission
// needs, and the query-path flags evaluated one time per request. The
// generic flows read FamiliesOn. Only the logs storage reads BodyJSONOn.
// generic flows read FamiliesOn. Only the logs storage reads BodyJSONOn, and
// only the traces storage reads TraceAttrsJSONOn.
type QueryInfo struct {
StartNs uint64
EndNs uint64
Signal telemetrytypes.Signal
Metric *telemetrytypes.MetricContext
FamiliesOn bool
BodyJSONOn bool
StartNs uint64
EndNs uint64
Signal telemetrytypes.Signal
Metric *telemetrytypes.MetricContext
FamiliesOn bool
BodyJSONOn bool
TraceAttrsJSONOn bool
}
// Absent is how a field key reads for a row that does not carry it, with

View File

@@ -38,3 +38,18 @@ var alertStateSeverity = map[AlertState]int{
func (a AlertState) Severity() int {
return alertStateSeverity[a]
}
// Display priority for list sorting, worst first from a user's view; deliberately
// NOT Severity(), which ranks disabled/nodata above firing for overall-state computation.
var alertStateDisplayRank = map[AlertState]int{
StateFiring: 5,
StateNoData: 4,
StatePending: 3,
StateRecovering: 2,
StateInactive: 1,
StateDisabled: 0,
}
func (a AlertState) DisplayRank() int {
return alertStateDisplayRank[a]
}

View File

@@ -0,0 +1,21 @@
package ruletypes
import (
"testing"
"github.com/stretchr/testify/assert"
)
// Both rankings must stay exhaustive: a new AlertState needs an entry in each.
func TestAlertStateRankingsAreExhaustive(t *testing.T) {
states := AlertState{}.Enum()
assert.Len(t, alertStateSeverity, len(states))
assert.Len(t, alertStateDisplayRank, len(states))
for _, s := range states {
state := s.(AlertState)
assert.Contains(t, alertStateSeverity, state, "missing severity for state %q", state)
assert.Contains(t, alertStateDisplayRank, state, "missing display rank for state %q", state)
}
}

129
pkg/types/ruletypes/list.go Normal file
View File

@@ -0,0 +1,129 @@
package ruletypes
import (
"slices"
"unicode/utf8"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/valuer"
)
const (
DefaultListLimit = 20
MaxListLimit = 200
MaxListQueryLen = 1024
)
var ErrCodeRuleListInvalid = errors.MustNewCode("rule_list_invalid")
type ListSort struct{ valuer.String }
var (
ListSortUpdatedAt = ListSort{valuer.NewString("updated_at")}
ListSortCreatedAt = ListSort{valuer.NewString("created_at")}
ListSortName = ListSort{valuer.NewString("name")}
ListSortState = ListSort{valuer.NewString("state")}
ListSortSeverity = ListSort{valuer.NewString("severity")}
)
func (ListSort) Enum() []any {
return []any{ListSortUpdatedAt, ListSortCreatedAt, ListSortName, ListSortState, ListSortSeverity}
}
func (s ListSort) IsValid() bool {
return slices.ContainsFunc(s.Enum(), func(v any) bool { return v == s })
}
type ListOrder struct{ valuer.String }
var (
ListOrderAsc = ListOrder{valuer.NewString("asc")}
ListOrderDesc = ListOrder{valuer.NewString("desc")}
)
func (ListOrder) Enum() []any {
return []any{ListOrderAsc, ListOrderDesc}
}
func (o ListOrder) IsValid() bool {
return slices.ContainsFunc(o.Enum(), func(v any) bool { return v == o })
}
type ListRulesParams struct {
Query string `query:"query"`
// gin cannot bind a slice of valuer enums; AlertStates converts these.
States []string `query:"states"`
Sort ListSort `query:"sort"`
Order ListOrder `query:"order"`
Limit int `query:"limit"`
Offset int `query:"offset"`
}
// Validate normalizes in place; an over-max limit is clamped, not rejected.
func (p *ListRulesParams) Validate() error {
if n := utf8.RuneCountInString(p.Query); n > MaxListQueryLen {
return errors.NewInvalidInputf(ErrCodeRuleListInvalid,
"query cannot be longer than %d characters, got %d", MaxListQueryLen, n)
}
if p.Sort.IsZero() {
p.Sort = ListSortUpdatedAt
} else if !p.Sort.IsValid() {
return errors.NewInvalidInputf(ErrCodeRuleListInvalid,
"invalid sort %q, expected one of: `updated_at`, `created_at`, `name`, `state`, `severity`", p.Sort)
}
if p.Order.IsZero() {
p.Order = ListOrderDesc
} else if !p.Order.IsValid() {
return errors.NewInvalidInputf(ErrCodeRuleListInvalid,
"invalid order %q, expected `asc` or `desc`", p.Order)
}
if p.Limit == 0 {
p.Limit = DefaultListLimit
} else if p.Limit < 0 {
return errors.NewInvalidInputf(ErrCodeRuleListInvalid,
"invalid limit %d, must be a positive integer", p.Limit)
} else if p.Limit > MaxListLimit {
p.Limit = MaxListLimit
}
if p.Offset < 0 {
return errors.NewInvalidInputf(ErrCodeRuleListInvalid,
"invalid offset %d, must be a non-negative integer", p.Offset)
}
if _, err := p.GetAlertStates(); err != nil {
return err
}
return nil
}
// GetAlertStates parses States; empty means no state filtering.
func (p *ListRulesParams) GetAlertStates() ([]AlertState, error) {
if len(p.States) == 0 {
return nil, nil
}
states := make([]AlertState, 0, len(p.States))
for _, raw := range p.States {
state, err := parseAlertState(raw)
if err != nil {
return nil, err
}
states = append(states, state)
}
return states, nil
}
func parseAlertState(raw string) (AlertState, error) {
state := AlertState{valuer.NewString(raw)}
if !slices.Contains(state.Enum(), any(state)) {
return AlertState{}, errors.NewInvalidInputf(ErrCodeRuleListInvalid,
"invalid state %q, expected one of: `firing`, `pending`, `recovering`, `inactive`, `nodata`, `disabled`", raw)
}
return state, nil
}

View File

@@ -0,0 +1,100 @@
package ruletypes
import (
"slices"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
qbtypesv5 "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
)
var ErrCodeRuleListFilterInvalid = errors.MustNewCode("rule_list_filter_invalid")
// DSLKey is a reserved (column-level) key in the rule list filter DSL.
type DSLKey string
const (
DSLKeyName DSLKey = "name"
DSLKeySeverity DSLKey = "severity"
DSLKeyCreatedBy DSLKey = "created_by"
DSLKeyUpdatedBy DSLKey = "updated_by"
DSLKeyCreatedAt DSLKey = "created_at"
DSLKeyUpdatedAt DSLKey = "updated_at"
DSLKeyAlertType DSLKey = "alert_type"
DSLKeyRuleType DSLKey = "rule_type"
// Label keys under this prefix are matched exactly (case-sensitive).
DSLLabelsKeyPrefix = "labels."
// Advertised in reservedKeywords; not itself a filterable key.
DSLKeyLabelsPlaceholder DSLKey = "labels.<key>"
)
func ReservedFilterKeys() []DSLKey {
keys := make([]DSLKey, 0, len(ReservedOps)+1)
for key := range ReservedOps {
keys = append(keys, key)
}
keys = append(keys, DSLKeyLabelsPlaceholder)
slices.SortFunc(keys, func(a, b DSLKey) int {
return strings.Compare(string(a), string(b))
})
return keys
}
// ReservedOps lists the operators each reserved DSL key accepts; `labels.<key>` terms use LabelsKeyOps.
var ReservedOps = map[DSLKey]map[qbtypesv5.FilterOperator]struct{}{
DSLKeyName: stringSearchOps(),
// severity aliases labels.severity, so it takes the labels operator set.
DSLKeySeverity: LabelsKeyOps,
DSLKeyCreatedBy: stringSearchOps(),
DSLKeyUpdatedBy: stringSearchOps(),
DSLKeyCreatedAt: numericRangeOps(),
DSLKeyUpdatedAt: numericRangeOps(),
DSLKeyAlertType: enumOps(),
DSLKeyRuleType: enumOps(),
}
// LabelsKeyOps operators target the label's value; EXISTS/NOT EXISTS test its presence.
var LabelsKeyOps = opsSet(
qbtypesv5.FilterOperatorEqual, qbtypesv5.FilterOperatorNotEqual,
qbtypesv5.FilterOperatorLike, qbtypesv5.FilterOperatorNotLike,
qbtypesv5.FilterOperatorILike, qbtypesv5.FilterOperatorNotILike,
qbtypesv5.FilterOperatorContains, qbtypesv5.FilterOperatorNotContains,
qbtypesv5.FilterOperatorIn, qbtypesv5.FilterOperatorNotIn,
qbtypesv5.FilterOperatorExists, qbtypesv5.FilterOperatorNotExists,
)
func stringSearchOps() map[qbtypesv5.FilterOperator]struct{} {
return opsSet(
qbtypesv5.FilterOperatorEqual, qbtypesv5.FilterOperatorNotEqual,
qbtypesv5.FilterOperatorLike, qbtypesv5.FilterOperatorNotLike,
qbtypesv5.FilterOperatorILike, qbtypesv5.FilterOperatorNotILike,
qbtypesv5.FilterOperatorContains, qbtypesv5.FilterOperatorNotContains,
qbtypesv5.FilterOperatorIn, qbtypesv5.FilterOperatorNotIn,
)
}
func numericRangeOps() map[qbtypesv5.FilterOperator]struct{} {
return opsSet(
qbtypesv5.FilterOperatorEqual, qbtypesv5.FilterOperatorNotEqual,
qbtypesv5.FilterOperatorLessThan, qbtypesv5.FilterOperatorLessThanOrEq,
qbtypesv5.FilterOperatorGreaterThan, qbtypesv5.FilterOperatorGreaterThanOrEq,
qbtypesv5.FilterOperatorBetween, qbtypesv5.FilterOperatorNotBetween,
)
}
func enumOps() map[qbtypesv5.FilterOperator]struct{} {
return opsSet(
qbtypesv5.FilterOperatorEqual, qbtypesv5.FilterOperatorNotEqual,
qbtypesv5.FilterOperatorIn, qbtypesv5.FilterOperatorNotIn,
)
}
func opsSet(ops ...qbtypesv5.FilterOperator) map[qbtypesv5.FilterOperator]struct{} {
m := make(map[qbtypesv5.FilterOperator]struct{}, len(ops))
for _, op := range ops {
m[op] = struct{}{}
}
return m
}

View File

@@ -0,0 +1,34 @@
package ruletypes
import (
"testing"
qbtypesv5 "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/stretchr/testify/assert"
)
func TestReservedFilterKeys(t *testing.T) {
assert.Equal(t, []DSLKey{
DSLKeyAlertType,
DSLKeyCreatedAt,
DSLKeyCreatedBy,
DSLKeyLabelsPlaceholder,
DSLKeyName,
DSLKeyRuleType,
DSLKeySeverity,
DSLKeyUpdatedAt,
DSLKeyUpdatedBy,
}, ReservedFilterKeys())
}
func TestFilterOpsExcludeRegexp(t *testing.T) {
for key, ops := range ReservedOps {
assert.NotEmpty(t, ops, "key %q has no operators", key)
assert.NotContains(t, ops, qbtypesv5.FilterOperatorRegexp, "key %q allows REGEXP", key)
assert.NotContains(t, ops, qbtypesv5.FilterOperatorNotRegexp, "key %q allows NOT REGEXP", key)
}
assert.NotContains(t, LabelsKeyOps, qbtypesv5.FilterOperatorRegexp)
assert.NotContains(t, LabelsKeyOps, qbtypesv5.FilterOperatorNotRegexp)
assert.Contains(t, LabelsKeyOps, qbtypesv5.FilterOperatorExists)
assert.Contains(t, LabelsKeyOps, qbtypesv5.FilterOperatorNotExists)
}

View File

@@ -0,0 +1,126 @@
package ruletypes
import (
"strings"
"testing"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestListRulesParamsValidate(t *testing.T) {
testCases := []struct {
name string
params ListRulesParams
wantErr string
wantSort ListSort
wantOrder ListOrder
wantLimit int
}{
{
name: "EmptyParams_Defaults",
params: ListRulesParams{},
wantSort: ListSortUpdatedAt,
wantOrder: ListOrderDesc,
wantLimit: DefaultListLimit,
},
{
name: "ExplicitValues_Kept",
params: ListRulesParams{Sort: ListSortSeverity, Order: ListOrderAsc, Limit: 50, Offset: 100},
wantSort: ListSortSeverity,
wantOrder: ListOrderAsc,
wantLimit: 50,
},
{
name: "OverMaxLimit_Clamped",
params: ListRulesParams{Limit: MaxListLimit + 1},
wantSort: ListSortUpdatedAt,
wantOrder: ListOrderDesc,
wantLimit: MaxListLimit,
},
{
name: "InvalidState_Rejected",
params: ListRulesParams{States: []string{"bogus"}},
wantErr: `invalid state "bogus"`,
},
{
name: "InvalidSort_Rejected",
params: ListRulesParams{Sort: ListSort{valuer.NewString("bogus")}},
wantErr: "invalid sort",
},
{
name: "InvalidOrder_Rejected",
params: ListRulesParams{Order: ListOrder{valuer.NewString("bogus")}},
wantErr: "invalid order",
},
{
name: "NegativeLimit_Rejected",
params: ListRulesParams{Limit: -1},
wantErr: "invalid limit",
},
{
name: "NegativeOffset_Rejected",
params: ListRulesParams{Offset: -1},
wantErr: "invalid offset",
},
{
name: "OverLongQuery_Rejected",
params: ListRulesParams{Query: strings.Repeat("a", MaxListQueryLen+1)},
wantErr: "query cannot be longer",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
err := tc.params.Validate()
if tc.wantErr != "" {
require.Error(t, err)
assert.Contains(t, err.Error(), tc.wantErr)
return
}
require.NoError(t, err)
assert.Equal(t, tc.wantSort, tc.params.Sort)
assert.Equal(t, tc.wantOrder, tc.params.Order)
assert.Equal(t, tc.wantLimit, tc.params.Limit)
})
}
}
func TestListRulesParamsAlertStates(t *testing.T) {
testCases := []struct {
name string
states []string
wantErr string
wantStates []AlertState
}{
{
name: "ValidStates_ParsedToTypedValues",
states: []string{"firing", "pending"},
wantStates: []AlertState{StateFiring, StatePending},
},
{
name: "AbsentStates_NoFiltering",
states: nil,
},
{
name: "InvalidState_Rejected",
states: []string{"bogus"},
wantErr: `invalid state "bogus"`,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
params := ListRulesParams{States: tc.states}
states, err := params.GetAlertStates()
if tc.wantErr != "" {
require.Error(t, err)
assert.Contains(t, err.Error(), tc.wantErr)
return
}
require.NoError(t, err)
assert.Equal(t, tc.wantStates, states)
})
}
}

View File

@@ -0,0 +1,190 @@
package ruletypes
import (
"cmp"
"encoding/json"
"slices"
"strings"
"github.com/SigNoz/signoz/pkg/types"
)
const MaxListLabelPairs = 1000
// ListableRule is the slim per-row shape of the list endpoint; the full rule stays behind get-by-id.
type ListableRule struct {
Id string `json:"id" required:"true"`
State AlertState `json:"state" required:"true"`
AlertName string `json:"alert" required:"true"`
Description string `json:"description,omitempty"`
AlertType AlertType `json:"alertType" required:"true"`
RuleType RuleType `json:"ruleType" required:"true"`
Disabled bool `json:"disabled"`
Labels map[string]string `json:"labels,omitempty"`
types.TimeAuditable
types.UserAuditable
}
// storedRuleData is the subset of the persisted rule data blob the list page needs.
type storedRuleData struct {
AlertName string `json:"alert"`
Description string `json:"description"`
AlertType AlertType `json:"alertType"`
RuleType RuleType `json:"ruleType"`
Disabled bool `json:"disabled"`
Labels map[string]string `json:"labels"`
}
// ToListableRule leaves State zero; the caller overlays evaluation state.
func (rule *StorableRule) ToListableRule() (*ListableRule, error) {
data := storedRuleData{}
if err := json.Unmarshal([]byte(rule.Data), &data); err != nil {
return nil, err
}
return &ListableRule{
Id: rule.ID.StringValue(),
AlertName: data.AlertName,
Description: data.Description,
AlertType: data.AlertType,
RuleType: data.RuleType,
Disabled: data.Disabled,
Labels: data.Labels,
TimeAuditable: rule.TimeAuditable,
UserAuditable: rule.UserAuditable,
}, nil
}
// NewListableRulesFromStorableRules converts rows, overlays evaluation state (absent means
// disabled) and applies the state filter; corrupt rows come back keyed by rule id for the
// caller to log.
func NewListableRulesFromStorableRules(storedRules []*StorableRule, stateByRuleID map[string]AlertState, stateFilter map[AlertState]struct{}) ([]*ListableRule, map[string]error) {
listableRules := make([]*ListableRule, 0, len(storedRules))
errByRuleID := make(map[string]error)
for _, rule := range storedRules {
listable, err := rule.ToListableRule()
if err != nil {
errByRuleID[rule.ID.StringValue()] = err
continue
}
if state, ok := stateByRuleID[listable.Id]; ok {
listable.State = state
} else {
listable.State = StateDisabled
listable.Disabled = true
}
if len(stateFilter) > 0 {
if _, ok := stateFilter[listable.State]; !ok {
continue
}
}
listableRules = append(listableRules, listable)
}
return listableRules, errByRuleID
}
// LabelPair is one distinct label key/value observed on the org's rules.
type LabelPair struct {
Key string `json:"key" required:"true"`
Value string `json:"value" required:"true"`
}
type ListableRules struct {
Rules []*ListableRule `json:"rules" required:"true" nullable:"false"`
Total int64 `json:"total" required:"true"`
Labels []LabelPair `json:"labels" required:"true" nullable:"false"`
ReservedKeywords []DSLKey `json:"reservedKeywords" required:"true" nullable:"false"`
}
func NewListableRules(rules []*ListableRule, total int64, labels []LabelPair) *ListableRules {
return &ListableRules{
Rules: rules,
Total: total,
Labels: labels,
ReservedKeywords: ReservedFilterKeys(),
}
}
var severityDisplayRank = map[string]int{
"critical": 4,
"error": 3,
"warning": 2,
"info": 1,
}
// Ties break on name then id ascending (order applies to the primary key only) so pages stay stable.
func SortListableRules(rules []*ListableRule, sortBy ListSort, order ListOrder) {
direction := 1
if order == ListOrderDesc {
direction = -1
}
slices.SortStableFunc(rules, func(a, b *ListableRule) int {
if c := direction * compareListableRules(a, b, sortBy); c != 0 {
return c
}
if c := strings.Compare(strings.ToLower(a.AlertName), strings.ToLower(b.AlertName)); c != 0 {
return c
}
return strings.Compare(a.Id, b.Id)
})
}
func compareListableRules(a, b *ListableRule, sortBy ListSort) int {
switch sortBy {
case ListSortName:
return strings.Compare(strings.ToLower(a.AlertName), strings.ToLower(b.AlertName))
case ListSortCreatedAt:
return a.CreatedAt.Compare(b.CreatedAt)
case ListSortState:
return cmp.Compare(a.State.DisplayRank(), b.State.DisplayRank())
case ListSortSeverity:
severityA := a.Labels["severity"]
severityB := b.Labels["severity"]
rankA := severityDisplayRank[strings.ToLower(severityA)]
rankB := severityDisplayRank[strings.ToLower(severityB)]
if rankA != rankB {
return cmp.Compare(rankA, rankB)
}
if rankA == 0 {
return strings.Compare(strings.ToLower(severityA), strings.ToLower(severityB))
}
return 0
}
return a.UpdatedAt.Compare(b.UpdatedAt)
}
// NewLabelPairsFromRawJSON skips blank or malformed entries and caps the result at limit.
func NewLabelPairsFromRawJSON(raws []string, limit int) []LabelPair {
set := make(map[LabelPair]struct{})
for _, raw := range raws {
if raw == "" || raw == "null" {
continue
}
labels := make(map[string]string)
if err := json.Unmarshal([]byte(raw), &labels); err != nil {
continue
}
for key, value := range labels {
set[LabelPair{Key: key, Value: value}] = struct{}{}
}
}
pairs := make([]LabelPair, 0, len(set))
for pair := range set {
pairs = append(pairs, pair)
}
slices.SortFunc(pairs, func(a, b LabelPair) int {
if c := strings.Compare(a.Key, b.Key); c != 0 {
return c
}
return strings.Compare(a.Value, b.Value)
})
if len(pairs) > limit {
pairs = pairs[:limit]
}
return pairs
}

View File

@@ -0,0 +1,261 @@
package ruletypes
import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func listableRule(name string, state AlertState, severity string, updatedAt time.Time) *ListableRule {
rule := &ListableRule{
AlertName: name,
State: state,
TimeAuditable: types.TimeAuditable{
UpdatedAt: updatedAt,
},
}
if severity != "" {
rule.Labels = map[string]string{"severity": severity}
}
return rule
}
func names(rules []*ListableRule) []string {
out := make([]string, 0, len(rules))
for _, rule := range rules {
out = append(out, rule.AlertName)
}
return out
}
func TestToListableRule(t *testing.T) {
created := time.Date(2026, 8, 1, 0, 0, 0, 0, time.UTC)
updated := time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)
storable := &StorableRule{
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
TimeAuditable: types.TimeAuditable{CreatedAt: created, UpdatedAt: updated},
UserAuditable: types.UserAuditable{CreatedBy: "creator@signoz.io", UpdatedBy: "updater@signoz.io"},
Data: `{"alert":"High CPU","description":"cpu is hot","alertType":"METRIC_BASED_ALERT","ruleType":"threshold_rule","disabled":true,"labels":{"severity":"critical"}}`,
}
listable, err := storable.ToListableRule()
require.NoError(t, err)
assert.Equal(t, storable.ID.StringValue(), listable.Id)
assert.Equal(t, "High CPU", listable.AlertName)
assert.Equal(t, "cpu is hot", listable.Description)
assert.Equal(t, AlertTypeMetric, listable.AlertType)
assert.Equal(t, RuleTypeThreshold, listable.RuleType)
assert.True(t, listable.Disabled)
assert.Equal(t, map[string]string{"severity": "critical"}, listable.Labels)
assert.Equal(t, created, listable.CreatedAt)
assert.Equal(t, "creator@signoz.io", listable.CreatedBy)
assert.Equal(t, updated, listable.UpdatedAt)
assert.Equal(t, "updater@signoz.io", listable.UpdatedBy)
assert.True(t, listable.State.IsZero())
_, err = (&StorableRule{Data: "not json"}).ToListableRule()
assert.Error(t, err)
}
func TestNewListableRulesFromStorableRules(t *testing.T) {
enabledID := valuer.GenerateUUID()
pausedID := valuer.GenerateUUID()
corruptID := valuer.GenerateUUID()
storedRules := []*StorableRule{
{
Identifiable: types.Identifiable{ID: enabledID},
Data: `{"alert":"cpu high","alertType":"METRIC_BASED_ALERT","ruleType":"threshold_rule"}`,
},
{
Identifiable: types.Identifiable{ID: pausedID},
Data: `{"alert":"mem high","alertType":"METRIC_BASED_ALERT","ruleType":"threshold_rule","disabled":true}`,
},
{
Identifiable: types.Identifiable{ID: corruptID},
Data: "not json",
},
}
stateByRuleID := map[string]AlertState{enabledID.StringValue(): StateFiring}
testCases := []struct {
name string
stateFilter map[AlertState]struct{}
wantNames []string
wantStates map[string]AlertState
}{
{
name: "NoFilter_KeepsAllParseableRows",
wantNames: []string{"cpu high", "mem high"},
wantStates: map[string]AlertState{"cpu high": StateFiring, "mem high": StateDisabled},
},
{
name: "FiringFilter_KeepsOverlaidState",
stateFilter: map[AlertState]struct{}{StateFiring: {}},
wantNames: []string{"cpu high"},
wantStates: map[string]AlertState{"cpu high": StateFiring},
},
{
name: "DisabledFilter_KeepsAbsentFromSnapshot",
stateFilter: map[AlertState]struct{}{StateDisabled: {}},
wantNames: []string{"mem high"},
wantStates: map[string]AlertState{"mem high": StateDisabled},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
listableRules, errByRuleID := NewListableRulesFromStorableRules(storedRules, stateByRuleID, testCase.stateFilter)
require.Len(t, errByRuleID, 1)
assert.Error(t, errByRuleID[corruptID.StringValue()])
assert.Equal(t, testCase.wantNames, names(listableRules))
for _, rule := range listableRules {
assert.Equal(t, testCase.wantStates[rule.AlertName], rule.State)
}
})
}
disabledRow, _ := NewListableRulesFromStorableRules(storedRules[1:2], stateByRuleID, nil)
require.Len(t, disabledRow, 1)
assert.True(t, disabledRow[0].Disabled)
}
func TestSortListableRules(t *testing.T) {
base := time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)
testCases := []struct {
name string
rules []*ListableRule
sortBy ListSort
order ListOrder
wantNames []string
}{
{
name: "StateDesc_DisplayPriority_FiringFirst",
rules: []*ListableRule{
listableRule("disabled", StateDisabled, "", base),
listableRule("nodata", StateNoData, "", base),
listableRule("firing", StateFiring, "", base),
listableRule("inactive", StateInactive, "", base),
listableRule("pending", StatePending, "", base),
listableRule("recovering", StateRecovering, "", base),
},
sortBy: ListSortState,
order: ListOrderDesc,
wantNames: []string{"firing", "nodata", "pending", "recovering", "inactive", "disabled"},
},
{
name: "SeverityDesc_KnownRanksThenCustomLexical",
rules: []*ListableRule{
listableRule("warn", StateInactive, "warning", base),
listableRule("custom-b", StateInactive, "bbb", base),
listableRule("crit", StateInactive, "critical", base),
listableRule("custom-a", StateInactive, "aaa", base),
listableRule("none", StateInactive, "", base),
},
sortBy: ListSortSeverity,
order: ListOrderDesc,
// desc flips the lexical compare between custom values too
wantNames: []string{"crit", "warn", "custom-b", "custom-a", "none"},
},
{
name: "NameAsc_CaseInsensitive",
rules: []*ListableRule{
listableRule("banana", StateInactive, "", base),
listableRule("Apple", StateInactive, "", base),
listableRule("cherry", StateInactive, "", base),
},
sortBy: ListSortName,
order: ListOrderAsc,
wantNames: []string{"Apple", "banana", "cherry"},
},
{
name: "UpdatedAtDesc_NewestFirst",
rules: []*ListableRule{
listableRule("old", StateInactive, "", base),
listableRule("new", StateInactive, "", base.Add(time.Hour)),
},
sortBy: ListSortUpdatedAt,
order: ListOrderDesc,
wantNames: []string{"new", "old"},
},
{
name: "StateDescTies_BreakOnNameAsc",
rules: []*ListableRule{
listableRule("banana", StateFiring, "", base),
listableRule("zebra", StateDisabled, "", base),
listableRule("Apple", StateFiring, "", base),
listableRule("cherry", StateFiring, "", base),
},
sortBy: ListSortState,
order: ListOrderDesc,
wantNames: []string{"Apple", "banana", "cherry", "zebra"},
},
{
name: "StateAsc_FlipsBuckets_TiebreakNameAsc",
rules: []*ListableRule{
listableRule("banana", StateFiring, "", base),
listableRule("zebra", StateDisabled, "", base),
listableRule("Apple", StateFiring, "", base),
listableRule("cherry", StateFiring, "", base),
},
sortBy: ListSortState,
order: ListOrderAsc,
wantNames: []string{"zebra", "Apple", "banana", "cherry"},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
SortListableRules(tc.rules, tc.sortBy, tc.order)
assert.Equal(t, tc.wantNames, names(tc.rules))
})
}
}
func TestSortListableRulesIdTiebreak(t *testing.T) {
base := time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)
for _, order := range []ListOrder{ListOrderAsc, ListOrderDesc} {
t.Run(order.StringValue(), func(t *testing.T) {
older := listableRule("dup", StateFiring, "", base)
older.Id = "01aaa"
newer := listableRule("dup", StateFiring, "", base)
newer.Id = "01bbb"
rules := []*ListableRule{newer, older}
SortListableRules(rules, ListSortState, order)
assert.Equal(t, []string{"01aaa", "01bbb"}, []string{rules[0].Id, rules[1].Id})
})
}
}
func TestNewLabelPairsFromRawJSON(t *testing.T) {
pairs := NewLabelPairsFromRawJSON([]string{
`{"team":"infra","severity":"critical"}`,
`{"team":"infra"}`,
`{"team":"payments"}`,
"",
"null",
"not-json",
}, MaxListLabelPairs)
assert.Equal(t, []LabelPair{
{Key: "severity", Value: "critical"},
{Key: "team", Value: "infra"},
{Key: "team", Value: "payments"},
}, pairs)
}
func TestNewLabelPairsFromRawJSONCap(t *testing.T) {
pairs := NewLabelPairsFromRawJSON([]string{`{"a":"1","b":"2","c":"3"}`}, 2)
assert.Len(t, pairs, 2)
}

View File

@@ -11,7 +11,8 @@ import (
)
type StorableRule struct {
bun.BaseModel `bun:"table:rule"`
// The alias must stay rule: the list filter compiler emits rule.<col> refs.
bun.BaseModel `bun:"table:rule,alias:rule"`
types.Identifiable
types.TimeAuditable
types.UserAuditable
@@ -58,6 +59,10 @@ type RuleStore interface {
EditRule(context.Context, *StorableRule, func(context.Context) error) error
DeleteRule(context.Context, valuer.UUID, valuer.UUID, func(context.Context) error) error
GetStoredRules(context.Context, string) ([]*StorableRule, error)
// GetStoredRulesMatching returns the org's rules matching a compiled filter clause; an empty clause matches all.
GetStoredRulesMatching(context.Context, string, string, []any) ([]*StorableRule, error)
// GetStoredRuleLabels returns each rule's labels as raw JSON text, empty string when absent.
GetStoredRuleLabels(context.Context, string) ([]string, error)
GetStoredRule(context.Context, valuer.UUID, valuer.UUID) (*StorableRule, error)
GetStoredRulesByMetricName(context.Context, string, string) ([]RuleAlert, error)
}

View File

@@ -218,6 +218,12 @@ func FieldKeyToMaterializedColumnNameForExists(key *TelemetryFieldKey) string {
))
}
// FieldKeyToMaterializedExistsCondition compares the exists column explicitly: a bare bool
// column defeats skip-index pruning across OR.
func FieldKeyToMaterializedExistsCondition(key *TelemetryFieldKey, exists bool) string {
return fmt.Sprintf("%s = %t", FieldKeyToMaterializedColumnNameForExists(key), exists)
}
type TelemetryFieldValues struct {
StringValues []string `json:"stringValues,omitempty"`
BoolValues []bool `json:"boolValues,omitempty"`

View File

@@ -22,3 +22,18 @@ func (Signal) Enum() []any {
SignalUnspecified,
}
}
// SignalFromText resolves a signal word to its Signal; ok is false for an
// unknown word.
func SignalFromText(text string) (Signal, bool) {
s := Signal{valuer.NewString(text)}
switch s {
case SignalTraces:
return SignalTraces, true
case SignalLogs:
return SignalLogs, true
case SignalMetrics:
return SignalMetrics, true
}
return Signal{}, false
}

View File

@@ -37,11 +37,13 @@ type MetadataStore interface {
// ListLogsJSONIndexes lists the JSON indexes for the logs table.
ListLogsJSONIndexes(ctx context.Context, filters ...string) ([]TelemetryFieldKeySkipIndex, error)
// ListPromotedPaths lists the promoted paths.
GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error)
// GetPromotedPaths lists the promoted paths recorded in the column
// evolution table for the entry's signal, column and field context.
GetPromotedPaths(ctx context.Context, entry EvolutionEntry, paths ...string) (map[string]bool, error)
// PromotePaths promotes the paths.
PromotePaths(ctx context.Context, paths ...string) error
// PromotePaths records promoted paths in the column evolution table as
// rows templated by entry; FieldName and ReleaseTime are set per path.
PromotePaths(ctx context.Context, entry EvolutionEntry, paths ...string) error
// GetFirstSeenFromMetricMetadata gets the first seen timestamp for a metric metadata lookup key.
GetFirstSeenFromMetricMetadata(ctx context.Context, lookupKeys []MetricMetadataLookupKey) (map[MetricMetadataLookupKey]int64, error)

View File

@@ -361,7 +361,7 @@ func (m *MockMetadataStore) SetTemporality(metricName string, temporality metric
}
// PromotePaths promotes the paths.
func (m *MockMetadataStore) PromotePaths(ctx context.Context, paths ...string) error {
func (m *MockMetadataStore) PromotePaths(_ context.Context, _ telemetrytypes.EvolutionEntry, paths ...string) error {
for _, path := range paths {
m.PromotedPathsMap[path] = true
}
@@ -369,7 +369,7 @@ func (m *MockMetadataStore) PromotePaths(ctx context.Context, paths ...string) e
}
// GetPromotedPaths returns the promoted paths.
func (m *MockMetadataStore) GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error) {
func (m *MockMetadataStore) GetPromotedPaths(_ context.Context, _ telemetrytypes.EvolutionEntry, _ ...string) (map[string]bool, error) {
return m.PromotedPathsMap, nil
}

View File

@@ -88,6 +88,40 @@ def create_alert_rule_with_channel(
return _create_alert_rule_with_channel
def delete_all_rules(signoz: types.SigNoz, token: str) -> None:
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/rules"),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
for rule in response.json()["data"]:
delete_response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v1/rules/{rule['id']}"),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert delete_response.status_code == HTTPStatus.OK, f"failed to delete rule {rule['id']}: {delete_response.text}"
@pytest.fixture(name="seed_alert_rules", scope="function")
def seed_alert_rules(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
create_notification_channel: Callable[[dict], str],
create_alert_rule: Callable[[dict], str],
) -> Callable[[dict, list[dict]], None]:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
def _seed_alert_rules(channel_config: dict, rules: list[dict]) -> None:
delete_all_rules(signoz, admin_token)
create_notification_channel(channel_config)
for rule in rules:
create_alert_rule(rule)
return _seed_alert_rules
def labels_to_map(labels: list[dict]) -> dict[str, str]:
"""Converts the label list shape of the v2 rule history APIs to a plain map."""
return {label["key"]["name"]: label["value"] for label in labels or []}

View File

@@ -0,0 +1,596 @@
from collections.abc import Callable
from http import HTTPStatus
import requests
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.types import Operation, SigNoz
BASE_URL = "/api/v3/rules"
SEED_CHANNEL = {"name": "list-rules-v3-channel", "email_configs": [{"to": "list-rules-v3@integration.test"}]}
EVALUATION = {"kind": "rolling", "spec": {"evalWindow": "5m0s", "frequency": "1m"}}
NOTIFICATION_SETTINGS = {
"groupBy": [],
"usePolicy": False,
"renotify": {"enabled": False, "interval": "30m", "alertStates": []},
}
METRIC_CONDITION = {
"thresholds": {
"kind": "basic",
"spec": [{"name": "critical", "target": 90, "matchType": "at_least_once", "op": "above", "channels": ["list-rules-v3-channel"]}],
},
"compositeQuery": {
"queryType": "builder",
"panelType": "graph",
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "metrics",
"aggregations": [{"metricName": "list_rules_v3_cpu", "timeAggregation": "avg", "spaceAggregation": "max"}],
},
}
],
},
"selectedQueryName": "A",
}
LOGS_CONDITION = {
"thresholds": {
"kind": "basic",
"spec": [{"name": "critical", "target": 100, "matchType": "at_least_once", "op": "above", "channels": ["list-rules-v3-channel"]}],
},
"compositeQuery": {
"queryType": "builder",
"panelType": "graph",
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "logs",
"aggregations": [{"expression": "count()"}],
"filter": {"expression": ""},
},
}
],
},
"selectedQueryName": "A",
}
PROMQL_CONDITION = {
"thresholds": {
"kind": "basic",
"spec": [{"name": "critical", "target": 1, "matchType": "at_least_once", "op": "below", "channels": ["list-rules-v3-channel"]}],
},
"compositeQuery": {
"queryType": "promql",
"panelType": "graph",
"queries": [{"type": "promql", "spec": {"name": "A", "query": '{"list_rules_v3_up"}'}}],
},
"selectedQueryName": "A",
}
SEED_RULES = [
{
"alert": "payment latency high",
"description": "p99 latency guard",
"alertType": "METRIC_BASED_ALERT",
"ruleType": "threshold_rule",
"condition": METRIC_CONDITION,
"labels": {"severity": "critical", "team": "payments", "k8s.cluster": "prod-1"},
"annotations": {"summary": "s", "description": "d"},
"evaluation": EVALUATION,
"notificationSettings": NOTIFICATION_SETTINGS,
"version": "v5",
"schemaVersion": "v2alpha1",
},
{
"alert": "payment gateway errors",
"description": "error rate watch",
"alertType": "LOGS_BASED_ALERT",
"ruleType": "threshold_rule",
"condition": LOGS_CONDITION,
"labels": {"severity": "warning", "team": "payments"},
"annotations": {"summary": "s", "description": "d"},
"evaluation": EVALUATION,
"notificationSettings": NOTIFICATION_SETTINGS,
"version": "v5",
"schemaVersion": "v2alpha1",
},
{
"alert": "checkout conversion drop",
"description": "funnel watcher",
"alertType": "METRIC_BASED_ALERT",
"ruleType": "threshold_rule",
"condition": METRIC_CONDITION,
"labels": {"severity": "important", "team": "checkout"},
"annotations": {"summary": "s", "description": "d"},
"disabled": True,
"evaluation": EVALUATION,
"notificationSettings": NOTIFICATION_SETTINGS,
"version": "v5",
"schemaVersion": "v2alpha1",
},
{
"alert": "infra cpu saturation",
"description": "node headroom",
"alertType": "METRIC_BASED_ALERT",
"ruleType": "threshold_rule",
"condition": METRIC_CONDITION,
"labels": {"team": "infra"},
"annotations": {"summary": "s", "description": "d"},
"evaluation": EVALUATION,
"notificationSettings": NOTIFICATION_SETTINGS,
"version": "v5",
"schemaVersion": "v2alpha1",
},
{
"alert": "prom uptime probe",
"description": "blackbox liveness",
"alertType": "METRIC_BASED_ALERT",
"ruleType": "promql_rule",
"condition": PROMQL_CONDITION,
"labels": {},
"annotations": {"summary": "s", "description": "d"},
"evaluation": EVALUATION,
"notificationSettings": NOTIFICATION_SETTINGS,
"version": "v5",
"schemaVersion": "v2alpha1",
},
]
# Labels deliberately collide with reserved DSL keys (name, state) for the collision tests.
COLLIDER_RULE = {
"alert": "ops shadow rule",
"description": "collision fixture",
"alertType": "METRIC_BASED_ALERT",
"ruleType": "threshold_rule",
"condition": METRIC_CONDITION,
"labels": {"name": "runbook", "state": "managed", "team": "ops"},
"annotations": {"summary": "s", "description": "d"},
"evaluation": EVALUATION,
"notificationSettings": NOTIFICATION_SETTINGS,
"version": "v5",
"schemaVersion": "v2alpha1",
}
RESERVED_KEYWORDS = [
"alert_type",
"created_at",
"created_by",
"labels.<key>",
"name",
"rule_type",
"severity",
"updated_at",
"updated_by",
]
def test_envelope_and_slim_rows(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES)
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
data = response.json()["data"]
assert data["total"] == 5
assert len(data["rules"]) == 5
assert data["reservedKeywords"] == RESERVED_KEYWORDS
label_pairs = [(pair["key"], pair["value"]) for pair in data["labels"]]
assert label_pairs == sorted(label_pairs), "label pairs must be sorted by key then value"
for expected_pair in [
("k8s.cluster", "prod-1"),
("severity", "critical"),
("severity", "important"),
("severity", "warning"),
("team", "checkout"),
("team", "infra"),
("team", "payments"),
]:
assert expected_pair in label_pairs, f"missing label pair {expected_pair}"
by_name = {rule["alert"]: rule for rule in data["rules"]}
assert set(by_name) == {r["alert"] for r in SEED_RULES}
for rule in data["rules"]:
for forbidden_field in ("condition", "annotations", "notificationSettings", "evaluation", "source", "version", "schemaVersion"):
assert forbidden_field not in rule, f"slim row leaked {forbidden_field}"
for required_field in ("id", "state", "alert", "alertType", "ruleType", "createdAt", "updatedAt"):
assert required_field in rule, f"slim row missing {required_field}"
assert rule["createdBy"] == USER_ADMIN_EMAIL
assert rule["updatedBy"] == USER_ADMIN_EMAIL
assert by_name["checkout conversion drop"]["state"] == "disabled"
assert by_name["checkout conversion drop"]["disabled"] is True
assert by_name["payment latency high"]["state"] == "inactive"
assert by_name["payment latency high"]["description"] == "p99 latency guard"
assert by_name["payment latency high"]["labels"] == {"severity": "critical", "team": "payments", "k8s.cluster": "prod-1"}
assert by_name["payment gateway errors"]["alertType"] == "LOGS_BASED_ALERT"
assert by_name["prom uptime probe"]["ruleType"] == "promql_rule"
def test_query_filters(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES)
cases = [
("name = 'payment latency high'", {"payment latency high"}),
("name CONTAINS 'payment'", {"payment latency high", "payment gateway errors"}),
# free text goes through LOWER() on both dialects, so a case mismatch must still match
("PAYMENT", {"payment latency high", "payment gateway errors"}),
# free text also matches the description field
("blackbox", {"prom uptime probe"}),
(f"created_by = '{USER_ADMIN_EMAIL}'", {r["alert"] for r in SEED_RULES}),
("created_at >= '2020-01-01T00:00:00Z'", {r["alert"] for r in SEED_RULES}),
("created_at < '2020-01-01T00:00:00Z'", set()),
("alert_type = 'LOGS_BASED_ALERT'", {"payment gateway errors"}),
("rule_type = 'promql_rule'", {"prom uptime probe"}),
("rule_type IN ['threshold_rule']", {"payment latency high", "payment gateway errors", "checkout conversion drop", "infra cpu saturation"}),
("labels.team = 'payments'", {"payment latency high", "payment gateway errors"}),
("labels.k8s.cluster = 'prod-1'", {"payment latency high"}),
("labels.team EXISTS", {"payment latency high", "payment gateway errors", "checkout conversion drop", "infra cpu saturation"}),
("labels.team NOT EXISTS", {"prom uptime probe"}),
("NOT (labels.team EXISTS)", {"prom uptime probe"}),
(
"(labels.team = 'payments' OR labels.team = 'infra') AND name NOT CONTAINS 'gateway'",
{"payment latency high", "infra cpu saturation"},
),
]
for query, expected_names in cases:
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"query": query},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK, f"query {query!r}: {response.text}"
data = response.json()["data"]
assert {rule["alert"] for rule in data["rules"]} == expected_names, f"query {query!r}"
assert data["total"] == len(expected_names), f"query {query!r}: total mismatch"
def test_bare_and_collision_keys(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES + [COLLIDER_RULE])
cases = [
# a bare non-reserved key is a label lookup, no labels. prefix needed
("team = 'infra'", {"infra cpu saturation"}),
("team = 'payments'", {"payment latency high", "payment gateway errors"}),
("team EXISTS", {"payment latency high", "payment gateway errors", "checkout conversion drop", "infra cpu saturation", "ops shadow rule"}),
# bare label keys stay case-sensitive
("Team = 'infra'", set()),
# state is not reserved, so it reads the rule's state label, not the evaluation state
("state = 'managed'", {"ops shadow rule"}),
# a reserved key matches the reserved field or a same-named label
("name = 'payment latency high'", {"payment latency high"}),
("name CONTAINS 'runbook'", {"ops shadow rule"}),
# a negative operator must exclude both interpretations
("name != 'runbook'", {r["alert"] for r in SEED_RULES}),
# the labels. prefix targets only the label on a collision
("labels.name = 'runbook'", {"ops shadow rule"}),
]
for query, expected_names in cases:
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"query": query},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK, f"query {query!r}: {response.text}"
data = response.json()["data"]
assert {rule["alert"] for rule in data["rules"]} == expected_names, f"query {query!r}"
assert data["total"] == len(expected_names), f"query {query!r}: total mismatch"
def test_label_missing_semantics(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES)
# A missing label uniformly evaluates as the empty string for value
# operators; presence is expressed with EXISTS / NOT EXISTS.
cases = [
("severity = ''", {"infra cpu saturation", "prom uptime probe"}),
("severity != ''", {"payment latency high", "payment gateway errors", "checkout conversion drop"}),
("severity != 'critical'", {"payment gateway errors", "checkout conversion drop", "infra cpu saturation", "prom uptime probe"}),
("severity EXISTS", {"payment latency high", "payment gateway errors", "checkout conversion drop"}),
("severity NOT EXISTS", {"infra cpu saturation", "prom uptime probe"}),
("severity = 'critical'", {"payment latency high"}),
("severity IN ['critical', 'warning']", {"payment latency high", "payment gateway errors"}),
("labels.team != 'payments'", {"checkout conversion drop", "infra cpu saturation", "prom uptime probe"}),
("labels.team NOT IN ['payments']", {"checkout conversion drop", "infra cpu saturation", "prom uptime probe"}),
("labels.team NOT CONTAINS 'pay'", {"checkout conversion drop", "infra cpu saturation", "prom uptime probe"}),
]
for query, expected_names in cases:
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"query": query},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK, f"query {query!r}: {response.text}"
data = response.json()["data"]
assert {rule["alert"] for rule in data["rules"]} == expected_names, f"query {query!r}"
assert data["total"] == len(expected_names), f"query {query!r}: total mismatch"
def test_states_param(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES)
# No telemetry is seeded, so enabled rules sit at inactive and the one
# disabled rule reads disabled, deterministic without waiting on evals.
cases = [
({"states": ["disabled"]}, {"checkout conversion drop"}),
({"states": ["inactive"]}, {"payment latency high", "payment gateway errors", "infra cpu saturation", "prom uptime probe"}),
({"states": ["inactive", "disabled"]}, {r["alert"] for r in SEED_RULES}),
({"states": ["firing"]}, set()),
({"states": ["disabled"], "query": "labels.team = 'checkout'"}, {"checkout conversion drop"}),
({"states": ["disabled"], "query": "labels.team = 'payments'"}, set()),
]
for params, expected_names in cases:
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params=params,
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK, f"params {params!r}: {response.text}"
data = response.json()["data"]
assert {rule["alert"] for rule in data["rules"]} == expected_names, f"params {params!r}"
assert data["total"] == len(expected_names), f"params {params!r}: total mismatch"
def test_sorting(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES)
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"sort": "name", "order": "asc"},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
assert [rule["alert"] for rule in response.json()["data"]["rules"]] == [
"checkout conversion drop",
"infra cpu saturation",
"payment gateway errors",
"payment latency high",
"prom uptime probe",
]
# state display priority: inactive (rank 1) outranks disabled (rank 0);
# the four inactive rules tie on state and must break on name asc
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"sort": "state", "order": "desc"},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
assert [rule["alert"] for rule in response.json()["data"]["rules"]] == [
"infra cpu saturation",
"payment gateway errors",
"payment latency high",
"prom uptime probe",
"checkout conversion drop",
]
# asc flips the state buckets but the name tiebreak stays ascending
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"sort": "state", "order": "asc"},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
assert [rule["alert"] for rule in response.json()["data"]["rules"]] == [
"checkout conversion drop",
"infra cpu saturation",
"payment gateway errors",
"payment latency high",
"prom uptime probe",
]
# severity: known ranks first (critical > warning), then custom values
# lexically, then rules without severity tie and break on name asc
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"sort": "severity", "order": "desc"},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
assert [rule["alert"] for rule in response.json()["data"]["rules"]] == [
"payment latency high",
"payment gateway errors",
"checkout conversion drop",
"infra cpu saturation",
"prom uptime probe",
]
for order in ("asc", "desc"):
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"sort": "created_at", "order": order},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
created_ats = [rule["createdAt"] for rule in response.json()["data"]["rules"]]
assert created_ats == sorted(created_ats, reverse=order == "desc"), f"created_at {order} not monotonic"
def test_pagination(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES)
pages = []
for offset in (0, 2, 4):
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"sort": "name", "order": "asc", "limit": 2, "offset": offset},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
data = response.json()["data"]
assert data["total"] == 5, f"offset {offset}: total must stay the full filtered count"
pages.append([rule["alert"] for rule in data["rules"]])
assert [len(page) for page in pages] == [2, 2, 1]
flattened = [name for page in pages for name in page]
assert len(flattened) == len(set(flattened)), "pages must be disjoint"
assert set(flattened) == {r["alert"] for r in SEED_RULES}
# state sort is almost all ties (four inactive rules); the name/id tiebreak
# must keep the pages disjoint and in the same order on every request
tie_pages = []
for offset in (0, 2, 4):
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"sort": "state", "order": "desc", "limit": 2, "offset": offset},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
tie_pages.append([rule["alert"] for rule in response.json()["data"]["rules"]])
assert [name for page in tie_pages for name in page] == [
"infra cpu saturation",
"payment gateway errors",
"payment latency high",
"prom uptime probe",
"checkout conversion drop",
], "tied rows must not shuffle between page requests"
# a past-the-end offset returns an empty page but keeps the real total
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"limit": 2, "offset": 50},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
data = response.json()["data"]
assert data["rules"] == []
assert data["total"] == 5
# an over-max limit is clamped, not rejected
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params={"limit": 6000},
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
assert response.json()["data"]["total"] == 5
def test_error_contract(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
cases = [
({"query": "created_by ==== ((("}, "rule_list_filter_invalid", "invalid filter query:"),
({"query": "team > 'infra'"}, "rule_list_filter_invalid", 'operator > is not allowed on the label filter "team"'),
({"query": "alert_type = 'bogus'"}, "rule_list_filter_invalid", "METRIC_BASED_ALERT"),
({"query": "name REGEXP 'x.*'"}, "rule_list_filter_invalid", "operator REGEXP is not allowed"),
({"query": "created_at >= 'yesterday'"}, "rule_list_filter_invalid", "invalid RFC3339 timestamp"),
({"query": "name LIKE 'prod\\\\'"}, "rule_list_filter_invalid", "must not end with an unescaped backslash"),
({"states": ["bogus"]}, "rule_list_invalid", 'invalid state "bogus"'),
({"sort": "bogus"}, "rule_list_invalid", "invalid sort"),
({"order": "bogus"}, "rule_list_invalid", "invalid order"),
({"limit": -1}, "rule_list_invalid", "invalid limit"),
({"offset": -1}, "rule_list_invalid", "invalid offset"),
]
for params, expected_code, expected_message_part in cases:
response = requests.get(
signoz.self.host_configs["8080"].get(BASE_URL),
params=params,
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.BAD_REQUEST, f"params {params!r}: {response.text}"
error = response.json()["error"]
assert error["code"] == expected_code, f"params {params!r}"
assert expected_message_part in error["message"], f"params {params!r}: {error['message']}"
def test_v2_list_still_serves_bare_array(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
seed_alert_rules: Callable[[dict, list[dict]], None],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
seed_alert_rules(SEED_CHANNEL, SEED_RULES)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/rules"),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
data = response.json()["data"]
assert isinstance(data, list), "deprecated v2 must keep returning a bare array"
assert {rule["alert"] for rule in data} == {r["alert"] for r in SEED_RULES}

View File

@@ -0,0 +1,30 @@
import pytest
from testcontainers.core.container import Network
from fixtures import types
from fixtures.signoz import create_signoz
@pytest.fixture(name="signoz", scope="package")
def signoz_trace_attributes_json(
network: Network,
zeus: types.TestContainerDocker,
gateway: types.TestContainerDocker,
sqlstore: types.TestContainerSQL,
clickhouse: types.TestContainerClickhouse,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.SigNoz:
return create_signoz(
network=network,
zeus=zeus,
gateway=gateway,
sqlstore=sqlstore,
clickhouse=clickhouse,
request=request,
pytestconfig=pytestconfig,
cache_key="signoz-trace-attributes-json",
env_overrides={
"SIGNOZ_FLAGGER_CONFIG_BOOLEAN_USE__TRACE__ATTRIBUTES__JSON": True,
},
)