Compare commits

..

2 Commits

Author SHA1 Message Date
Pandey
22e55340d7 fix(dashboards-v2): round-trip zero-valued threshold + query fields (#12158)
Some checks are pending
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
Release Drafter / update_release_draft (push) Waiting to run
* fix(dashboardtypes): accept threshold value of 0 on create

A NumberPanel/TimeSeries/Table threshold with `value: 0` (a legitimate
value the SigNoz UI emits by default) was rejected on create with
`dashboard_invalid_input` "Field validation for 'Value' failed on the
'required' tag".

go-playground/validator's `required` treats a numeric field equal to its
zero value as "missing", so `validate:"required"` on the float `Value`
wrongly rejected 0. Drop `validate:"required"` from `Value` on
ThresholdWithLabel and ComparisonThreshold; keep `required:"true"` since
the field is always present in the schema (0 is a valid present value, not
an absent one), so the OpenAPI/generated client are unaffected. `Color`
keeps both tags — an empty colour is genuinely invalid.

Drop the two "missing value" cases from TestValidateRequiredFields, which
asserted the removed invariant.

* fix(querybuildertypesv5): round-trip zero-valued query spec fields

A dashboard/alert query that sets a zero-valued field — `disabled: false`,
`legend: ""`, or an explicit empty `groupBy`/`order`/`selectFields`/etc. —
created fine but the GET response omitted it, so a typed client that echoes
what it sent (Terraform, SDKs, PUT-after-GET) read back `null`/absent and
reported drift. `,omitempty` dropped these zero values on the way out.

Fix the create -> GET asymmetry:

- Slice fields use `,omitzero` instead of `,omitempty`. `omitzero` omits a
  nil slice (field never set stays absent) but keeps an explicit non-nil
  `[]`, so an empty array round-trips as `[]` and there is no `null`
  regression. Applied to groupBy, order, selectFields, aggregations,
  functions, secondaryAggregations and function args across the builder,
  formula, trace-operator and join specs, plus ListPanelSpec.selectFields.
- Scalars `disabled` (bool) and `legend` (string) drop the tag entirely;
  `omitzero`/`omitempty` both suppress false/"", so the only way to
  round-trip them is to always serialize.

Result types in resp.go keep `,omitempty` — they are server-computed and
never round-tripped. Regenerate docs/api/openapi.yml and the frontend
client: the omitzero slices are now `nullable: true` in the schema (never
null on the wire, but the generated types gain `| null`, which existing
consumers already handle via `?? []`).

* test(dashboard): round-trip serialization for zero-valued fields

Add a v2 dashboards integration test that creates one minimal dashboard
(stripped from SigNoz/dashboards cicd-perses.json) and asserts the
create -> GET round-trip preserves every zero-valued field the fix targets:

- threshold value 0 (ComparisonThreshold + ThresholdWithLabel) is accepted
  on create and echoed back
- builder slices set to an explicit [] (groupBy/order/selectFields/functions)
  round-trip as [], while a bare builder's unset slices stay absent (never
  null) on read
- scalars disabled/legend always echo false/""

Table-driven: one equality table for round-tripped values and one absence
table for omitted slices.

* test(dashboard): fold round-trip test into 03_v2_dashboard

Move test_dashboard_v2_roundtrip_preserves_zero_values alongside the other
v2 dashboard tests (test_create_rejects_*, lifecycle, ...) instead of a
standalone file, with the dashboard payload inlined per this suite's style.
2026-07-18 19:52:54 +00:00
Tushar Vats
dd8cbe0844 fix: has function family revamp (#12089)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
2026-07-18 16:43:26 +00:00
25 changed files with 2963 additions and 511 deletions

View File

@@ -3039,6 +3039,7 @@ components:
selectFields:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
nullable: true
type: array
type: object
DashboardtypesListSort:
@@ -6552,6 +6553,7 @@ components:
args:
items:
$ref: '#/components/schemas/Querybuildertypesv5FunctionArg'
nullable: true
type: array
name:
$ref: '#/components/schemas/Querybuildertypesv5FunctionName'
@@ -6722,6 +6724,7 @@ components:
functions:
items:
$ref: '#/components/schemas/Querybuildertypesv5Function'
nullable: true
type: array
having:
$ref: '#/components/schemas/Querybuildertypesv5Having'
@@ -6734,6 +6737,7 @@ components:
order:
items:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
nullable: true
type: array
type: object
Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5LogAggregation:
@@ -6741,6 +6745,7 @@ components:
aggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5LogAggregation'
nullable: true
type: array
cursor:
type: string
@@ -6751,10 +6756,12 @@ components:
functions:
items:
$ref: '#/components/schemas/Querybuildertypesv5Function'
nullable: true
type: array
groupBy:
items:
$ref: '#/components/schemas/Querybuildertypesv5GroupByKey'
nullable: true
type: array
having:
$ref: '#/components/schemas/Querybuildertypesv5Having'
@@ -6771,14 +6778,17 @@ components:
order:
items:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
nullable: true
type: array
secondaryAggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5SecondaryAggregation'
nullable: true
type: array
selectFields:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
nullable: true
type: array
signal:
enum:
@@ -6796,6 +6806,7 @@ components:
aggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5MetricAggregation'
nullable: true
type: array
cursor:
type: string
@@ -6806,10 +6817,12 @@ components:
functions:
items:
$ref: '#/components/schemas/Querybuildertypesv5Function'
nullable: true
type: array
groupBy:
items:
$ref: '#/components/schemas/Querybuildertypesv5GroupByKey'
nullable: true
type: array
having:
$ref: '#/components/schemas/Querybuildertypesv5Having'
@@ -6826,14 +6839,17 @@ components:
order:
items:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
nullable: true
type: array
secondaryAggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5SecondaryAggregation'
nullable: true
type: array
selectFields:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
nullable: true
type: array
signal:
enum:
@@ -6851,6 +6867,7 @@ components:
aggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5TraceAggregation'
nullable: true
type: array
cursor:
type: string
@@ -6861,10 +6878,12 @@ components:
functions:
items:
$ref: '#/components/schemas/Querybuildertypesv5Function'
nullable: true
type: array
groupBy:
items:
$ref: '#/components/schemas/Querybuildertypesv5GroupByKey'
nullable: true
type: array
having:
$ref: '#/components/schemas/Querybuildertypesv5Having'
@@ -6881,14 +6900,17 @@ components:
order:
items:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
nullable: true
type: array
secondaryAggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5SecondaryAggregation'
nullable: true
type: array
selectFields:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
nullable: true
type: array
signal:
enum:
@@ -6906,6 +6928,7 @@ components:
aggregations:
items:
$ref: '#/components/schemas/Querybuildertypesv5TraceAggregation'
nullable: true
type: array
cursor:
type: string
@@ -6918,10 +6941,12 @@ components:
functions:
items:
$ref: '#/components/schemas/Querybuildertypesv5Function'
nullable: true
type: array
groupBy:
items:
$ref: '#/components/schemas/Querybuildertypesv5GroupByKey'
nullable: true
type: array
having:
$ref: '#/components/schemas/Querybuildertypesv5Having'
@@ -6936,12 +6961,14 @@ components:
order:
items:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
nullable: true
type: array
returnSpansFrom:
type: string
selectFields:
items:
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
nullable: true
type: array
stepInterval:
$ref: '#/components/schemas/Querybuildertypesv5Step'
@@ -7181,6 +7208,7 @@ components:
groupBy:
items:
$ref: '#/components/schemas/Querybuildertypesv5GroupByKey'
nullable: true
type: array
limit:
type: integer
@@ -7189,6 +7217,7 @@ components:
order:
items:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
nullable: true
type: array
stepInterval:
$ref: '#/components/schemas/Querybuildertypesv5Step'

View File

@@ -3489,9 +3489,9 @@ export enum Querybuildertypesv5FunctionNameDTO {
}
export interface Querybuildertypesv5FunctionDTO {
/**
* @type array
* @type array,null
*/
args?: Querybuildertypesv5FunctionArgDTO[];
args?: Querybuildertypesv5FunctionArgDTO[] | null;
name?: Querybuildertypesv5FunctionNameDTO;
}
@@ -3593,18 +3593,18 @@ export interface Querybuildertypesv5SecondaryAggregationDTO {
*/
expression?: string;
/**
* @type array
* @type array,null
*/
groupBy?: Querybuildertypesv5GroupByKeyDTO[];
groupBy?: Querybuildertypesv5GroupByKeyDTO[] | null;
/**
* @type integer
*/
limit?: number;
limitBy?: Querybuildertypesv5LimitByDTO;
/**
* @type array
* @type array,null
*/
order?: Querybuildertypesv5OrderByDTO[];
order?: Querybuildertypesv5OrderByDTO[] | null;
stepInterval?: Querybuildertypesv5StepDTO;
}
@@ -3634,9 +3634,9 @@ export enum TelemetrytypesSourceDTO {
}
export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5LogAggregationDTO {
/**
* @type array
* @type array,null
*/
aggregations?: Querybuildertypesv5LogAggregationDTO[];
aggregations?: Querybuildertypesv5LogAggregationDTO[] | null;
/**
* @type string
*/
@@ -3647,13 +3647,13 @@ export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTyp
disabled?: boolean;
filter?: Querybuildertypesv5FilterDTO;
/**
* @type array
* @type array,null
*/
functions?: Querybuildertypesv5FunctionDTO[];
functions?: Querybuildertypesv5FunctionDTO[] | null;
/**
* @type array
* @type array,null
*/
groupBy?: Querybuildertypesv5GroupByKeyDTO[];
groupBy?: Querybuildertypesv5GroupByKeyDTO[] | null;
having?: Querybuildertypesv5HavingDTO;
/**
* @type string
@@ -3673,17 +3673,17 @@ export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTyp
*/
offset?: number;
/**
* @type array
* @type array,null
*/
order?: Querybuildertypesv5OrderByDTO[];
order?: Querybuildertypesv5OrderByDTO[] | null;
/**
* @type array
* @type array,null
*/
secondaryAggregations?: Querybuildertypesv5SecondaryAggregationDTO[];
secondaryAggregations?: Querybuildertypesv5SecondaryAggregationDTO[] | null;
/**
* @type array
* @type array,null
*/
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[];
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[] | null;
/**
* @enum logs
* @type string
@@ -3759,9 +3759,9 @@ export enum Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQue
}
export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5MetricAggregationDTO {
/**
* @type array
* @type array,null
*/
aggregations?: Querybuildertypesv5MetricAggregationDTO[];
aggregations?: Querybuildertypesv5MetricAggregationDTO[] | null;
/**
* @type string
*/
@@ -3772,13 +3772,13 @@ export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTyp
disabled?: boolean;
filter?: Querybuildertypesv5FilterDTO;
/**
* @type array
* @type array,null
*/
functions?: Querybuildertypesv5FunctionDTO[];
functions?: Querybuildertypesv5FunctionDTO[] | null;
/**
* @type array
* @type array,null
*/
groupBy?: Querybuildertypesv5GroupByKeyDTO[];
groupBy?: Querybuildertypesv5GroupByKeyDTO[] | null;
having?: Querybuildertypesv5HavingDTO;
/**
* @type string
@@ -3798,17 +3798,17 @@ export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTyp
*/
offset?: number;
/**
* @type array
* @type array,null
*/
order?: Querybuildertypesv5OrderByDTO[];
order?: Querybuildertypesv5OrderByDTO[] | null;
/**
* @type array
* @type array,null
*/
secondaryAggregations?: Querybuildertypesv5SecondaryAggregationDTO[];
secondaryAggregations?: Querybuildertypesv5SecondaryAggregationDTO[] | null;
/**
* @type array
* @type array,null
*/
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[];
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[] | null;
/**
* @enum metrics
* @type string
@@ -3834,9 +3834,9 @@ export enum Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQue
}
export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTypesQuerybuildertypesQuerybuildertypesv5TraceAggregationDTO {
/**
* @type array
* @type array,null
*/
aggregations?: Querybuildertypesv5TraceAggregationDTO[];
aggregations?: Querybuildertypesv5TraceAggregationDTO[] | null;
/**
* @type string
*/
@@ -3847,13 +3847,13 @@ export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTyp
disabled?: boolean;
filter?: Querybuildertypesv5FilterDTO;
/**
* @type array
* @type array,null
*/
functions?: Querybuildertypesv5FunctionDTO[];
functions?: Querybuildertypesv5FunctionDTO[] | null;
/**
* @type array
* @type array,null
*/
groupBy?: Querybuildertypesv5GroupByKeyDTO[];
groupBy?: Querybuildertypesv5GroupByKeyDTO[] | null;
having?: Querybuildertypesv5HavingDTO;
/**
* @type string
@@ -3873,17 +3873,17 @@ export interface Querybuildertypesv5QueryBuilderQueryGithubComSigNozSignozPkgTyp
*/
offset?: number;
/**
* @type array
* @type array,null
*/
order?: Querybuildertypesv5OrderByDTO[];
order?: Querybuildertypesv5OrderByDTO[] | null;
/**
* @type array
* @type array,null
*/
secondaryAggregations?: Querybuildertypesv5SecondaryAggregationDTO[];
secondaryAggregations?: Querybuildertypesv5SecondaryAggregationDTO[] | null;
/**
* @type array
* @type array,null
*/
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[];
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[] | null;
/**
* @enum traces
* @type string
@@ -4272,9 +4272,9 @@ export enum DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboa
}
export interface DashboardtypesListPanelSpecDTO {
/**
* @type array
* @type array,null
*/
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[];
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[] | null;
}
export interface DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpecDTO {
@@ -4344,9 +4344,9 @@ export interface Querybuildertypesv5QueryBuilderFormulaDTO {
*/
expression?: string;
/**
* @type array
* @type array,null
*/
functions?: Querybuildertypesv5FunctionDTO[];
functions?: Querybuildertypesv5FunctionDTO[] | null;
having?: Querybuildertypesv5HavingDTO;
/**
* @type string
@@ -4361,9 +4361,9 @@ export interface Querybuildertypesv5QueryBuilderFormulaDTO {
*/
name?: string;
/**
* @type array
* @type array,null
*/
order?: Querybuildertypesv5OrderByDTO[];
order?: Querybuildertypesv5OrderByDTO[] | null;
}
export enum Querybuildertypesv5QueryEnvelopeFormulaDTOType {
@@ -4380,9 +4380,9 @@ export interface Querybuildertypesv5QueryEnvelopeFormulaDTO {
export interface Querybuildertypesv5QueryBuilderTraceOperatorDTO {
/**
* @type array
* @type array,null
*/
aggregations?: Querybuildertypesv5TraceAggregationDTO[];
aggregations?: Querybuildertypesv5TraceAggregationDTO[] | null;
/**
* @type string
*/
@@ -4397,13 +4397,13 @@ export interface Querybuildertypesv5QueryBuilderTraceOperatorDTO {
expression?: string;
filter?: Querybuildertypesv5FilterDTO;
/**
* @type array
* @type array,null
*/
functions?: Querybuildertypesv5FunctionDTO[];
functions?: Querybuildertypesv5FunctionDTO[] | null;
/**
* @type array
* @type array,null
*/
groupBy?: Querybuildertypesv5GroupByKeyDTO[];
groupBy?: Querybuildertypesv5GroupByKeyDTO[] | null;
having?: Querybuildertypesv5HavingDTO;
/**
* @type string
@@ -4422,17 +4422,17 @@ export interface Querybuildertypesv5QueryBuilderTraceOperatorDTO {
*/
offset?: number;
/**
* @type array
* @type array,null
*/
order?: Querybuildertypesv5OrderByDTO[];
order?: Querybuildertypesv5OrderByDTO[] | null;
/**
* @type string
*/
returnSpansFrom?: string;
/**
* @type array
* @type array,null
*/
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[];
selectFields?: TelemetrytypesTelemetryFieldKeyDTO[] | null;
stepInterval?: Querybuildertypesv5StepDTO;
}

View File

@@ -732,7 +732,11 @@ func (v *filterExpressionVisitor) VisitFunctionCall(ctx *grammar.FunctionCallCon
return ErrorConditionLiteral
}
value := params[1:]
value, err := normalizeFunctionValue(operator, functionName, params[1:])
if err != nil {
v.errors = append(v.errors, err.Error())
return ErrorConditionLiteral
}
conds, ok := v.buildConditions(key, MatchingFieldKeys(key, v.fieldKeys), operator, value)
if !ok {
@@ -748,6 +752,44 @@ func (v *filterExpressionVisitor) VisitFunctionCall(ctx *grammar.FunctionCallCon
return v.builder.Or(conds...)
}
// normalizeFunctionValue validates and normalizes the value argument(s) of a has-family
// function call, returning them in the wrapper slice the condition builder unwraps.
//
// - has/hasToken take exactly one scalar value. More than one argument, or an array
// argument, is rejected rather than silently dropping the extras.
// - hasAny/hasAll take a set of values, supplied either as a single array literal
// (hasAny(k, ['a','b'])) or as several scalar arguments (hasAny(k, 'a', 'b')); the
// latter are folded into one list so no argument is silently ignored.
func normalizeFunctionValue(operator qbtypes.FilterOperator, functionName string, valueParams []any) (any, error) {
switch operator {
case qbtypes.FilterOperatorHas, qbtypes.FilterOperatorHasToken:
if len(valueParams) != 1 {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "function `%s` expects exactly one value argument", functionName)
}
if _, isArray := valueParams[0].([]any); isArray {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "function `%s` expects a single scalar value, not an array", functionName)
}
return valueParams, nil
case qbtypes.FilterOperatorHasAny, qbtypes.FilterOperatorHasAll:
// A single array literal is already the value set.
if len(valueParams) == 1 {
if _, isArray := valueParams[0].([]any); isArray {
return valueParams, nil
}
}
// Otherwise fold the positional scalar arguments into one list.
values := make([]any, 0, len(valueParams))
for _, p := range valueParams {
if _, isArray := p.([]any); isArray {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "function `%s` expects either a single array literal or scalar values, not a mix of the two", functionName)
}
values = append(values, p)
}
return []any{values}, nil
}
return valueParams, nil
}
// VisitFunctionParamList handles the parameter list for function calls.
func (v *filterExpressionVisitor) VisitFunctionParamList(ctx *grammar.FunctionParamListContext) any {
params := ctx.AllFunctionParam()

View File

@@ -40,13 +40,11 @@ func isBodyJSONSearch(key *telemetrytypes.TelemetryFieldKey, columns []*schema.C
return false
}
// conditionForArrayFunction builds `has/hasAny/hasAll(<arrayFieldExpr>, value)` over a
// body JSON array field. The field expression uses the JSON accessor (flag on) or
// legacy string extraction (flag off); value[0] is the needle.
// conditionForArrayFunction builds has/hasAny/hasAll over a body JSON path — via the JSON
// access plan (flag on) or legacy typed extraction (flag off).
func (c *conditionBuilder) conditionForArrayFunction(
ctx context.Context,
orgID valuer.UUID,
startNs, endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
operator qbtypes.FilterOperator,
value any,
@@ -63,24 +61,48 @@ func (c *conditionBuilder) conditionForArrayFunction(
needle = args[0]
}
var fieldExpr string
if c.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID)) {
fe, err := c.fm.FieldFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", err
}
fieldExpr = fe
} else {
// legacy string-body path; value drives array-type inference (e.g. `[*]` paths)
fieldExpr, _ = GetBodyJSONKey(ctx, key, qbtypes.FilterOperatorUnknown, value)
// JSON access plan: data-type collision handling, nested array paths.
valueType, needle := InferDataType(needle, operator, key)
return NewJSONConditionBuilder(key, valueType).buildArrayFunctionCondition(operator, needle, sb)
}
return fmt.Sprintf("%s(%s, %s)", operator.FunctionName(), fieldExpr, sb.Var(needle)), nil
// legacy string-body path: type-matched array extraction, OR-ed with a scalar comparison
// for a scalar body value (coalesced to false so NOT has() matches missing-key rows).
elemType := legacyElemType(needle)
arrayExpr := getBodyJSONArrayKey(key, elemType)
scalarExpr, scalarGuard, hasScalar := getBodyJSONScalarKey(key, elemType)
if list, ok := needle.([]any); ok {
vals := make([]any, len(list))
for i, v := range list {
vals[i] = legacyCoerceNeedle(v, elemType)
}
arrayCond := fmt.Sprintf("%s(%s, %s)", operator.FunctionName(), arrayExpr, sb.Var(vals))
if !hasScalar {
return arrayCond, nil
}
var membership string
if operator == qbtypes.FilterOperatorHasAll {
eqs := make([]string, len(vals))
for i, v := range vals {
eqs[i] = sb.E(scalarExpr, v)
}
membership = sb.And(eqs...)
} else {
membership = sb.In(scalarExpr, vals...)
}
return fmt.Sprintf("(%s OR ifNull(%s, false))", arrayCond, sb.And(membership, scalarGuard)), nil
}
typedNeedle := legacyCoerceNeedle(needle, elemType)
arrayCond := fmt.Sprintf("%s(%s, %s)", operator.FunctionName(), arrayExpr, sb.Var(typedNeedle))
if !hasScalar {
return arrayCond, nil
}
return fmt.Sprintf("(%s OR ifNull(%s, false))", arrayCond, sb.And(sb.E(scalarExpr, typedNeedle), scalarGuard)), nil
}
// conditionForHasToken builds `hasToken(LOWER(<bodyColumn>), LOWER(<needle>))`, a
// full-text token search over the body column. It resolves the column from the key
// name + use_json_body flag, validates the field/value, and tags errors with the doc URL.
// conditionForHasToken builds a hasToken full-text search over the body column, resolving the
// column from the key name + use_json_body flag.
func (c *conditionBuilder) conditionForHasToken(
ctx context.Context,
orgID valuer.UUID,
@@ -94,27 +116,36 @@ func (c *conditionBuilder) conditionForHasToken(
needle = args[0]
}
bodyJSONEnabled := c.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID))
columnName := LogsV2BodyColumn
if bodyJSONEnabled {
if key.Name != LogsV2BodyColumn && key.Name != bodyMessageField {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"function `hasToken` only supports body/body.message field as first parameter").WithUrl(hasTokenFunctionDocURL)
}
columnName = bodyMessageField
} else if key.Name != LogsV2BodyColumn {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"function `hasToken` only supports body field as first parameter").WithUrl(hasTokenFunctionDocURL)
}
// hasToken matches string tokens only.
if _, ok := needle.(string); !ok {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"function `hasToken` expects value parameter to be a string").WithUrl(hasTokenFunctionDocURL)
}
return fmt.Sprintf("hasToken(LOWER(%s), LOWER(%s))", columnName, sb.Var(needle)), nil
bodyJSONEnabled := c.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID))
if !bodyJSONEnabled {
// legacy: token search over the plain body string column only.
if key.Name != LogsV2BodyColumn {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"function `hasToken` only supports body field as first parameter").WithUrl(hasTokenFunctionDocURL)
}
return fmt.Sprintf("hasToken(LOWER(%s), LOWER(%s))", LogsV2BodyColumn, sb.Var(needle)), nil
}
// JSON mode: a bare body/body.message key searches the body.message column; any other body
// field is a token search over its JSON string field, incl. strings nested in arrays.
// `body.message` resolves to a body-context key named `message`, so match that too — else it
// falls through and emits dynamicElement over the already-typed String column, which errors.
if key.Name == LogsV2BodyColumn || key.Name == bodyMessageField ||
(key.FieldContext == telemetrytypes.FieldContextBody && key.Name == messageSubField) {
return fmt.Sprintf("hasToken(LOWER(%s), LOWER(%s))", bodyMessageField, sb.Var(needle)), nil
}
if key.FieldContext == telemetrytypes.FieldContextBody {
return NewJSONConditionBuilder(key, telemetrytypes.FieldDataTypeString).buildTokenFunctionCondition(needle, sb)
}
return "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"function `hasToken` only supports the body field or a body JSON string field as first parameter").WithUrl(hasTokenFunctionDocURL)
}
func (c *conditionBuilder) conditionFor(
@@ -126,8 +157,7 @@ func (c *conditionBuilder) conditionFor(
value any,
sb *sqlbuilder.SelectBuilder,
) (string, error) {
// hasToken is a token search over the body column resolved purely from the key
// name + flag, independent of column resolution, so handle it before anything else.
// hasToken resolves from the key name + flag alone (no column resolution), so handle it first.
if operator == qbtypes.FilterOperatorHasToken {
return c.conditionForHasToken(ctx, orgID, key, value, sb)
}
@@ -137,10 +167,9 @@ func (c *conditionBuilder) conditionFor(
return "", err
}
// has/hasAny/hasAll build `has(<arrayFieldExpr>, value)` over body JSON arrays
// rather than going through the normal operator paths, so handle them up front.
// has/hasAny/hasAll take the body-JSON path, not the normal operator paths.
if operator.IsArrayFunctionOperator() {
return c.conditionForArrayFunction(ctx, orgID, startNs, endNs, key, operator, value, columns, sb)
return c.conditionForArrayFunction(ctx, orgID, key, operator, value, columns, sb)
}
// TODO(Piyush): Update this to support multiple JSON columns based on evolutions
@@ -401,23 +430,6 @@ func (c *conditionBuilder) ConditionFor(
}
}
// has/hasAny/hasAll need an array field: in JSON-body mode drop non-array matches so a
// scalar errors clearly instead of failing at ClickHouse runtime (legacy mode skips this).
if operator.IsArrayFunctionOperator() &&
c.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID)) {
arrayKeys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(keys))
for _, k := range keys {
if k.FieldDataType.IsArray() {
arrayKeys = append(arrayKeys, k)
}
}
if len(arrayKeys) == 0 {
return nil, warnings, errors.NewInvalidInputf(errors.CodeInvalidInput,
"function `%s` expects key parameter to be an array field; no array fields found", operator.FunctionName())
}
keys = arrayKeys
}
conds := make([]string, 0, len(keys))
for _, k := range keys {
cond, err := c.conditionForKey(ctx, orgID, startNs, endNs, k, operator, value, sb)

View File

@@ -44,34 +44,66 @@ func TestFilterExprLogsBodyJSON(t *testing.T) {
category: "json",
query: "has(body.requestor_list[*], 'index_service')",
shouldPass: true,
expectedQuery: `WHERE has(JSONExtract(JSON_QUERY(body, '$."requestor_list"[*]'), 'Array(String)'), ?)`,
expectedArgs: []any{"index_service"},
expectedQuery: `WHERE (has(JSONExtract(JSON_QUERY(body, '$."requestor_list"[*]'), 'Array(Nullable(String))'), ?) OR ifNull((JSON_VALUE(body, '$."requestor_list"') = ? AND JSONType(body, 'requestor_list') NOT IN ('Array', 'Object', 'Null')), false))`,
expectedArgs: []any{"index_service", "index_service"},
expectedErrorContains: "",
},
{
category: "json",
query: "has(body.int_numbers[*], 2)",
shouldPass: true,
expectedQuery: `WHERE has(JSONExtract(JSON_QUERY(body, '$."int_numbers"[*]'), 'Array(Float64)'), ?)`,
expectedArgs: []any{float64(2)},
expectedQuery: `WHERE (has(JSONExtract(JSON_QUERY(body, '$."int_numbers"[*]'), 'Array(Nullable(Float64))'), ?) OR ifNull((JSONExtract(JSON_VALUE(body, '$."int_numbers"'), 'Nullable(Float64)') = ? AND JSONType(body, 'int_numbers') NOT IN ('Array', 'Object', 'Null')), false))`,
expectedArgs: []any{float64(2), float64(2)},
expectedErrorContains: "",
},
{
category: "json",
query: "has(body.bool[*], true)",
shouldPass: true,
expectedQuery: `WHERE has(JSONExtract(JSON_QUERY(body, '$."bool"[*]'), 'Array(Bool)'), ?)`,
expectedArgs: []any{true},
expectedQuery: `WHERE (has(JSONExtract(JSON_QUERY(body, '$."bool"[*]'), 'Array(Nullable(String))'), ?) OR ifNull((JSON_VALUE(body, '$."bool"') = ? AND JSONType(body, 'bool') NOT IN ('Array', 'Object', 'Null')), false))`,
expectedArgs: []any{"true", "true"},
expectedErrorContains: "",
},
{
category: "json",
query: "NOT has(body.nested_num[*].float_nums[*], 2.2)",
shouldPass: true,
expectedQuery: `WHERE NOT (has(JSONExtract(JSON_QUERY(body, '$."nested_num"[*]."float_nums"[*]'), 'Array(Float64)'), ?))`,
expectedQuery: `WHERE NOT (has(JSONExtract(JSON_QUERY(body, '$."nested_num"[*]."float_nums"[*]'), 'Array(Nullable(Float64))'), ?))`,
expectedArgs: []any{float64(2.2)},
expectedErrorContains: "",
},
{
category: "json",
query: "has(body.tags, 'production')",
shouldPass: true,
expectedQuery: `WHERE (has(JSONExtract(JSON_QUERY(body, '$."tags"[*]'), 'Array(Nullable(String))'), ?) OR ifNull((JSON_VALUE(body, '$."tags"') = ? AND JSONType(body, 'tags') NOT IN ('Array', 'Object', 'Null')), false))`,
expectedArgs: []any{"production", "production"},
expectedErrorContains: "",
},
{
category: "json",
query: "hasAny(body.tags, ['critical', 'test'])",
shouldPass: true,
expectedQuery: `WHERE (hasAny(JSONExtract(JSON_QUERY(body, '$."tags"[*]'), 'Array(Nullable(String))'), ?) OR ifNull((JSON_VALUE(body, '$."tags"') IN (?, ?) AND JSONType(body, 'tags') NOT IN ('Array', 'Object', 'Null')), false))`,
expectedArgs: []any{[]any{"critical", "test"}, "critical", "test"},
expectedErrorContains: "",
},
{
category: "json",
query: "hasAll(body.tags, ['production', 'web'])",
shouldPass: true,
expectedQuery: `WHERE (hasAll(JSONExtract(JSON_QUERY(body, '$."tags"[*]'), 'Array(Nullable(String))'), ?) OR ifNull(((JSON_VALUE(body, '$."tags"') = ? AND JSON_VALUE(body, '$."tags"') = ?) AND JSONType(body, 'tags') NOT IN ('Array', 'Object', 'Null')), false))`,
expectedArgs: []any{[]any{"production", "web"}, "production", "web"},
expectedErrorContains: "",
},
{
category: "json",
query: "has(body.ids, \"200\")",
shouldPass: true,
expectedQuery: `WHERE (has(JSONExtract(JSON_QUERY(body, '$."ids"[*]'), 'Array(Nullable(Int64))'), ?) OR ifNull((JSONExtract(JSON_VALUE(body, '$."ids"'), 'Nullable(Int64)') = ? AND JSONType(body, 'ids') NOT IN ('Array', 'Object', 'Null')), false))`,
expectedArgs: []any{int64(200), int64(200)},
expectedErrorContains: "",
},
{
category: "json",
query: "body.message = hello",

View File

@@ -1561,6 +1561,25 @@ func TestFilterExprLogs(t *testing.T) {
expectedArgs: []any{"download"},
expectedErrorContains: "function `hasToken` expects value parameter to be a string",
},
// extra / mis-shaped value arguments are rejected, not silently dropped.
{
category: "hasExtraArgs",
query: "has(body.tags[*], \"a\", \"b\")",
shouldPass: false,
expectedErrorContains: "function `has` expects exactly one value argument",
},
{
category: "hasArrayArg",
query: "has(body.tags[*], [\"a\", \"b\"])",
shouldPass: false,
expectedErrorContains: "function `has` expects a single scalar value, not an array",
},
{
category: "hasTokenExtraArgs",
query: "hasToken(body, \"a\", \"b\")",
shouldPass: false,
expectedErrorContains: "function `hasToken` expects exactly one value argument",
},
// Basic materialized key
{

View File

@@ -96,6 +96,18 @@ func applyNotCondition(operator qbtypes.FilterOperator) (bool, qbtypes.FilterOpe
return false, operator
}
// branchArrayExpr returns the ClickHouse array expression for a given array-type branch
// at this hop. The JSON branch reads Array(JSON(...)) directly; the Dynamic branch filters
// the Array(Dynamic) down to its JSON elements and maps them to JSON.
func (c *jsonConditionBuilder) branchArrayExpr(node *telemetrytypes.JSONAccessNode, branch telemetrytypes.JSONAccessBranchType) string {
fieldPath := node.FieldPath()
if branch == telemetrytypes.BranchDynamic {
dynBaseExpr := fmt.Sprintf("dynamicElement(%s, 'Array(Dynamic)')", fieldPath)
return fmt.Sprintf("arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), %s))", dynBaseExpr)
}
return fmt.Sprintf("dynamicElement(%s, 'Array(JSON(max_dynamic_types=%d, max_dynamic_paths=%d))')", fieldPath, node.MaxDynamicTypes, node.MaxDynamicPaths)
}
// buildAccessNodeBranches builds conditions for each branch of the access node.
func (c *jsonConditionBuilder) buildAccessNodeBranches(current *telemetrytypes.JSONAccessNode, operator qbtypes.FilterOperator, value any, sb *sqlbuilder.SelectBuilder) (string, error) {
if current == nil {
@@ -103,31 +115,15 @@ func (c *jsonConditionBuilder) buildAccessNodeBranches(current *telemetrytypes.J
}
currAlias := current.Alias()
fieldPath := current.FieldPath()
// Determine availability of Array(JSON) and Array(Dynamic) at this hop
hasArrayJSON := current.Branches[telemetrytypes.BranchJSON] != nil
hasArrayDynamic := current.Branches[telemetrytypes.BranchDynamic] != nil
// Then, at this hop, compute child per branch and wrap
// At this hop, compute the child condition per array branch (JSON before Dynamic) and
// wrap each in arrayExists over the corresponding array expression.
branches := make([]string, 0, 2)
if hasArrayJSON {
jsonArrayExpr := fmt.Sprintf("dynamicElement(%s, 'Array(JSON(max_dynamic_types=%d, max_dynamic_paths=%d))')", fieldPath, current.MaxDynamicTypes, current.MaxDynamicPaths)
childGroupJSON, err := c.recurseArrayHops(current.Branches[telemetrytypes.BranchJSON], operator, value, sb)
for _, branch := range current.BranchesInOrder() {
childGroup, err := c.recurseArrayHops(current.Branches[branch], operator, value, sb)
if err != nil {
return "", err
}
branches = append(branches, fmt.Sprintf("arrayExists(%s-> %s, %s)", currAlias, childGroupJSON, jsonArrayExpr))
}
if hasArrayDynamic {
dynBaseExpr := fmt.Sprintf("dynamicElement(%s, 'Array(Dynamic)')", fieldPath)
dynFilteredExpr := fmt.Sprintf("arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), %s))", dynBaseExpr)
// Create the Query for Dynamic array
childGroupDyn, err := c.recurseArrayHops(current.Branches[telemetrytypes.BranchDynamic], operator, value, sb)
if err != nil {
return "", err
}
branches = append(branches, fmt.Sprintf("arrayExists(%s-> %s, %s)", currAlias, childGroupDyn, dynFilteredExpr))
branches = append(branches, fmt.Sprintf("arrayExists(%s-> %s, %s)", currAlias, childGroup, c.branchArrayExpr(current, branch)))
}
if len(branches) == 1 {
@@ -309,6 +305,174 @@ func (c *jsonConditionBuilder) buildArrayMembershipCondition(node *telemetrytype
return fmt.Sprintf("arrayExists(%s -> %s, %s)", key, op, arrayExpr), nil
}
// buildArrayFunctionCondition builds a has/hasAny/hasAll condition over a body JSON path,
// with contains-all semantics uniform across every leaf shape:
// - has(v) = the path HAS v
// - hasAny([v...]) = the path has ANY listed value (OR of has)
// - hasAll([v...]) = the path has ALL listed values (AND of has)
//
// "the path has v" is an existential match resolved per leaf shape: for an array-typed leaf
// (top-level `body.tags`, or nested `body.education[].scores`) it is native membership; for a
// scalar leaf — whether reached through an array hop (`body.items[].sku`) or a plain scalar
// path (`body.level`) — it is `<elem> = v`, wrapped in arrayExists over any array hops. So
// `hasAll(body.education[].name, ['a','b'])` = "some element is a AND some element is b", and
// for a plain scalar hasAll collapses to has (a one-element set can hold at most one value).
//
// Element comparisons reuse DataTypeCollisionHandledFieldName so a numeric literal against an
// Int64 array (or a numeric literal against a String array) no longer silently misses.
func (c *jsonConditionBuilder) buildArrayFunctionCondition(operator qbtypes.FilterOperator, value any, sb *sqlbuilder.SelectBuilder) (string, error) {
if len(c.key.JSONPlan) == 0 {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "function `%s` could not resolve a JSON access plan for field `%s`", operator.FunctionName(), c.key.Name)
}
switch operator {
case qbtypes.FilterOperatorHas, qbtypes.FilterOperatorHasAny:
return c.buildOredRootChains(func(node *telemetrytypes.JSONAccessNode) (string, error) {
return c.arrayFunctionLeaf(node, operator, value, sb)
}, sb)
case qbtypes.FilterOperatorHasAll:
// contains-all: AND of a per-value "has" so the AND sits outside the array hops.
values := toAnyList(value)
conditions := make([]string, 0, len(values))
for _, v := range values {
v := v
cond, err := c.buildOredRootChains(func(node *telemetrytypes.JSONAccessNode) (string, error) {
return c.arrayFunctionLeaf(node, qbtypes.FilterOperatorHas, v, sb)
}, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
}
if len(conditions) == 1 {
return conditions[0], nil
}
return sb.And(conditions...), nil
}
return "", qbtypes.ErrUnsupportedOperator
}
// buildOredRootChains applies leafFn down every JSONPlan root (base + promoted), wrapping each
// in its arrayExists chain, and ORs the per-root results.
func (c *jsonConditionBuilder) buildOredRootChains(leafFn func(*telemetrytypes.JSONAccessNode) (string, error), sb *sqlbuilder.SelectBuilder) (string, error) {
conditions := make([]string, 0, len(c.key.JSONPlan))
for _, root := range c.key.JSONPlan {
cond, err := c.buildArrayExistsChain(root, leafFn, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
}
if len(conditions) == 1 {
return conditions[0], nil
}
return sb.Or(conditions...), nil
}
// buildArrayExistsChain wraps the terminal condition (produced by leafFn) in an arrayExists
// over every array hop between the root and the terminal. For a terminal root (a top-level
// array leaf) it simply returns leafFn(root).
func (c *jsonConditionBuilder) buildArrayExistsChain(node *telemetrytypes.JSONAccessNode, leafFn func(*telemetrytypes.JSONAccessNode) (string, error), sb *sqlbuilder.SelectBuilder) (string, error) {
if node == nil {
return "", errors.NewInternalf(CodeArrayNavigationFailed, "navigation failed, current node is nil")
}
if node.IsTerminal {
return leafFn(node)
}
branches := make([]string, 0, 2)
for _, branch := range node.BranchesInOrder() {
childCond, err := c.buildArrayExistsChain(node.Branches[branch], leafFn, sb)
if err != nil {
return "", err
}
branches = append(branches, fmt.Sprintf("arrayExists(%s-> %s, %s)", node.Alias(), childCond, c.branchArrayExpr(node, branch)))
}
if len(branches) == 1 {
return branches[0], nil
}
return sb.Or(branches...), nil
}
// arrayFunctionLeaf builds the existential comparison for has/hasAny at a terminal node (hasAll
// composes from has in buildArrayFunctionCondition). For an array leaf it delegates to native
// membership; for a scalar leaf it compares the element directly.
func (c *jsonConditionBuilder) arrayFunctionLeaf(node *telemetrytypes.JSONAccessNode, operator qbtypes.FilterOperator, value any, sb *sqlbuilder.SelectBuilder) (string, error) {
if node.TerminalConfig.ElemType.IsArray {
return c.arrayLeafMembership(node, operator, value, sb)
}
switch operator {
case qbtypes.FilterOperatorHas:
return c.arrayFuncScalarLeaf(node, qbtypes.FilterOperatorEqual, value, sb)
case qbtypes.FilterOperatorHasAny:
return c.arrayFuncScalarLeaf(node, qbtypes.FilterOperatorIn, toAnyList(value), sb)
}
return "", qbtypes.ErrUnsupportedOperator
}
// arrayLeafMembership builds native membership for an array-typed leaf, reusing
// buildArrayMembershipCondition (which handles data-type collisions on each element).
func (c *jsonConditionBuilder) arrayLeafMembership(node *telemetrytypes.JSONAccessNode, operator qbtypes.FilterOperator, value any, sb *sqlbuilder.SelectBuilder) (string, error) {
switch operator {
case qbtypes.FilterOperatorHas:
return c.buildArrayMembershipCondition(node, qbtypes.FilterOperatorEqual, value, sb)
case qbtypes.FilterOperatorHasAny:
return c.buildArrayMembershipCondition(node, qbtypes.FilterOperatorIn, toAnyList(value), sb)
}
return "", qbtypes.ErrUnsupportedOperator
}
// arrayFuncScalarLeaf builds `<elemExpr> <op> value` for a scalar leaf reached through an
// array hop, applying data-type collision handling like the standard primitive path.
// Coalesced to false so a missing key is a non-match, not NULL (NOT has() must match it).
func (c *jsonConditionBuilder) arrayFuncScalarLeaf(node *telemetrytypes.JSONAccessNode, operator qbtypes.FilterOperator, value any, sb *sqlbuilder.SelectBuilder) (string, error) {
fieldExpr := fmt.Sprintf("dynamicElement(%s, '%s')", node.FieldPath(), node.TerminalConfig.ElemType.StringValue())
fieldExpr, value = querybuilder.DataTypeCollisionHandledFieldName(node.TerminalConfig.Key, value, fieldExpr, operator)
cond, err := c.applyOperator(sb, fieldExpr, operator, value)
if err != nil {
return "", err
}
return fmt.Sprintf("ifNull(%s, false)", cond), nil
}
// buildTokenFunctionCondition builds a hasToken search over a body JSON string field:
// hasToken(LOWER(<elem>), LOWER(?)) wrapped in arrayExists over any array hops between the
// root and the terminal. The field must resolve to a String leaf or a String array.
func (c *jsonConditionBuilder) buildTokenFunctionCondition(needle any, sb *sqlbuilder.SelectBuilder) (string, error) {
if len(c.key.JSONPlan) == 0 {
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "function `hasToken` could not resolve a JSON access plan for field `%s`", c.key.Name)
}
return c.buildOredRootChains(func(node *telemetrytypes.JSONAccessNode) (string, error) {
return c.tokenLeaf(node, needle, sb)
}, sb)
}
// tokenLeaf builds the hasToken match at a terminal node: a direct match for a String leaf
// (coalesced to false, as in arrayFuncScalarLeaf), or an arrayExists over the elements for a
// String array leaf. hasToken is string-only, so any other element type is rejected.
func (c *jsonConditionBuilder) tokenLeaf(node *telemetrytypes.JSONAccessNode, needle any, sb *sqlbuilder.SelectBuilder) (string, error) {
switch node.TerminalConfig.ElemType {
case telemetrytypes.String:
fieldExpr := fmt.Sprintf("dynamicElement(%s, 'String')", node.FieldPath())
return fmt.Sprintf("ifNull(hasToken(LOWER(%s), LOWER(%s)), false)", fieldExpr, sb.Var(needle)), nil
case telemetrytypes.ArrayString:
arrayExpr := fmt.Sprintf("dynamicElement(%s, '%s')", node.FieldPath(), node.TerminalConfig.ElemType.StringValue())
return fmt.Sprintf("arrayExists(x -> hasToken(LOWER(x), LOWER(%s)), %s)", sb.Var(needle), arrayExpr), nil
default:
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "function `hasToken` only supports string fields; field `%s` is `%s`", c.key.Name, node.TerminalConfig.Key.FieldDataType.StringValue())
}
}
// toAnyList normalizes a has-family value into a slice; a scalar becomes a one-element list.
func toAnyList(value any) []any {
if list, ok := value.([]any); ok {
return list
}
return []any{value}
}
func (c *jsonConditionBuilder) applyOperator(sb *sqlbuilder.SelectBuilder, fieldExpr string, operator qbtypes.FilterOperator, value any) (string, error) {
switch operator {
case qbtypes.FilterOperatorEqual:

View File

@@ -603,7 +603,7 @@ func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
name: "Simple has filter",
filter: "has(body.education[].parameters, 1.65)",
expected: TestExpected{
WhereClause: "(has(arrayFlatten(arrayConcat(arrayMap(`body_v2.education`->dynamicElement(`body_v2.education`.`parameters`, 'Array(Nullable(Float64))'), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')))), ?) OR has(arrayFlatten(arrayConcat(arrayMap(`body_v2.education`->dynamicElement(`body_v2.education`.`parameters`, 'Array(Dynamic)'), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')))), ?))",
WhereClause: "(arrayExists(`body_v2.education`-> arrayExists(x -> toFloat64(x) = ?, dynamicElement(`body_v2.education`.`parameters`, 'Array(Nullable(Float64))')), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')) OR arrayExists(`body_v2.education`-> arrayExists(x -> accurateCastOrNull(x, 'Float64') = ?, arrayFilter(x->(dynamicType(x) IN ('String', 'Int64', 'Float64', 'Bool')), dynamicElement(`body_v2.education`.`parameters`, 'Array(Dynamic)'))), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')))",
Args: []any{1.65, 1.65, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
Warnings: []string{
"Key `education[].parameters` is ambiguous, found 2 different combinations of field context / data type: [name=education[].parameters,context=body,datatype=[]float64 name=education[].parameters,context=body,datatype=[]dynamic].",
@@ -614,8 +614,8 @@ func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
name: "Flat path hasAll filter",
filter: "hasAll(body.user.permissions, ['read', 'write'])",
expected: TestExpected{
WhereClause: "hasAll(dynamicElement(body_v2.`user.permissions`, 'Array(Nullable(String))'), ?)",
Args: []any{[]any{"read", "write"}, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
WhereClause: "(arrayExists(x -> x = ?, dynamicElement(body_v2.`user.permissions`, 'Array(Nullable(String))')) AND arrayExists(x -> x = ?, dynamicElement(body_v2.`user.permissions`, 'Array(Nullable(String))')))",
Args: []any{"read", "write", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
@@ -740,8 +740,8 @@ func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
name: "Nested path hasAny filter",
filter: "hasAny(education[].awards[].participated[].members, ['Piyush', 'Tushar'])",
expected: TestExpected{
WhereClause: "hasAny(arrayFlatten(arrayConcat(arrayMap(`body_v2.education`->arrayConcat(arrayMap(`body_v2.education[].awards`->arrayConcat(arrayMap(`body_v2.education[].awards[].participated`->dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(JSON(max_dynamic_types=4, max_dynamic_paths=0))')), arrayMap(`body_v2.education[].awards[].participated`->dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))'), arrayMap(x->assumeNotNull(dynamicElement(x, 'JSON')), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(Dynamic)'))))), dynamicElement(`body_v2.education`.`awards`, 'Array(JSON(max_dynamic_types=8, max_dynamic_paths=0))')), arrayMap(`body_v2.education[].awards`->arrayConcat(arrayMap(`body_v2.education[].awards[].participated`->dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=256))')), arrayMap(`body_v2.education[].awards[].participated`->dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))'), arrayMap(x->assumeNotNull(dynamicElement(x, 'JSON')), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(Dynamic)'))))), arrayMap(x->assumeNotNull(dynamicElement(x, 'JSON')), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education`.`awards`, 'Array(Dynamic)'))))), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')))), ?)",
Args: []any{[]any{"Piyush", "Tushar"}, "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
WhereClause: "arrayExists(`body_v2.education`-> (arrayExists(`body_v2.education[].awards`-> (arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> x IN (?, ?), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(JSON(max_dynamic_types=4, max_dynamic_paths=0))')) OR arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> x IN (?, ?), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(Dynamic)'))))), dynamicElement(`body_v2.education`.`awards`, 'Array(JSON(max_dynamic_types=8, max_dynamic_paths=0))')) OR arrayExists(`body_v2.education[].awards`-> (arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> x IN (?, ?), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=256))')) OR arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> x IN (?, ?), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(Dynamic)'))))), arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education`.`awards`, 'Array(Dynamic)'))))), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{"Piyush", "Tushar", "Piyush", "Tushar", "Piyush", "Tushar", "Piyush", "Tushar", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
@@ -756,8 +756,8 @@ func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
name: "dynamic_array_element_compare_HAS_STRING",
filter: "has(interests[].entities[].product_codes, '2002')",
expected: TestExpected{
WhereClause: "has(arrayFlatten(arrayConcat(arrayMap(`body_v2.interests`->arrayMap(`body_v2.interests[].entities`->dynamicElement(`body_v2.interests[].entities`.`product_codes`, 'Array(Dynamic)'), dynamicElement(`body_v2.interests`.`entities`, 'Array(JSON(max_dynamic_types=8, max_dynamic_paths=0))')), dynamicElement(body_v2.`interests`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')))), ?)",
Args: []any{"2002", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
WhereClause: "arrayExists(`body_v2.interests`-> arrayExists(`body_v2.interests[].entities`-> arrayExists(x -> accurateCastOrNull(x, 'Float64') = ?, arrayFilter(x->(dynamicType(x) IN ('String', 'Int64', 'Float64', 'Bool')), dynamicElement(`body_v2.interests[].entities`.`product_codes`, 'Array(Dynamic)'))), dynamicElement(`body_v2.interests`.`entities`, 'Array(JSON(max_dynamic_types=8, max_dynamic_paths=0))')), dynamicElement(body_v2.`interests`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{int64(2002), "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
@@ -772,10 +772,146 @@ func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
name: "dynamic_array_element_compare_HAS_INT",
filter: "has(interests[].entities[].product_codes, 1001)",
expected: TestExpected{
WhereClause: "has(arrayFlatten(arrayConcat(arrayMap(`body_v2.interests`->arrayMap(`body_v2.interests[].entities`->dynamicElement(`body_v2.interests[].entities`.`product_codes`, 'Array(Dynamic)'), dynamicElement(`body_v2.interests`.`entities`, 'Array(JSON(max_dynamic_types=8, max_dynamic_paths=0))')), dynamicElement(body_v2.`interests`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')))), ?)",
WhereClause: "arrayExists(`body_v2.interests`-> arrayExists(`body_v2.interests[].entities`-> arrayExists(x -> accurateCastOrNull(x, 'Float64') = ?, arrayFilter(x->(dynamicType(x) IN ('String', 'Int64', 'Float64', 'Bool')), dynamicElement(`body_v2.interests[].entities`.`product_codes`, 'Array(Dynamic)'))), dynamicElement(`body_v2.interests`.`entities`, 'Array(JSON(max_dynamic_types=8, max_dynamic_paths=0))')), dynamicElement(body_v2.`interests`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{float64(1001), "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
// ── scalar leaf reached through an array ───
{
name: "Nested primitive leaf has",
filter: "has(body.education[].name, 'IIT')",
expected: TestExpected{
WhereClause: "arrayExists(`body_v2.education`-> ifNull(dynamicElement(`body_v2.education`.`name`, 'String') = ?, false), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{"IIT", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
name: "Nested primitive leaf hasAny",
filter: "hasAny(body.education[].name, ['IIT', 'MIT'])",
expected: TestExpected{
WhereClause: "arrayExists(`body_v2.education`-> ifNull(dynamicElement(`body_v2.education`.`name`, 'String') IN (?, ?), false), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{"IIT", "MIT", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
// contains-all: single-value hasAll over a nested leaf collapses to has.
name: "Nested primitive leaf hasAll single collapses to has",
filter: "hasAll(body.education[].name, 'IIT')",
expected: TestExpected{
WhereClause: "arrayExists(`body_v2.education`-> ifNull(dynamicElement(`body_v2.education`.`name`, 'String') = ?, false), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{"IIT", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
// contains-all: "some element is IIT AND some element is MIT" (AND of per-value has).
name: "Nested primitive leaf hasAll multi (contains-all)",
filter: "hasAll(body.education[].name, ['IIT', 'MIT'])",
expected: TestExpected{
WhereClause: "(arrayExists(`body_v2.education`-> ifNull(dynamicElement(`body_v2.education`.`name`, 'String') = ?, false), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')) AND arrayExists(`body_v2.education`-> ifNull(dynamicElement(`body_v2.education`.`name`, 'String') = ?, false), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))')))",
Args: []any{"IIT", "MIT", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
// ── numeric literal against an Int64 array is collision-handled ───
{
name: "Nested Int64 array has collision",
filter: "has(body.education[].scores, 90)",
expected: TestExpected{
WhereClause: "arrayExists(`body_v2.education`-> arrayExists(x -> toFloat64(x) = ?, dynamicElement(`body_v2.education`.`scores`, 'Array(Nullable(Int64))')), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{float64(90), "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
// ── hasAny folds multiple scalar arguments into one value set ─────
{
name: "hasAny folds multiple scalar args",
filter: "hasAny(body.user.permissions, 'read', 'write')",
expected: TestExpected{
WhereClause: "arrayExists(x -> x IN (?, ?), dynamicElement(body_v2.`user.permissions`, 'Array(Nullable(String))'))",
Args: []any{"read", "write", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
// ── hasToken over JSON string fields (nested leaf, top-level array, nested array) ──
{
name: "hasToken nested string leaf",
filter: "hasToken(body.education[].name, 'harvard')",
expected: TestExpected{
WhereClause: "arrayExists(`body_v2.education`-> ifNull(hasToken(LOWER(dynamicElement(`body_v2.education`.`name`, 'String')), LOWER(?)), false), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{"harvard", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
name: "hasToken top-level string array",
filter: "hasToken(body.user.permissions, 'admin')",
expected: TestExpected{
WhereClause: "arrayExists(x -> hasToken(LOWER(x), LOWER(?)), dynamicElement(body_v2.`user.permissions`, 'Array(Nullable(String))'))",
Args: []any{"admin", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
name: "hasToken nested string array",
filter: "hasToken(body.education[].awards[].participated[].members, 'piyush')",
expected: TestExpected{
WhereClause: "arrayExists(`body_v2.education`-> (arrayExists(`body_v2.education[].awards`-> (arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> hasToken(LOWER(x), LOWER(?)), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(JSON(max_dynamic_types=4, max_dynamic_paths=0))')) OR arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> hasToken(LOWER(x), LOWER(?)), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(Dynamic)'))))), dynamicElement(`body_v2.education`.`awards`, 'Array(JSON(max_dynamic_types=8, max_dynamic_paths=0))')) OR arrayExists(`body_v2.education[].awards`-> (arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> hasToken(LOWER(x), LOWER(?)), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=256))')) OR arrayExists(`body_v2.education[].awards[].participated`-> arrayExists(x -> hasToken(LOWER(x), LOWER(?)), dynamicElement(`body_v2.education[].awards[].participated`.`members`, 'Array(Nullable(String))')), arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education[].awards`.`participated`, 'Array(Dynamic)'))))), arrayMap(x->dynamicElement(x, 'JSON'), arrayFilter(x->(dynamicType(x) = 'JSON'), dynamicElement(`body_v2.education`.`awards`, 'Array(Dynamic)'))))), dynamicElement(body_v2.`education`, 'Array(JSON(max_dynamic_types=16, max_dynamic_paths=0))'))",
Args: []any{"piyush", "piyush", "piyush", "piyush", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
// ── hasToken over the message field: bare body and explicit body.message are
// equivalent, both target the body.message column directly (no dynamicElement) ──
{
name: "hasToken bare body",
filter: "hasToken(body, 'production')",
expected: TestExpected{
WhereClause: "hasToken(LOWER(body.message), LOWER(?))",
Args: []any{"production", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
Warnings: []string{bodySearchDefaultWarning},
},
},
{
name: "hasToken explicit body.message",
filter: "hasToken(body.message, 'production')",
expected: TestExpected{
WhereClause: "hasToken(LOWER(body.message), LOWER(?))",
Args: []any{"production", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
// ── scalar (non-array) leaf: treated as a single-element set ─────────────
{
name: "Scalar leaf has",
filter: "has(body.user.name, 'alice')",
expected: TestExpected{
WhereClause: "ifNull(dynamicElement(body_v2.`user.name`, 'String') = ?, false)",
Args: []any{"alice", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
name: "Scalar leaf hasAny",
filter: "hasAny(body.user.name, ['alice', 'bob'])",
expected: TestExpected{
WhereClause: "ifNull(dynamicElement(body_v2.`user.name`, 'String') IN (?, ?), false)",
Args: []any{"alice", "bob", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
// contains-all: single-value hasAll over a plain scalar collapses to has.
name: "Scalar leaf hasAll single collapses to has",
filter: "hasAll(body.user.name, 'alice')",
expected: TestExpected{
WhereClause: "ifNull(dynamicElement(body_v2.`user.name`, 'String') = ?, false)",
Args: []any{"alice", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
{
// contains-all over a one-element set: the scalar must equal every value, so
// distinct values never match.
name: "Scalar leaf hasAll multi (contains-all)",
filter: "hasAll(body.user.name, ['alice', 'bob'])",
expected: TestExpected{
WhereClause: "(ifNull(dynamicElement(body_v2.`user.name`, 'String') = ?, false) AND ifNull(dynamicElement(body_v2.`user.name`, 'String') = ?, false))",
Args: []any{"alice", "bob", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
},
}
for _, c := range cases {

View File

@@ -130,3 +130,121 @@ func GetBodyJSONKey(_ context.Context, key *telemetrytypes.TelemetryFieldKey, op
func GetBodyJSONKeyForExists(_ context.Context, key *telemetrytypes.TelemetryFieldKey, _ qbtypes.FilterOperator, _ any) string {
return fmt.Sprintf("JSON_EXISTS(body, '$.%s')", getBodyJSONPath(key))
}
// legacyElemType infers the has-family element type from the needle (legacy has no schema). It
// scans EVERY value so the chosen array type and all coerced needles agree — else ClickHouse
// raises "no supertype ... String" (code 386). Int64 stays distinct from Float64 so a quoted
// integer is exact past 2^53 (unquoted literals already arrive as float64, parsed upstream).
func legacyElemType(needle any) telemetrytypes.FieldDataType {
list, ok := needle.([]any)
if !ok {
list = []any{needle}
}
if len(list) == 0 {
return telemetrytypes.FieldDataTypeString
}
allInt, allNumeric := true, true
for _, v := range list {
switch t := v.(type) {
case float32, float64:
allInt = false
case int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64:
// integer Go types stay int-exact
case string:
if _, err := strconv.ParseInt(t, 10, 64); err != nil {
allInt = false
}
if _, err := strconv.ParseFloat(t, 64); err != nil {
allNumeric = false
}
default:
// booleans (and anything else) -> String; a bool renders to 'true'/'false', so a
// bool needle only matches genuine JSON booleans, not truthy numbers/strings.
allInt, allNumeric = false, false
}
}
switch {
case allInt:
return telemetrytypes.FieldDataTypeInt64
case allNumeric:
return telemetrytypes.FieldDataTypeFloat64
default:
return telemetrytypes.FieldDataTypeString
}
}
// legacyCoerceNeedle coerces a needle to elem type dt so its bound-arg type matches the
// extracted column (legacyElemType guarantees it's coercible).
func legacyCoerceNeedle(v any, dt telemetrytypes.FieldDataType) any {
switch dt {
case telemetrytypes.FieldDataTypeInt64:
if s, ok := v.(string); ok {
if i, err := strconv.ParseInt(s, 10, 64); err == nil {
return i
}
}
return v
case telemetrytypes.FieldDataTypeFloat64:
if s, ok := v.(string); ok {
f, _ := strconv.ParseFloat(s, 64)
return f
}
return v
default:
return bodyArrayNeedleString(v)
}
}
// getBodyJSONArrayKey extracts the leaf as Array(Nullable(<dt>)) — Nullable so a value of a
// different JSON type maps to NULL instead of corrupting (e.g. a non-numeric string → 0).
func getBodyJSONArrayKey(key *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) string {
arrKey := *key
if !strings.HasSuffix(arrKey.Name, "[*]") && !strings.HasSuffix(arrKey.Name, "[]") {
arrKey.Name += "[*]"
}
return fmt.Sprintf("JSONExtract(JSON_QUERY(body, '$.%s'), 'Array(Nullable(%s))')", getBodyJSONPath(&arrKey), dt.CHDataType())
}
// getBodyJSONScalarKey builds the single-element-set fallback for a scalar body value: the leaf
// extracted as a scalar of type dt, plus a guard restricting it to a genuinely scalar body. The
// guard is required because JSON_VALUE returns '' for an array/object/missing value, which would
// otherwise zero-value match (has(x,0) / has(x,false) / has(x,'') on any array). ok=false when
// the path still traverses an array ([*]/[]).
func getBodyJSONScalarKey(key *telemetrytypes.TelemetryFieldKey, dt telemetrytypes.FieldDataType) (expr string, guard string, ok bool) {
name := strings.TrimSuffix(strings.TrimSuffix(key.Name, "[*]"), "[]")
if strings.Contains(name, "[") {
return "", "", false
}
scalarKey := *key
scalarKey.Name = name
path := getBodyJSONPath(&scalarKey)
if dt == telemetrytypes.FieldDataTypeString {
expr = fmt.Sprintf("JSON_VALUE(body, '$.%s')", path)
} else {
// Nullable so a scalar of a different type (e.g. a bool/string where a number is
// searched) extracts to NULL rather than the type's default (0/false), which would
// otherwise zero-value match has(x, 0).
expr = fmt.Sprintf("JSONExtract(JSON_VALUE(body, '$.%s'), 'Nullable(%s)')", path, dt.CHDataType())
}
keys := strings.Split(name, ".")
for i, k := range keys {
keys[i] = "'" + k + "'"
}
guard = fmt.Sprintf("JSONType(body, %s) NOT IN ('Array', 'Object', 'Null')", strings.Join(keys, ", "))
return expr, guard, true
}
func bodyArrayNeedleString(v any) string {
switch t := v.(type) {
case string:
return t
case bool:
return strconv.FormatBool(t)
case float64:
return strconv.FormatFloat(t, 'f', -1, 64)
case float32:
return strconv.FormatFloat(float64(t), 'f', -1, 64)
default:
return fmt.Sprintf("%v", t)
}
}

View File

@@ -496,8 +496,8 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE has(JSONExtract(JSON_QUERY(body, '$.\"user_names\"[*]'), 'Array(String)'), ?) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"john_doe", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE (has(JSONExtract(JSON_QUERY(body, '$.\"user_names\"[*]'), 'Array(Nullable(String))'), ?) OR ifNull((JSON_VALUE(body, '$.\"user_names\"') = ? AND JSONType(body, 'user_names') NOT IN ('Array', 'Object', 'Null')), false)) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"john_doe", "john_doe", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
},
expectedErr: nil,
},

View File

@@ -1018,21 +1018,14 @@ func TestValidateRequiredFields(t *testing.T) {
data: wrapVariable("signoz/CustomVariable", `{}`),
wantContain: "CustomValue",
},
{
name: "ThresholdWithLabel missing value",
data: wrapPanel("signoz/TimeSeriesPanel", `{"thresholds": [{"color": "Red", "label": "high"}]}`),
wantContain: "Value",
},
// Value is intentionally not validate:"required" — 0 is a legitimate threshold
// (go-playground's required rejects a zero float), so a missing/zero value is
// accepted and only Color remains required on these threshold structs.
{
name: "ThresholdWithLabel missing color",
data: wrapPanel("signoz/TimeSeriesPanel", `{"thresholds": [{"value": 100, "label": "high", "color": ""}]}`),
wantContain: "Color",
},
{
name: "ComparisonThreshold missing value",
data: wrapPanel("signoz/NumberPanel", `{"thresholds": [{"operator": "above", "format": "text", "color": "Red"}]}`),
wantContain: "Value",
},
{
name: "ComparisonThreshold missing color",
data: wrapPanel("signoz/NumberPanel", `{"thresholds": [{"value": 100, "operator": "above", "format": "text", "color": ""}]}`),

View File

@@ -207,7 +207,7 @@ type HistogramBuckets struct {
}
type ListPanelSpec struct {
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitempty" validate:"dive"`
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitzero" validate:"dive"`
}
// ══════════════════════════════════════════════
@@ -252,14 +252,20 @@ type Legend struct {
}
type ThresholdWithLabel struct {
Value float64 `json:"value" validate:"required" required:"true"`
// Value is always present in the schema (required:"true"), but 0 is a legitimate
// threshold, so it drops validate:"required" — go-playground's required treats a
// zero float as unset and would wrongly reject value: 0.
Value float64 `json:"value" required:"true"`
Unit string `json:"unit"`
Color string `json:"color" validate:"required" required:"true"`
Label string `json:"label"`
}
type ComparisonThreshold struct {
Value float64 `json:"value" validate:"required" required:"true"`
// Value is always present in the schema (required:"true"), but 0 is a legitimate
// threshold, so it drops validate:"required" — go-playground's required treats a
// zero float as unset and would wrongly reject value: 0.
Value float64 `json:"value" required:"true"`
Operator ComparisonOperator `json:"operator"`
Unit string `json:"unit"`
Color string `json:"color" validate:"required" required:"true"`

View File

@@ -619,9 +619,9 @@ type SecondaryAggregation struct {
// if any, it will be used as the alias of the aggregation in the result
Alias string `json:"alias,omitempty"`
// groupBy fields to group by
GroupBy []GroupByKey `json:"groupBy,omitempty"`
GroupBy []GroupByKey `json:"groupBy,omitzero"`
// order by keys and directions
Order []OrderBy `json:"order,omitempty"`
Order []OrderBy `json:"order,omitzero"`
// limit the maximum number of rows to return
Limit int `json:"limit,omitempty"`
// limitBy fields to limit by
@@ -691,7 +691,7 @@ type Function struct {
Name FunctionName `json:"name"`
// args is the arguments to the function
Args []FunctionArg `json:"args,omitempty"`
Args []FunctionArg `json:"args,omitzero"`
}
// Copy creates a deep copy of Function.

View File

@@ -26,22 +26,22 @@ type QueryBuilderQuery[T any] struct {
// we want to support multiple aggregations
// currently supported: []Aggregation, []MetricAggregation
Aggregations []T `json:"aggregations,omitempty"`
Aggregations []T `json:"aggregations,omitzero"`
// disabled if true, the query will not be executed
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled"`
// search query is simple string
Filter *Filter `json:"filter,omitempty"`
// group by keys to group by
GroupBy []GroupByKey `json:"groupBy,omitempty"`
GroupBy []GroupByKey `json:"groupBy,omitzero"`
// order by keys and directions
Order []OrderBy `json:"order,omitempty"`
Order []OrderBy `json:"order,omitzero"`
// select columns to select
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitempty"`
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitzero"`
// limit the maximum number of rows to return
Limit int `json:"limit,omitempty"`
@@ -61,12 +61,12 @@ type QueryBuilderQuery[T any] struct {
// secondary aggregation to apply to the query
// on top of the primary aggregation
SecondaryAggregations []SecondaryAggregation `json:"secondaryAggregations,omitempty"`
SecondaryAggregations []SecondaryAggregation `json:"secondaryAggregations,omitzero"`
// functions to apply to the query
Functions []Function `json:"functions,omitempty"`
Functions []Function `json:"functions,omitzero"`
Legend string `json:"legend,omitempty"`
Legend string `json:"legend"`
// ShiftBy is extracted from timeShift function for internal use
// This field is not serialized to JSON

View File

@@ -8,7 +8,7 @@ type ClickHouseQuery struct {
// disabled if true, the query will not be executed
Disabled bool `json:"disabled"`
Legend string `json:"legend,omitempty"`
Legend string `json:"legend"`
}
// Copy creates a deep copy of the ClickHouseQuery.

View File

@@ -22,10 +22,10 @@ type QueryBuilderFormula struct {
// expression to apply to the query
Expression string `json:"expression"`
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled"`
// order by keys and directions
Order []OrderBy `json:"order,omitempty"`
Order []OrderBy `json:"order,omitzero"`
// limit the maximum number of rows to return
Limit int `json:"limit,omitempty"`
@@ -34,9 +34,9 @@ type QueryBuilderFormula struct {
Having *Having `json:"having,omitempty"`
// functions to apply to the formula result
Functions []Function `json:"functions,omitempty"`
Functions []Function `json:"functions,omitzero"`
Legend string `json:"legend,omitempty"`
Legend string `json:"legend"`
}
// Copy creates a deep copy of the QueryBuilderFormula.

View File

@@ -38,7 +38,7 @@ func (q QueryRef) Copy() QueryRef {
type QueryBuilderJoin struct {
Name string `json:"name"`
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled"`
// references into flat registry of queries
Left QueryRef `json:"left"`
@@ -50,18 +50,18 @@ type QueryBuilderJoin struct {
// primary aggregations: if empty ⇒ raw columns. Untyped — joins are deferred
// (see the commented JoinAggregation below).
Aggregations []any `json:"aggregations,omitempty"`
Aggregations []any `json:"aggregations,omitzero"`
// select columns to select
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitempty"`
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitzero"`
// post-join clauses (also used for aggregated joins)
Filter *Filter `json:"filter,omitempty"`
GroupBy []GroupByKey `json:"groupBy,omitempty"`
GroupBy []GroupByKey `json:"groupBy,omitzero"`
Having *Having `json:"having,omitempty"`
Order []OrderBy `json:"order,omitempty"`
Order []OrderBy `json:"order,omitzero"`
Limit int `json:"limit,omitempty"`
SecondaryAggregations []SecondaryAggregation `json:"secondaryAggregations,omitempty"`
Functions []Function `json:"functions,omitempty"`
SecondaryAggregations []SecondaryAggregation `json:"secondaryAggregations,omitzero"`
Functions []Function `json:"functions,omitzero"`
}
// JoinAggregation modelled a join aggregation as a trace/log/metric oneOf. Deferred:

View File

@@ -12,7 +12,7 @@ type PromQuery struct {
// stats if true, the query will return stats
Stats bool `json:"stats"`
Legend string `json:"legend,omitempty"`
Legend string `json:"legend"`
}
// Copy creates a deep copy of the PromQuery.

View File

@@ -349,11 +349,6 @@ func roundToNonZeroDecimals(val float64, n int) float64 {
order := math.Floor(math.Log10(absVal))
scale := math.Pow(10, -order+float64(n)-1)
rounded := math.Round(val*scale) / scale
if math.IsNaN(rounded) || math.IsInf(rounded, 0) {
// scale overflowed for subnormal values (order below ~-308); the
// finite input must stay finite or it serializes as "NaN".
return val
}
// Clean up floating point precision
str := strconv.FormatFloat(rounded, 'f', -1, 64)

View File

@@ -395,36 +395,3 @@ func TestRawData_MarshalJSON(t *testing.T) {
})
}
}
func TestRoundToNonZeroDecimals(t *testing.T) {
cases := []struct {
name string
in float64
n int
want float64
}{
{"whole number", 42, 3, 42},
{"ge one rounds to three decimals", 123.45678, 3, 123.457},
{"ge one rounds to two decimals", 123.45678, 2, 123.46},
{"trailing zeros trimmed", 1.5, 3, 1.5},
{"lt one keeps three significant digits", 0.0001234567, 3, 0.000123},
{"lt one keeps two significant digits", 0.0001234567, 2, 0.00012},
{"negative", -123.45678, 3, -123.457},
{"zero", 0, 3, 0},
// val*multiplier overflows to +Inf for near-max values (#12151).
{"near max stays finite", 1.6e308, 3, 1.6e308},
{"negative near max stays finite", -1.6e308, 3, -1.6e308},
// scale overflows to +Inf for subnormal values; Inf/Inf made NaN.
{"smallest subnormal stays finite", 5e-324, 3, 5e-324},
{"subnormal stays finite", 1e-310, 3, 1e-310},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.want, roundToNonZeroDecimals(tc.in, tc.n))
})
}
assert.True(t, math.IsNaN(roundToNonZeroDecimals(math.NaN(), 3)))
assert.Equal(t, math.Inf(1), roundToNonZeroDecimals(math.Inf(1), 3))
assert.Equal(t, math.Inf(-1), roundToNonZeroDecimals(math.Inf(-1), 3))
}

View File

@@ -38,7 +38,7 @@ const (
type QueryBuilderTraceOperator struct {
Name string `json:"name"`
Disabled bool `json:"disabled,omitempty"`
Disabled bool `json:"disabled"`
Expression string `json:"expression"`
@@ -48,11 +48,11 @@ type QueryBuilderTraceOperator struct {
ReturnSpansFrom string `json:"returnSpansFrom,omitempty"`
// Trace-specific ordering (only span_count and trace_duration allowed)
Order []OrderBy `json:"order,omitempty"`
Order []OrderBy `json:"order,omitzero"`
Aggregations []TraceAggregation `json:"aggregations,omitempty"`
Aggregations []TraceAggregation `json:"aggregations,omitzero"`
StepInterval Step `json:"stepInterval,omitempty"`
GroupBy []GroupByKey `json:"groupBy,omitempty"`
GroupBy []GroupByKey `json:"groupBy,omitzero"`
// having clause to apply to the aggregated query results
Having *Having `json:"having,omitempty"`
@@ -61,11 +61,11 @@ type QueryBuilderTraceOperator struct {
Offset int `json:"offset,omitempty"`
Cursor string `json:"cursor,omitempty"`
Legend string `json:"legend,omitempty"`
Legend string `json:"legend"`
// Other post-processing options
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitempty"`
Functions []Function `json:"functions,omitempty"`
SelectFields []telemetrytypes.TelemetryFieldKey `json:"selectFields,omitzero"`
Functions []Function `json:"functions,omitzero"`
// Internal parsed representation (not exposed in JSON)
ParsedExpression *TraceOperand `json:"-"`

View File

@@ -1561,3 +1561,192 @@ def test_dashboard_v2_rejects_comma_separated_aggregation(
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
# ─── round-trip serialization of zero-valued fields ──────────────────────────
# A minimal dashboard stripped from SigNoz/dashboards (cicd-perses.json), whose
# NumberPanel carries a real `threshold value: 0`. It packs every field whose zero
# value the create -> GET round-trip must preserve: thresholds with value 0 (which
# the required-tag validation used to reject on create), builder slices set to an
# explicit [] that must echo back as [] (yet stay absent, never null, when unset),
# and scalars disabled/legend that must always echo false/"".
def test_dashboard_v2_roundtrip_preserves_zero_values(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
headers = {"Authorization": f"Bearer {token}"}
dashboard = {
"schemaVersion": "v6",
"name": "roundtrip-zero-values",
"tags": [],
"spec": {
"display": {"name": "Roundtrip Zero Values"},
"panels": {
# NumberPanel: ComparisonThreshold value 0 + a builder query whose
# zero-valued slices/scalars are all set explicitly.
"number": {
"kind": "Panel",
"spec": {
"display": {"name": "number"},
"plugin": {
"kind": "signoz/NumberPanel",
"spec": {"thresholds": [{"value": 0, "operator": "above_or_equal", "color": "#c2780b", "format": "background"}]},
},
"queries": [
{
"kind": "scalar",
"spec": {
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "logs",
"aggregations": [{"expression": "count()"}],
"disabled": False,
"legend": "",
"groupBy": [],
"order": [],
"selectFields": [],
"functions": [],
},
}
},
}
],
},
},
# TimeSeriesPanel: ThresholdWithLabel value 0 + a bare builder query
# whose unset slices must be omitted (not null) on read-back.
"timeseries": {
"kind": "Panel",
"spec": {
"display": {"name": "timeseries"},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {"thresholds": [{"value": 0, "color": "#c2780b"}]},
},
"queries": [
{
"kind": "time_series",
"spec": {
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}]},
}
},
}
],
},
},
# PromQL query: legend/disabled must echo "" / false.
"promql": {
"kind": "Panel",
"spec": {
"display": {"name": "promql"},
"plugin": {"kind": "signoz/TimeSeriesPanel", "spec": {}},
"queries": [
{
"kind": "time_series",
"spec": {
"plugin": {
"kind": "signoz/PromQLQuery",
"spec": {"name": "A", "query": "up", "disabled": False, "legend": ""},
}
},
}
],
},
},
# ClickHouse query: legend/disabled must echo "" / false.
"clickhouse": {
"kind": "Panel",
"spec": {
"display": {"name": "clickhouse"},
"plugin": {"kind": "signoz/TimeSeriesPanel", "spec": {}},
"queries": [
{
"kind": "time_series",
"spec": {
"plugin": {
"kind": "signoz/ClickHouseSQL",
"spec": {"name": "A", "query": "SELECT 1", "disabled": False, "legend": ""},
}
},
}
],
},
},
},
},
}
# Create also asserts Bug 1: a threshold with value 0 is no longer rejected.
response = requests.post(
signoz.self.host_configs["8080"].get(BASE_URL),
json=dashboard,
headers=headers,
timeout=5,
)
assert response.status_code == HTTPStatus.CREATED, response.text
dashboard_id = response.json()["data"]["id"]
try:
response = requests.get(
signoz.self.host_configs["8080"].get(f"{BASE_URL}/{dashboard_id}"),
headers=headers,
timeout=5,
)
assert response.status_code == HTTPStatus.OK, response.text
panels = response.json()["data"]["spec"]["panels"]
def query_spec(panel_id: str) -> dict:
return panels[panel_id]["spec"]["queries"][0]["spec"]["plugin"]["spec"]
def threshold(panel_id: str) -> dict:
return panels[panel_id]["spec"]["plugin"]["spec"]["thresholds"][0]
number = query_spec("number")
timeseries = query_spec("timeseries")
promql = query_spec("promql")
clickhouse = query_spec("clickhouse")
# A value the create round-trips back verbatim: (description, actual, expected).
roundtrip_cases = [
("comparison threshold value 0", threshold("number")["value"], 0),
("threshold-with-label value 0", threshold("timeseries")["value"], 0),
("builder disabled false", number["disabled"], False),
("builder legend empty", number["legend"], ""),
("builder empty groupBy", number["groupBy"], []),
("builder empty order", number["order"], []),
("builder empty selectFields", number["selectFields"], []),
("builder empty functions", number["functions"], []),
("bare builder disabled false", timeseries["disabled"], False),
("bare builder legend empty", timeseries["legend"], ""),
("promql disabled false", promql["disabled"], False),
("promql legend empty", promql["legend"], ""),
("clickhouse disabled false", clickhouse["disabled"], False),
("clickhouse legend empty", clickhouse["legend"], ""),
]
for description, actual, expected in roundtrip_cases:
assert actual == expected, description
# An unset slice is omitted, never serialized as null: (description, spec, key).
absent_cases = [
("bare builder omits groupBy", timeseries, "groupBy"),
("bare builder omits order", timeseries, "order"),
("bare builder omits selectFields", timeseries, "selectFields"),
("bare builder omits functions", timeseries, "functions"),
]
for description, spec, key in absent_cases:
assert key not in spec, description
finally:
requests.delete(
signoz.self.host_configs["8080"].get(f"{BASE_URL}/{dashboard_id}"),
headers=headers,
timeout=5,
)

View File

@@ -563,214 +563,6 @@ def test_logs_json_body_nested_keys(
assert all(code == 200 for code in status_codes)
def test_logs_json_body_array_membership(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_logs: Callable[[list[Logs]], None],
) -> None:
"""
Setup:
Insert logs with JSON bodies containing arrays
Tests:
1. Search by has(body.tags[*], "value") - string array
2. Search by has(body.ids[*], 123) - numeric array
3. Search by has(body.flags[*], true) - boolean array
"""
now = datetime.now(tz=UTC)
log1_body = json.dumps(
{
"tags": ["production", "api", "critical"],
"ids": [100, 200, 300],
"flags": [True, False, True],
"users": [
{"name": "alice", "role": "admin"},
{"name": "bob", "role": "user"},
],
}
)
log2_body = json.dumps(
{
"tags": ["staging", "api", "test"],
"ids": [200, 400, 500],
"flags": [False, False, True],
"users": [
{"name": "charlie", "role": "user"},
{"name": "david", "role": "admin"},
],
}
)
log3_body = json.dumps(
{
"tags": ["production", "web", "important"],
"ids": [100, 600, 700],
"flags": [True, True, False],
"users": [
{"name": "alice", "role": "admin"},
{"name": "eve", "role": "user"},
],
}
)
insert_logs(
[
Logs(
timestamp=now - timedelta(seconds=3),
resources={"service.name": "app-service"},
attributes={},
body=log1_body,
severity_text="INFO",
),
Logs(
timestamp=now - timedelta(seconds=2),
resources={"service.name": "app-service"},
attributes={},
body=log2_body,
severity_text="INFO",
),
Logs(
timestamp=now - timedelta(seconds=1),
resources={"service.name": "app-service"},
attributes={},
body=log3_body,
severity_text="INFO",
),
]
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# Test 1: Search by has(body.tags[*], "production")
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v5/query_range"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
json={
"schemaVersion": "v1",
"start": int((now - timedelta(seconds=10)).timestamp() * 1000),
"end": int(now.timestamp() * 1000),
"requestType": "raw",
"compositeQuery": {
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "logs",
"disabled": False,
"limit": 100,
"offset": 0,
"filter": {"expression": 'has(body.tags[*], "production")'},
"order": [
{"key": {"name": "timestamp"}, "direction": "desc"},
],
"aggregations": [{"expression": "count()"}],
},
}
]
},
"formatOptions": {"formatTableResultForUI": False, "fillGaps": False},
},
)
assert response.status_code == HTTPStatus.OK
assert response.json()["status"] == "success"
results = response.json()["data"]["data"]["results"]
assert len(results) == 1
rows = results[0]["rows"]
assert len(rows) == 2 # log1 and log3 have "production" in tags
tags_list = [json.loads(row["data"]["body"])["tags"] for row in rows]
assert all("production" in tags for tags in tags_list)
# Test 2: Search by has(body.ids[*], 200)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v5/query_range"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
json={
"schemaVersion": "v1",
"start": int((now - timedelta(seconds=10)).timestamp() * 1000),
"end": int(now.timestamp() * 1000),
"requestType": "raw",
"compositeQuery": {
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "logs",
"disabled": False,
"limit": 100,
"offset": 0,
"filter": {"expression": "has(body.ids[*], 200)"},
"order": [
{"key": {"name": "timestamp"}, "direction": "desc"},
],
"aggregations": [{"expression": "count()"}],
},
}
]
},
"formatOptions": {"formatTableResultForUI": False, "fillGaps": False},
},
)
assert response.status_code == HTTPStatus.OK
assert response.json()["status"] == "success"
results = response.json()["data"]["data"]["results"]
assert len(results) == 1
rows = results[0]["rows"]
assert len(rows) == 2 # log1 and log2 have 200 in ids
ids_list = [json.loads(row["data"]["body"])["ids"] for row in rows]
assert all(200 in ids for ids in ids_list)
# Test 3: Search by has(body.flags[*], true)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v5/query_range"),
timeout=2,
headers={"authorization": f"Bearer {token}"},
json={
"schemaVersion": "v1",
"start": int((now - timedelta(seconds=10)).timestamp() * 1000),
"end": int(now.timestamp() * 1000),
"requestType": "raw",
"compositeQuery": {
"queries": [
{
"type": "builder_query",
"spec": {
"name": "A",
"signal": "logs",
"disabled": False,
"limit": 100,
"offset": 0,
"filter": {"expression": "has(body.flags[*], true)"},
"order": [
{"key": {"name": "timestamp"}, "direction": "desc"},
],
"aggregations": [{"expression": "count()"}],
},
}
]
},
"formatOptions": {"formatTableResultForUI": False, "fillGaps": False},
},
)
assert response.status_code == HTTPStatus.OK
assert response.json()["status"] == "success"
results = response.json()["data"]["data"]["results"]
assert len(results) == 1
rows = results[0]["rows"]
assert len(rows) == 3 # All logs have true in flags
flags_list = [json.loads(row["data"]["body"])["flags"] for row in rows]
assert all(True in flags for flags in flags_list)
def test_logs_json_body_listing(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument

File diff suppressed because it is too large Load Diff