#### Description
- A logs filter on a bare key that lives in **both** resource and
another context (body or scope) ANDed the two: the resource candidate
built the `__resource_filter` fingerprint CTE while the other candidate
landed as a required main-query term, so the query matched almost
nothing.
- `ResolveLogicalFields` only preferred resource over `attribute`.
Generalized it to prefer resource over **any** other context (attribute,
body, scope, …); other contexts stay reachable via their qualified names
(e.g. `body.service.name`).
#### Issues closed by this PR
ClosesSigNoz/engineering-pod#6086
Part of https://github.com/SigNoz/platform-pod/issues/3158
#### Additional Information
Generalized rather than special-casing body/scope, since any future
context would hit the same fingerprint-CTE trap.
The semantic convention family as first call citizen revealed that the
current state of the query builder needs a bit refactoring for long term
maintenance.
The `FieldMapper` and `ConditionBuilder` are now one abstraction
`Storage`.
A storage now answers
- what the compiler cannot know i.e one read per field key (the bare
SQL, the membership present or absent, what an absent row reads, and
whether the read keeps its type or filters only).
- the fallback for a key metadata does not report
- its traits
- and one Condition compilation part.
And we introduce a new type to use in the system, `Resolved`
```
// Resolved is what resolution produces for one key: its meanings, and how
// they came to be. It is the only thing the compilers receive. Compile it
// with the operator and value it was resolved with.
type Resolved struct {
Key *telemetrytypes.TelemetryFieldKey
Fields []*telemetrytypes.LogicalField
// FromFallback: the fields came from the storage's fallback, not from
// metadata matches.
FromFallback bool
// Ambiguous: the matches held several interpretations.
Ambiguous bool
// Skipped: the storage contributes nothing for this key.
Skipped bool
Warnings []string
}
```
The prepared SQL has no changes, where it changed, it specifically made
the expression better by removing the redundant part.
- The prepared SQL remains identical with this refactoring
- No changes to integration tests
Assisted-by: Claude Fable 5.1
#### Description
- Every user-controlled field name that reaches generated SQL goes
through the new `pkg/clickhousesql` package (`Identifier`,
`StringLiteral`, `Literal`, `LikePattern`): map reads and `mapContains`,
JSON sub-column paths and the JSON body access plan, labels, fingerprint
labels, materialized column names, select aliases, group-by and order-by
references, the legacy string-body JSONPath, and the raw SQL in the
trace funnel, trace detail and infra monitoring modules. Filter
expressions built from request or telemetry values use
`querybuilder.FilterStringLiteral`. The same package now also renders
dashboard variable values in the querier, LIKE patterns in the metadata
store and label lists in the PromQL transpiler, which each had their own
escaping.
- A `$` followed by a digit, `{` or `?` is written as `\x24`, which
ClickHouse decodes in identifiers and literals. Those are the forms the
tools react to: go-sqlbuilder resolves `$0` in a compiled fragment to
its own WHERE clause and recurses until the stack overflows, and
clickhouse-go rejects a query mixing `$<digits>` with `?` arguments. Any
other `$` stays literal, so materialized column names keep their `$$`
and render exactly as before; a key like `http.2xx` becomes ``
`attribute_string_http$\x242xx` `` instead of failing in the driver.
- Compiled sqlbuilder fragments (Select, GroupBy, OrderBy, raw Where
text) are wrapped with `sqlbuilder.Escape`; the metrics builder escapes
its compiled time-series subquery, which is compiled a second time when
joined.
- The raw statement validator (`ErrIfStatementIsNotValid`,
`LogIfStatementIsNotValid`) moves from
`pkg/querybuilder/clickhouse_sql.go` to
`pkg/clickhousesql/statement.go`. Its `Code*` identifiers drop the
`ClickHouseSQL` prefix; the code strings are unchanged.
- Unit tests round-trip the helpers over hostile names and drive them
through the modules' raw SQL;
`tests/integration/tests/queriercommon/08_field_name_quoting.py` and
`querier_json_body/07_field_name_quoting.py` query such names through
the logs, traces and metrics builders against a real ClickHouse.
#### Additional Information
- `docs/contributing/go/clickhousesql.md` documents the quoting
functions, where `sqlbuilder.Escape` belongs, the `$` rule and the
statement validator; `.claude/rules/go-contrib.md` points at it.
- `pkg/clickhousesql` is a leaf package so `telemetrytypes` (JSON access
plan) and `querybuilder` share one implementation without a cycle.
- For names without special characters the generated SQL is byte
identical.
- Not covered here: the legacy v3/v4 query_range builders and the
`pkg/query-service/utils` quoting helpers (`QuoteEscapedString`,
`QuoteEscapedStringForContains`, `ClickHouseFormattedValue`,
`AddBackTickToFormatTag`), the collector's `JSONSubColumnIndexExpr`, and
aggregation arguments naming a key that contains a backtick (rejected by
the SQL parser, a 500 as before).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
#### Description
- `IN` and `NOT IN` now route each value back through the condition
builder with `=` / `!=` instead of assembling the comparisons a second
time, so whatever a builder does for a scalar comparison applies to the
list form too. Applied to logs, traces and audit — the three that
already fanned a list out into per-value comparisons.
- Fixes `body.<path>[*] IN [...]` with `use_json_body` off returning a
**500**. The list shape made the path extract as `Array(String)`, and
ClickHouse refuses to compare an array to a scalar (code 130);
extracting per value reads the field instead. The new case in
`querierlogs/06_json_body.py` fails on `main` and passes here — verified
both ways against a real ClickHouse.
#### Additional Information
- **resourcefilter is deliberately left out**: it asserts the key index
filter (`labels LIKE '%key%'`) once for the whole list, alongside one
value filter per value. A recursed arm derives its own key filter, so
the same predicate would be repeated per value — `(e1 AND kIdx AND l1)
OR (e2 AND kIdx AND l2)` instead of `(e1 OR e2) AND kIdx AND (l1 OR
l2)`. Same rows either way, but no reason to emit the duplicate.
- **telemetrymetadata is deliberately left out**: it applies a
key-existence guard at a single exit, so a recursed arm comes back
already wrapped and the guard would either nest or the case would have
to skip the shared tail — losing the invariant that every condition is
guarded.
- **metrics and rulestatehistory** build a real `sb.In`; there is no
per-value fan-out to delegate to.
- Mixed-type lists are safe: `DataTypeCollisionHandledFieldName`
normalises the whole list before the loop, so `IN ('200', 5)` emits
identical SQL before and after.
- Base of a stack — the follow-ups add index-friendly predicates to `=`,
which `IN` then picks up.
- Replace the search() stub: the logs condition builder fans a case-insensitive
match of the term across every searchable column (log columns, body/body_v2,
attribute + resource maps).
- Grammar: searchCall takes a valueList (parser regenerated), so the scoped form
search('term', body, resource) parses and narrows the fan-out to those contexts.
- The visitor emits FilterOperatorSearch and flags the statement scan-heavy; the
statement builder attaches a CostGuard the querier enforces via EXPLAIN ESTIMATE
against a per-shard budget, cumulative across buckets on the window-list path.
- body_v2 gets its own, lower budget (search_max_scan_rows_json_body): toString()
rebuilds every document and no skip index prunes.
- Logs-only: other signals reject search().
- Unit tests for the per-context fan-out SQL and both budgets; integration suites
run the same matrix over the legacy body and over body_v2.
- Embed statementbuilder.Config into querier.Config with mapstructure ",squash";
keys move to querier.skip_resource_fingerprint.* (env SIGNOZ_QUERIER_*).
- Drop the standalone statementbuilder section and its config factory.
- Pass statementbuilder.Config wholesale into NewLogQueryStatementBuilder.
These fired at info on every query build across the audit, logs and traces
builders. The behavior is settled, so drop them to debug and remove the
TODOs that asked for exactly this.
Remove the statementbuilder.Builders aggregate. The querier chain (querier.New,
signozquerier.NewFactory, NewQuerierProviderFactories) now takes the six per-signal
statement builders individually, and signoz.go::newQueryStack returns them directly
via a plain multi-return. The statementbuilder package is now config-only.
Remove each <signal>statementbuilder/new.go and fold its NewFactory into that
package's statement_builder.go (traces' NewOperatorFactory into
trace_operator_statement_builder.go), giving the layout: struct -> NewFactory ->
New<X>QueryStatementBuilder.
Split the five telemetry<signal> packages, which mixed three concerns, into
three per-signal layers with a cycle-free dependency direction:
- telemetryschema/<signal>telemetryschema — primitives (const + table
selection, field_mapper, condition_builder, trace helpers); leaf layer.
- statementbuilder/<signal>statementbuilder — SQL generation. The parent
statementbuilder package is contract-only (the Builders bundle + Config);
each sub-package exposes a factory.ProviderFactory[..., statementbuilder.Config]
whose New internalizes FieldMapper/ConditionBuilder/AggExprRewriter and reads
SkipResourceFingerprint. Traces exposes two factories (query + operator).
telemetryresourcefilter moves here as statementbuilder/resourcefilter.
- telemetrymetadata — key/value resolution; NewTelemetryMetaStore collapses
from 24 args to (settings, telemetrystore, flagger), sourcing table names
from the schema constants.
Centralize query-stack assembly in signoz.go via newQueryStack: build the
single metadata store, run each per-signal statement-builder factory, assemble
the statementbuilder.Builders bundle, and build the bucket cache — once. This
is the only place that imports the concrete sub-packages (so the edge runs
subs -> parent, cycle-free), and it removes the duplicate metadata store that
signozquerier used to build, leaving signozquerier a thin
querier.New(*statementbuilder.Builders) adapter.
Also:
- statementbuilder.Config owns SkipResourceFingerprint (moved off
querier.Config). YAML key moves querier.skip_resource_fingerprint ->
statementbuilder.skip_resource_fingerprint.
- Querier interface moves into querier.go (interfaces.go removed); BucketCache
-> bucket_cache.go, Handler -> api.go.
- Add pkg/querier/queriertest.MockQuerier.
