5 Commits

Author SHA1 Message Date
Naman Verma
fd032291f9 feat: add heatmap support in query (#12764)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Heatmap support here is only for metrics (except exponential histograms)
via all three query types: builder, clickhouse and promql. Logs and
traces can be plugged in into this later.

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR

Closes https://github.com/SigNoz/pulse-pod/issues/311
2026-09-11 10:19:43 +00:00
Srikanth Chekuri
8e00c04056 refactor(qb): quote field names with the ClickHouse quoting helpers (#12593)
#### Description

- Every user-controlled field name that reaches generated SQL goes
through the new `pkg/clickhousesql` package (`Identifier`,
`StringLiteral`, `Literal`, `LikePattern`): map reads and `mapContains`,
JSON sub-column paths and the JSON body access plan, labels, fingerprint
labels, materialized column names, select aliases, group-by and order-by
references, the legacy string-body JSONPath, and the raw SQL in the
trace funnel, trace detail and infra monitoring modules. Filter
expressions built from request or telemetry values use
`querybuilder.FilterStringLiteral`. The same package now also renders
dashboard variable values in the querier, LIKE patterns in the metadata
store and label lists in the PromQL transpiler, which each had their own
escaping.
- A `$` followed by a digit, `{` or `?` is written as `\x24`, which
ClickHouse decodes in identifiers and literals. Those are the forms the
tools react to: go-sqlbuilder resolves `$0` in a compiled fragment to
its own WHERE clause and recurses until the stack overflows, and
clickhouse-go rejects a query mixing `$<digits>` with `?` arguments. Any
other `$` stays literal, so materialized column names keep their `$$`
and render exactly as before; a key like `http.2xx` becomes ``
`attribute_string_http$\x242xx` `` instead of failing in the driver.
- Compiled sqlbuilder fragments (Select, GroupBy, OrderBy, raw Where
text) are wrapped with `sqlbuilder.Escape`; the metrics builder escapes
its compiled time-series subquery, which is compiled a second time when
joined.
- The raw statement validator (`ErrIfStatementIsNotValid`,
`LogIfStatementIsNotValid`) moves from
`pkg/querybuilder/clickhouse_sql.go` to
`pkg/clickhousesql/statement.go`. Its `Code*` identifiers drop the
`ClickHouseSQL` prefix; the code strings are unchanged.
- Unit tests round-trip the helpers over hostile names and drive them
through the modules' raw SQL;
`tests/integration/tests/queriercommon/08_field_name_quoting.py` and
`querier_json_body/07_field_name_quoting.py` query such names through
the logs, traces and metrics builders against a real ClickHouse.

#### Additional Information

- `docs/contributing/go/clickhousesql.md` documents the quoting
functions, where `sqlbuilder.Escape` belongs, the `$` rule and the
statement validator; `.claude/rules/go-contrib.md` points at it.
- `pkg/clickhousesql` is a leaf package so `telemetrytypes` (JSON access
plan) and `querybuilder` share one implementation without a cycle.
- For names without special characters the generated SQL is byte
identical.
- Not covered here: the legacy v3/v4 query_range builders and the
`pkg/query-service/utils` quoting helpers (`QuoteEscapedString`,
`QuoteEscapedStringForContains`, `ClickHouseFormattedValue`,
`AddBackTickToFormatTag`), the collector's `JSONSubColumnIndexExpr`, and
aggregation arguments naming a key that contains a backtick (rejected by
the SQL parser, a 500 as before).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-09-10 13:19:23 +00:00
Tushar Vats
a7fd14eac9 refactor(metrics): alias group-by columns positionally like logs and traces (#12464)
A metric label may be named after a column the generated query builds
for itself, and the metrics and meter builders selected group-by columns
under the label's own name — so `group by ts` or `group by value`
produced SQL with two columns of that name, which ClickHouse rejects.

### What

- Metrics and meter now alias group-by columns
`__GROUP_BY_KEY_<i>_<name>`, the scheme the logs and traces statement
builders already use.
- `pkg/querier/consume.go` already strips that prefix on all three read
paths (time-series, scalar, raw), so API responses are unchanged.
- Metrics' `ColumnExpressionFor` now returns the bare expression like
the logs and traces mappers. It was the only one returning an aliased
expression (`expr AS <name>`), which `agg_rewrite.go` splices inside a
function argument — giving `sum(expr AS <name>)` if metrics ever grows
expression aggregations. Callers alias and escape, as logs does.
- The histogram pipeline derives its CTE-side query once — `le` appended
last, plus the existing rate/sum rewrite — instead of mutating the query
and restoring it around the whole pipeline. The final select takes the
original minus `le`, so the remaining keys hold the positions their CTE
aliases were built from.

With a label named `ts`, before:

```sql
SELECT ts, `ts`, multiIf(…) … GROUP BY fingerprint, ts, `ts`
```

and after:

```sql
SELECT ts, `__GROUP_BY_KEY_0_ts`, multiIf(…) …
```

A label named `value` was the quieter case — the spatial CTE selected it
next to the aggregate of the same name:

```sql
SELECT ts, `value`, sum(per_series_value) AS value …
```

### Notes

- Meter comes along because it holds a
`*metricsstatementbuilder.StatementBuilder` and calls the shared
`BuildFinalSelect`; aliasing metrics alone would leave meter ordering by
an alias its own select never produced. `GroupByColumnAlias` /
`GroupByAliases` are exported for it, alongside the `GetKeySelectors` /
`RateTmpl` already shared across that boundary.
- Meter had the identical collision, so this fixes it there too.

### Testing

- `TestGroupByAliasAvoidsColumnCollision` covers `ts`, `value`,
`fingerprint` and an ordinary label, in both the metrics and meter
builders; all three collision cases fail without the change.
- `reduced_test.go` gains `histogram_p99_group_by` and
`gauge_avg_avg_group_by` — the reduced path had no group-by coverage at
all, so neither the aliases in its four CTE builders nor the union's
`ORDER BY` were exercised. The histogram case pins that both `UNION ALL`
branches emit the same columns.
- `test_histogram_count_no_param` pins the `SELECT *` branch, where `le`
stays unaliased so `ORDER BY toFloat64(le)` resolves.
- Both new behaviours were mutation-checked: appending `le` first
instead of last, and returning the bare name from `GroupByColumnAlias`,
each turn the relevant tests red.
- Twelve expected-SQL blobs regenerated across the metrics and meter
statement builder tests — alias-only diffs.
- `go test ./...` green, `make go-lint` clean.

Fixes https://github.com/SigNoz/engineering-pod/issues/5868
2026-08-13 16:35:36 +00:00
Tushar Vats
4e45620b72 refactor(statementbuilder): drop Builders bundle; fold factories into statement_builder.go (#12330)
Remove the statementbuilder.Builders aggregate. The querier chain (querier.New,
signozquerier.NewFactory, NewQuerierProviderFactories) now takes the six per-signal
statement builders individually, and signoz.go::newQueryStack returns them directly
via a plain multi-return. The statementbuilder package is now config-only.

Remove each <signal>statementbuilder/new.go and fold its NewFactory into that
package's statement_builder.go (traces' NewOperatorFactory into
trace_operator_statement_builder.go), giving the layout: struct -> NewFactory ->
New<X>QueryStatementBuilder.
2026-07-29 10:46:55 +00:00
Tushar Vats
e2e7caf1ca refactor(querier): 3-layer per-signal query architecture (#12304)
Split the five telemetry<signal> packages, which mixed three concerns, into
three per-signal layers with a cycle-free dependency direction:

- telemetryschema/<signal>telemetryschema — primitives (const + table
  selection, field_mapper, condition_builder, trace helpers); leaf layer.
- statementbuilder/<signal>statementbuilder — SQL generation. The parent
  statementbuilder package is contract-only (the Builders bundle + Config);
  each sub-package exposes a factory.ProviderFactory[..., statementbuilder.Config]
  whose New internalizes FieldMapper/ConditionBuilder/AggExprRewriter and reads
  SkipResourceFingerprint. Traces exposes two factories (query + operator).
  telemetryresourcefilter moves here as statementbuilder/resourcefilter.
- telemetrymetadata — key/value resolution; NewTelemetryMetaStore collapses
  from 24 args to (settings, telemetrystore, flagger), sourcing table names
  from the schema constants.

Centralize query-stack assembly in signoz.go via newQueryStack: build the
single metadata store, run each per-signal statement-builder factory, assemble
the statementbuilder.Builders bundle, and build the bucket cache — once. This
is the only place that imports the concrete sub-packages (so the edge runs
subs -> parent, cycle-free), and it removes the duplicate metadata store that
signozquerier used to build, leaving signozquerier a thin
querier.New(*statementbuilder.Builders) adapter.

Also:
- statementbuilder.Config owns SkipResourceFingerprint (moved off
  querier.Config). YAML key moves querier.skip_resource_fingerprint ->
  statementbuilder.skip_resource_fingerprint.
- Querier interface moves into querier.go (interfaces.go removed); BucketCache
  -> bucket_cache.go, Handler -> api.go.
- Add pkg/querier/queriertest.MockQuerier.
2026-07-29 09:24:08 +00:00