mirror of
https://github.com/SigNoz/signoz.git
synced 2026-08-06 05:00:42 +01:00
a627a7395f
An allowlist compiler (classify/rewrite) evaluates proven PromQL shapes
entirely inside ClickHouse on the timeSeries*ToGrid aggregate functions
(CH >= 25.6): one row per output series comes back instead of every raw
sample. Everything not provably equivalent falls back to the engine over
the native querier; transpilable subtrees under non-transpilable nodes run
hybrid (materialized as synthetic series, engine on top). TryExecuteRange
slots into the serve/shadow paths, which until now ran engine-only.
Window-sliver filtering folded in: when the window is narrower than the
step, a lattice predicate admits only the samples any grid point can see -
measured 74s/28GiB -> 16s/4.3GiB on a 36k-series 1w rate, and a
2.67B-sample 1w case that died at 150GiB completes in 19s/17GiB. Over
slivered rows the last-style gates lift and disjoint over_time forms drop
the divisibility gate.
The classification golden freezes the routing decision (full/hybrid/
fallback + reason) for every conformance-corpus expression: silently
falling back costs the pushdown, silently transpiling an unproven shape
risks wrong numbers - both now surface in review as a golden diff.
The dual-leg conformance suite already earned its keep on its first
transpiled run:
- It caught the classifier reading a duration expression's offset
(x offset step()) as zero - offset expressions parse WITHOUT the
experimental-parser flag, so they reach production. Such selectors are
now refused and the engine evaluates them exactly.
- It caught name-drop assembly treating temporally-disjoint same-labelset
twins as separate series (-{job="api"} spanning http_requests and
http_errors 400'd; hybrid -metric_a or -metric_b returned duplicate {}
series). Both paths now merge by labelset slot-wise, raising the
engine's duplicate error only on a same-timestamp conflict - the
engine's actual rule.
- The 12 remaining divergences are one class, recorded with causes in
known_divergences_v2.json (the swap scorecard): the engine sums with
Kahan compensation and an overflow-free incremental mean, ClickHouse's
sum/avg/arraySum are naive - visible only at 1e100-class cancellation
and near-max-float overflow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>