refactor(querybuilder): derive the allowed table function message from the map

Values the map by the spelling to name back to the caller, so the message comes off the same data the lookup uses and cannot drift from it. Drops the test that existed only to catch that drift.
test(querybuilder): address review on the generator allow list
2026-08-02 19:30:33 +01:00 · 2026-08-02 17:32:19 +05:30 · 2026-08-02 17:32:19 +05:30 · 2026-08-02 17:32:18 +05:30 · 2026-08-02 11:35:08 +00:00 · 2026-08-02 09:28:15 +00:00
17 changed files with 427 additions and 121 deletions
--- a/.github/workflows/cacheci.yml
+++ b/.github/workflows/cacheci.yml
@@ -0,0 +1,92 @@
+name: cacheci
+
+on:
+  push:
+    branches:
+      - main
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  actions: write
+
+# Cancelling mid-rotation is safe: the sequential delete-then-save order
+# leaves at most one key missing at any moment.
+concurrency:
+  group: cacheci
+  cancel-in-progress: true
+
+jobs:
+  tests:
+    runs-on: ubuntu-latest
+    steps:
+      - name: checkout
+        uses: actions/checkout@v4
+      - name: restore
+        id: restore
+        uses: actions/cache/restore@v4
+        with:
+          path: ${{ runner.temp }}/cacheci
+          key: tests-primary
+          restore-keys: |
+            tests-secondary
+      - name: inject
+        if: steps.restore.outputs.cache-matched-key != ''
+        run: |
+          cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
+          FROM busybox:1.37
+          RUN --mount=type=cache,target=/root/.cache/go-build \
+              --mount=type=cache,target=/go/pkg/mod \
+              --mount=type=cache,target=/pnpm/store \
+              --mount=type=bind,target=/restored \
+              tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
+              tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
+              tar -xf /restored/pnpm-store.tar -C /pnpm/store
+          EOF
+          docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
+      - name: build
+        run: |
+          docker build -f cmd/enterprise/Dockerfile.integration --build-arg TARGETARCH=amd64 --build-arg ZEUSURL=http://zeus:8080 .
+          docker build -f cmd/enterprise/Dockerfile.with-web.integration --build-arg TARGETARCH=amd64 --build-arg ZEUSURL=http://zeus:8080 .
+      # docker cp instead of --output type=local (the local exporter stalls on
+      # multi-GB outputs); tarballs instead of raw trees so the host never hits
+      # the permission and symlink semantics that broke docker cp.
+      - name: extract
+        run: |
+          rm -rf "$RUNNER_TEMP/cacheci"
+          mkdir -p "$RUNNER_TEMP/cacheci" "$RUNNER_TEMP/extract-context"
+          cat > "$RUNNER_TEMP/extract.Dockerfile" <<'EOF'
+          FROM busybox:1.37
+          RUN --mount=type=cache,target=/root/.cache/go-build \
+              --mount=type=cache,target=/go/pkg/mod \
+              --mount=type=cache,target=/pnpm/store \
+              mkdir -p /out && \
+              tar -cf /out/go-build.tar -C /root/.cache/go-build . && \
+              tar -cf /out/go-mod.tar -C /go/pkg/mod . && \
+              tar -cf /out/pnpm-store.tar -C /pnpm/store .
+          EOF
+          docker build -f "$RUNNER_TEMP/extract.Dockerfile" -t cacheci-extract "$RUNNER_TEMP/extract-context"
+          id=$(docker create cacheci-extract)
+          docker cp "$id":/out/. "$RUNNER_TEMP/cacheci/"
+          docker rm "$id"
+      # Fixed cache keys are immutable, so each key must be deleted before it
+      # can be saved again. Rotating primary and secondary one after the other
+      # keeps at least one key restorable for concurrent test runs.
+      - name: delete-primary
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: gh cache delete tests-primary --repo "$GITHUB_REPOSITORY" || true
+      - name: save-primary
+        uses: actions/cache/save@v4
+        with:
+          path: ${{ runner.temp }}/cacheci
+          key: tests-primary
+      - name: delete-secondary
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: gh cache delete tests-secondary --repo "$GITHUB_REPOSITORY" || true
+      - name: save-secondary
+        uses: actions/cache/save@v4
+        with:
+          path: ${{ runner.temp }}/cacheci
+          key: tests-secondary
--- a/.github/workflows/e2eci.yaml
+++ b/.github/workflows/e2eci.yaml
@@ -75,6 +75,30 @@ jobs:
          docker rm pw
          echo "PLAYWRIGHT_BROWSERS_PATH=$RUNNER_TEMP/ms-playwright" >> "$GITHUB_ENV"
          cd tests/e2e && pnpm playwright install-deps ${{ matrix.project }}
+      # Restore-only: the cacheci workflow owns cache saves. Seeds the
+      # BuildKit cache mounts so the in-test image build is incremental.
+      - name: restore
+        id: restore
+        uses: actions/cache/restore@v4
+        with:
+          path: ${{ runner.temp }}/cacheci
+          key: tests-primary
+          restore-keys: |
+            tests-secondary
+      - name: inject
+        if: steps.restore.outputs.cache-matched-key != ''
+        run: |
+          cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
+          FROM busybox:1.37
+          RUN --mount=type=cache,target=/root/.cache/go-build \
+              --mount=type=cache,target=/go/pkg/mod \
+              --mount=type=cache,target=/pnpm/store \
+              --mount=type=bind,target=/restored \
+              tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
+              tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
+              tar -xf /restored/pnpm-store.tar -C /pnpm/store
+          EOF
+          docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
      - name: bring-up-stack
        run: |
          cd tests && \
--- a/.github/workflows/integrationci.yaml
+++ b/.github/workflows/integrationci.yaml
@@ -110,6 +110,30 @@ jobs:
          sudo mv chromedriver-linux64/chromedriver /usr/local/bin/chromedriver
          chromedriver -version
          google-chrome-stable --version
+      # Restore-only: the cacheci workflow owns cache saves. Seeds the
+      # BuildKit cache mounts so the in-test image build is incremental.
+      - name: restore
+        id: restore
+        uses: actions/cache/restore@v4
+        with:
+          path: ${{ runner.temp }}/cacheci
+          key: tests-primary
+          restore-keys: |
+            tests-secondary
+      - name: inject
+        if: steps.restore.outputs.cache-matched-key != ''
+        run: |
+          cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
+          FROM busybox:1.37
+          RUN --mount=type=cache,target=/root/.cache/go-build \
+              --mount=type=cache,target=/go/pkg/mod \
+              --mount=type=cache,target=/pnpm/store \
+              --mount=type=bind,target=/restored \
+              tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
+              tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
+              tar -xf /restored/pnpm-store.tar -C /pnpm/store
+          EOF
+          docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
      - name: run
        run: |
          cd tests && \
--- a/4
+++ b/4
@@ -209,8 +209,8 @@ py-lint: ## Run ruff check across the shared tests project
 	@cd tests && uv run ruff check --fix .

 .PHONY: py-test-setup
-py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests
-	@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --capture=no integration/bootstrap/setup.py::test_setup
+py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests, rebuilding signoz from the current sources
+	@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup

 .PHONY: py-test-teardown
 py-test-teardown: ## Tear down the shared SigNoz backend
--- a/cmd/enterprise/Dockerfile.integration
+++ b/cmd/enterprise/Dockerfile.integration
@@ -4,9 +4,13 @@ ARG OS="linux"
 ARG TARGETARCH
 ARG ZEUSURL

+# HOME comes from the build user, not the image config; declare it so the
+# /root paths below trace to it.
+ENV HOME=/root
+
 # This path is important for stacktraces
 WORKDIR $GOPATH/src/github.com/signoz/signoz
-WORKDIR /root
+WORKDIR $HOME

 RUN set -eux; \
 	apt-get update; \
@@ -14,23 +18,36 @@ RUN set -eux; \
 		g++ \
 		gcc \
 		libc6-dev \
-		make \
 		pkg-config \
 	; \
 	rm -rf /var/lib/apt/lists/*

+# Keep the literal cache-mount targets below in sync with these. The caches
+# are shared with Dockerfile.with-web.integration (same target paths).
+ENV GOCACHE=$HOME/.cache/go-build
+ENV GOMODCACHE=$GOPATH/pkg/mod
+
 COPY go.mod go.sum ./

-RUN go mod download
+RUN --mount=type=cache,target=/go/pkg/mod \
+	go mod download

 COPY ./cmd/ ./cmd/
 COPY ./ee/ ./ee/
 COPY ./pkg/ ./pkg/
 COPY ./templates /root/templates

-COPY Makefile Makefile
-RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
-RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
+# Invoked directly instead of via make so Makefile changes don't invalidate
+# this layer; the Makefile's git-derived ldflags resolve to empty in here
+# anyway (.git is dockerignored).
+RUN --mount=type=cache,target=/go/pkg/mod \
+	--mount=type=cache,target=/root/.cache/go-build \
+	GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
+	-ldflags "-s -w \
+		-X github.com/SigNoz/signoz/pkg/version.version=integration \
+		-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
+		-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
+		-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"

 RUN chmod 755 /root /root/signoz

--- a/cmd/enterprise/Dockerfile.with-web.integration
+++ b/cmd/enterprise/Dockerfile.with-web.integration
@@ -1,10 +1,23 @@
 FROM node:22-bookworm AS build

 WORKDIR /opt/
-COPY ./frontend/ ./
+
+# HOME comes from the build user, not the image config.
+ENV HOME=/root
+# pnpm's store lives at $PNPM_HOME/store — a dedicated directory pnpm
+# manages. Keep the literal cache-mount targets below in sync.
+ENV PNPM_HOME=/pnpm
 ENV NODE_OPTIONS=--max-old-space-size=8192
+
 RUN CI=1 npm i -g pnpm@10
-RUN CI=1 pnpm install
+
+# pnpm fetch resolves from the lockfile alone and runs no lifecycle scripts;
+# the repo's postinstall needs source files that are not copied yet.
+COPY ./frontend/package.json ./frontend/pnpm-lock.yaml ./frontend/pnpm-workspace.yaml ./
+RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm fetch
+
+COPY ./frontend/ ./
+RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm install --offline
 RUN CI=1 pnpm build

 FROM golang:1.25-bookworm
@@ -13,9 +26,13 @@ ARG OS="linux"
 ARG TARGETARCH
 ARG ZEUSURL

+# HOME comes from the build user, not the image config; declare it so the
+# /root paths below trace to it.
+ENV HOME=/root
+
 # This path is important for stacktraces
 WORKDIR $GOPATH/src/github.com/signoz/signoz
-WORKDIR /root
+WORKDIR $HOME

 RUN set -eux; \
 	apt-get update; \
@@ -23,23 +40,36 @@ RUN set -eux; \
 		g++ \
 		gcc \
 		libc6-dev \
-		make \
 		pkg-config \
 	; \
 	rm -rf /var/lib/apt/lists/*

+# Keep the literal cache-mount targets below in sync with these. The caches
+# are shared with Dockerfile.integration (same target paths).
+ENV GOCACHE=$HOME/.cache/go-build
+ENV GOMODCACHE=$GOPATH/pkg/mod
+
 COPY go.mod go.sum ./

-RUN go mod download
+RUN --mount=type=cache,target=/go/pkg/mod \
+	go mod download

 COPY ./cmd/ ./cmd/
 COPY ./ee/ ./ee/
 COPY ./pkg/ ./pkg/
 COPY ./templates /root/templates

-COPY Makefile Makefile
-RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
-RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
+# Invoked directly instead of via make so Makefile changes don't invalidate
+# this layer; the Makefile's git-derived ldflags resolve to empty in here
+# anyway (.git is dockerignored).
+RUN --mount=type=cache,target=/go/pkg/mod \
+	--mount=type=cache,target=/root/.cache/go-build \
+	GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
+	-ldflags "-s -w \
+		-X github.com/SigNoz/signoz/pkg/version.version=integration \
+		-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
+		-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
+		-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"

 COPY --from=build /opt/build ./web/

--- a/docs/contributing/tests/e2e.md
+++ b/docs/contributing/tests/e2e.md
@@ -30,11 +30,11 @@ yarn install:browsers   # one-time Playwright browser install

 ### Starting the Test Environment

-To spin up the backend stack (SigNoz, ClickHouse, Postgres, Zookeeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
+To spin up the backend stack (SigNoz, ClickHouse, Postgres, ClickHouse Keeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:

 ```bash
 cd tests
-uv run pytest --basetemp=./tmp/ -vv --reuse --with-web \
+uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --with-web \
  e2e/bootstrap/setup.py::test_setup
 ```

@@ -45,8 +45,13 @@ This command will:
 - Start the HTTP seeder container (`tests/seeder/` — exposing `/telemetry/{traces,logs,metrics}` POST + DELETE)
 - Write backend coordinates to `tests/e2e/.env.local` (loaded by `playwright.config.ts` via dotenv)
 - Keep containers running via the `--reuse` flag
+- Rebuild the SigNoz container from the current sources via the `--rebuild` flag

-The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start.
+The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start; later builds are incremental.
+
+### Rebuilding After Source Changes
+
+The `--with-web` image bakes the built frontend in, so neither backend nor frontend changes are picked up while `--reuse` keeps the container running. `--rebuild` fixes that for both: it kills the SigNoz container, rebuilds the image incrementally (go build cache + pnpm store — a frontend-only change rebuilds in about a minute), and starts a fresh one while databases, mocks, migrations, and the seeder stay reused. The setup command above passes it, so the iteration loop is: change code → re-run the setup command → re-run your specs. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.

 ### Stopping the Test Environment

@@ -281,13 +286,16 @@ The full `playwright.config.ts` is the source of truth. Common things to tweak:
 The same pytest flags integration tests expose work here, since E2E reuses the shared fixture graph:

 - `--reuse` — keep containers warm between runs (required for all iteration).
+- `--rebuild` — recreate the SigNoz container from the current sources (backend and, with `--with-web`, frontend) while the rest of the stack stays up. Requires `--reuse`.
 - `--teardown` — tear everything down.
+- `--clean` — prune the docker build caches, forcing the next image build to start cold.
 - `--with-web` — build the frontend into the SigNoz container. **Required for E2E**; integration tests don't need it.
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/integration.md`.
+- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/tests/integration.md`.

 ## What should I remember?

- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build; you only want to pay that once.
+- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build on a cold start; later builds are incremental.
+- **Changed backend or frontend code? Re-run the setup command** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
 - **Don't teardown before setup.** `--reuse` correctly handles partially-set-up state, so chaining teardown → setup wastes time.
 - **Prefer UI-driven flows.** Playwright captures BE requests in the trace; a parallel `fetch` probe is almost always redundant. Drop to `page.request.*` only when the UI can't reach what you need.
 - **Use `page.waitForResponse` on UI clicks** to assert BE contracts — it still exercises the UI trigger path.
--- a/docs/contributing/tests/integration.md
+++ b/docs/contributing/tests/integration.md
@@ -37,13 +37,34 @@ make py-test-setup
 Under the hood this runs, from `tests/`:

 ```bash
-uv run pytest --basetemp=./tmp/ -vv --reuse integration/bootstrap/setup.py::test_setup
+uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup
 ```

 This command will:
- Start all required services (ClickHouse, PostgreSQL, Zookeeper, SigNoz, Zeus mock, gateway mock)
+- Start all required services (ClickHouse, PostgreSQL, ClickHouse Keeper, SigNoz, Zeus mock, gateway mock)
 - Register an admin user
 - Keep containers running via the `--reuse` flag
+- Rebuild the SigNoz container from the current sources via the `--rebuild` flag
+
+### Rebuilding After Source Changes
+
+`--reuse` keeps the running SigNoz container, which means backend source changes are not picked up. `--rebuild` fixes exactly that: it kills the existing SigNoz container, rebuilds the image (incremental — only changed packages recompile thanks to the build cache), and starts a fresh one, while everything else (databases, mocks, migrations) stays reused. `make py-test-setup` passes it by default, so the iteration loop is simply:
+
+```bash
+make py-test-setup      # (re)build signoz from your current sources
+uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
+# ... edit backend code or tests ...
+make py-test-setup      # pick up the backend changes
+uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
+```
+
+The same applies to the e2e stack. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.
+
+Some suites define their own SigNoz variant in a suite-local `conftest.py` (`create_signoz(..., cache_key=...)` — e.g. `basepath`, `metricreduction`, `querier_json_body`). Those containers are not touched by `make py-test-setup`, which only rebuilds the default instance. For such suites, pass `--rebuild` on the suite run itself — it rebuilds every SigNoz variant the run instantiates:
+
+```bash
+uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild integration/tests/<suite>/
+```

 ### Stopping the Test Environment

@@ -56,11 +77,21 @@ make py-test-teardown
 Which runs:

 ```bash
-uv run pytest --basetemp=./tmp/ -vv --teardown integration/bootstrap/setup.py::test_teardown
+uv run pytest --basetemp=./tmp/ -vv --teardown --capture=no integration/bootstrap/setup.py::test_teardown
 ```

 This destroys the running integration test setup and cleans up resources.

+### Cleaning the Image Build Cache
+
+The `signoz:integration` image build keeps its Go build and module caches in BuildKit cache mounts, so rebuilds only recompile what changed. These caches survive `--teardown` (they belong to the Docker builder, not to any container). If a cache ever needs to be nuked — suspected corruption, disk pressure, or to force a genuinely cold build — pass the `--clean` flag:
+
+```bash
+uv run pytest --basetemp=./tmp/ -vv --teardown --clean integration/bootstrap/setup.py::test_teardown
+```
+
+`--clean` prunes the docker build artifacts backing the incremental image build at session start, so the next build starts from a clean slate. Images and regular layer cache stay intact, but note the pruning is host-wide — it clears build caches for other projects too, not just SigNoz's. The flag composes with any invocation — passing it on a normal `--reuse` run simply makes the next image build start cold (~3–4 minutes instead of seconds).
+
 ## Understanding the Integration Test Framework

 Python and pytest form the foundation of the integration testing framework. Testcontainers are used to spin up disposable integration environments. WireMock is used to spin up **test doubles** of external services (Zeus cloud API, gateway, etc.).
@@ -99,7 +130,7 @@ tests/
 │       ├── passwordauthn/
 │       ├── querier/
 │       └── ...
-└── e2e/                     # Playwright suite (see docs/contributing/e2e.md)
+└── e2e/                     # Playwright suite (see docs/contributing/tests/e2e.md)
 ```

 Each test suite follows these principles:
@@ -224,9 +255,9 @@ Tests can be configured using pytest options:
 - `--sqlstore-provider` — Choose the SQL store provider (default: `postgres`)
 - `--sqlite-mode` — SQLite journal mode: `delete` or `wal` (default: `delete`). Only relevant when `--sqlstore-provider=sqlite`.
 - `--postgres-version` — PostgreSQL version (default: `15`)
- `--clickhouse-version` — ClickHouse version (default: `25.5.6`)
- `--zookeeper-version` — Zookeeper version (default: `3.7.1`)
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.2`)
+- `--clickhouse-version` — ClickHouse version, also used for ClickHouse Keeper (default: `25.12.5`)
+- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.6`)
+- `--with-web` — Build the frontend into the SigNoz image (required for e2e)

 Example:

@@ -239,6 +270,7 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
 ## What should I remember?

 - **Always use the `--reuse` flag** when setting up the environment or running tests to keep containers warm. Without it every run rebuilds the stack (~4 mins).
+- **Changed backend code? Re-run `make py-test-setup`** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
 - **Use the `--teardown` flag** only when cleaning up — mixing `--teardown` with `--reuse` is a contradiction.
 - **Do not pre-emptively teardown before setup.** If the stack is partially up, `--reuse` picks up from wherever it is. `make py-test-teardown` then `make py-test-setup` wastes minutes.
 - **Follow the naming convention** with two-digit numeric prefixes (`01_`, `02_`) for ordered test execution within a suite.
@@ -247,5 +279,5 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
 - **Use descriptive test names** that clearly indicate what is being tested.
 - **Leverage fixtures** for common setup. The shared fixture package is at `tests/fixtures/` — reuse before adding new ones.
 - **Test both success and failure scenarios** (4xx / 5xx paths) to ensure robust functionality.
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — black + isort + autoflake + pylint.
+- **Run `make py-fmt` and `make py-lint` before committing** Python changes — ruff format + ruff check.
 - **`--sqlite-mode=wal` does not work on macOS.** The integration test environment runs SigNoz inside a Linux container with the SQLite database file mounted from the macOS host. WAL mode requires shared memory between connections, and connections crossing the VM boundary (macOS host ↔ Linux container) cannot share the WAL index, resulting in `SQLITE_IOERR_SHORT_READ`. WAL mode is tested in CI on Linux only.
--- a/go.mod
+++ b/go.mod
@@ -4,7 +4,7 @@ go 1.25.7

 require (
 	dario.cat/mergo v1.0.2
-	github.com/AfterShip/clickhouse-sql-parser v0.5.3
+	github.com/AfterShip/clickhouse-sql-parser v0.5.4
 	github.com/ClickHouse/clickhouse-go/v2 v2.44.0
 	github.com/DATA-DOG/go-sqlmock v1.5.2
 	github.com/SigNoz/clickhouse-go-mock v0.14.0
--- a/go.sum
+++ b/go.sum
@@ -66,8 +66,8 @@ dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
 filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
 filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
-github.com/AfterShip/clickhouse-sql-parser v0.5.3 h1:6iap8XGjuSjD3w7r1UNrg66ljBugcv2P39s4eo/ZLRw=
-github.com/AfterShip/clickhouse-sql-parser v0.5.3/go.mod h1:Qi3qvPTfZb/aFwI5V4WFOahgjsLJa4MzVijIAfwOhDw=
+github.com/AfterShip/clickhouse-sql-parser v0.5.4 h1:yiCQaMq8EO+dpKdnpP9YYd/ne6MSuOXgsMsNL33NiTI=
+github.com/AfterShip/clickhouse-sql-parser v0.5.4/go.mod h1:Qi3qvPTfZb/aFwI5V4WFOahgjsLJa4MzVijIAfwOhDw=
 github.com/Azure/azure-sdk-for-go v68.0.0+incompatible h1:fcYLmCpyNYRnvJbPerq7U0hS+6+I79yEDJBqVNcqUzU=
 github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0 h1:fou+2+WFTib47nS+nz/ozhEBnvU96bKHy6LjRsY4E28=
 github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0/go.mod h1:t76Ruy8AHvUAC8GfMWJMa0ElSbuIcO03NLpynfbgsPA=
--- a/pkg/querybuilder/clickhouse_sql.go
+++ b/pkg/querybuilder/clickhouse_sql.go
@@ -3,6 +3,8 @@ package querybuilder
 import (
 	"context"
 	"log/slog"
+	"maps"
+	"slices"
 	"strings"

 	chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
@@ -25,7 +27,23 @@ var internalDatabases = map[string]struct{}{
 	"information_schema": {},
 }

-// The parser's grammar has gaps against SQL that ClickHouse itself accepts. See TestErrIfStatementIsNotValid_ShouldPassButFails.
+// generatorTableFunctions compute their rows from their arguments alone. They open no file or socket, reach no other host, and name no table, database or dictionary, so none of them can read through anything the rules here exist to protect. Can be used to build a dense axis to join a sparse series against. Every other table function is refused.
+//
+// Keyed by the lowercased name so that matching is case-insensitive, valued by the spelling to name it back to the caller.
+//
+// TODO(@therealpandey): take a deployment level allow list on top of this, so an operator can permit more without a release.
+var generatorTableFunctions = map[string]string{
+	"numbers":         "numbers",
+	"numbers_mt":      "numbers_mt",
+	"zeros":           "zeros",
+	"zeros_mt":        "zeros_mt",
+	"generateseries":  "generateSeries",
+	"generate_series": "generate_series",
+}
+
+var generatorTableFunctionsMessage = "allowed table functions are " + strings.Join(slices.Sorted(maps.Values(generatorTableFunctions)), ", ")
+
+// The parser's grammar has gaps against SQL that ClickHouse itself accepts.
 func ErrIfStatementIsNotValid(query string) (err error) {
 	defer func() {
 		// The parser has a history of panicking on malformed input rather than returning an error.
@@ -52,8 +70,17 @@ func ErrIfStatementIsNotValid(query string) (err error) {
 	visitor := &chparser.DefaultASTVisitor{Visit: func(node chparser.Expr) error {
 		switch expr := node.(type) {
 		case *chparser.TableFunctionExpr:
-			// Source table functions remain usable in ClickHouse read-only mode.
-			return errors.NewInvalidInputf(CodeClickHouseSQLTableFunction, "ClickHouse table functions are not allowed in SQL queries: %s", chparser.Format(expr.Name))
+			// Source table functions remain usable in ClickHouse read-only mode. Arguments are
+			// visited before this, so a read smuggled into one is already refused by the time
+			// an allowed generator gets here.
+			name := chparser.Format(expr.Name)
+			if _, ok := generatorTableFunctions[strings.ToLower(name)]; ok {
+				return nil
+			}
+
+			return errors.
+				NewInvalidInputf(CodeClickHouseSQLTableFunction, "ClickHouse table functions are not allowed in SQL queries: %s", name).
+				WithAdditional(generatorTableFunctionsMessage)

 		case *chparser.TableIdentifier:
 			// Reading these is unaffected by ClickHouse read-only mode.
--- a/pkg/querybuilder/clickhouse_sql_test.go
+++ b/pkg/querybuilder/clickhouse_sql_test.go
@@ -2,12 +2,11 @@ package querybuilder

 import (
 	"testing"
+	"time"

 	"github.com/SigNoz/signoz/pkg/errors"

-	chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
 	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
 )

 func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
@@ -21,6 +20,9 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
 		{"CommonTableExpression", "WITH t AS (SELECT fingerprint FROM signoz_metrics.time_series_v4) SELECT * FROM t"},
 		{"Join", "SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.b"},
 		{"GlobalIn", "SELECT a FROM t WHERE a GLOBAL IN (SELECT b FROM t2)"},
+		// GLOBAL parsed only when the join type was omitted, and only before IN. https://github.com/AfterShip/clickhouse-sql-parser/pull/293
+		{"GlobalLeftJoin", "SELECT * FROM t1 GLOBAL LEFT JOIN t2 ON t1.a = t2.a"},
+		{"GlobalNotIn", "SELECT a FROM t WHERE a GLOBAL NOT IN (SELECT b FROM t2)"},
 		{"Union", "SELECT * FROM t UNION ALL SELECT * FROM t2"},
 		{"Intersect", "SELECT * FROM t INTERSECT SELECT * FROM t2"},
 		{"WindowFunction", "SELECT sum(v) OVER (PARTITION BY a ORDER BY t) FROM t"},
@@ -36,18 +38,50 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
 		// order by interval
 		{"OrderByInterval", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval ORDER BY interval"},
 		{"OrderByIntervalAndDirection", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS `interval` ORDER BY `interval` ASC"},
-		// Unspaced, so rejected until the parser stopped lexing a signed literal after a
-		// closing bracket. The spaced form above no longer needs to be spaced.
-		// https://github.com/AfterShip/clickhouse-sql-parser/issues/286
+		// `interval` is a unit keyword, so unquoting it was rejected everywhere the parser
+		// expected a plain identifier. https://github.com/AfterShip/clickhouse-sql-parser/pull/296
+		{"OrderByUnquotedIntervalAsc", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval FROM t GROUP BY interval ORDER BY interval ASC"},
+		{"OrderByUnquotedIntervalDesc", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval FROM t GROUP BY interval ORDER BY interval DESC"},
+		{"UnquotedIntervalInGroupByTuple", "SELECT a FROM t GROUP BY (`service.name`, `service.version`, interval)"},
+		{"UnquotedIntervalProductionQuery", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, resource_string_service$$name AS `service.name`, attributes_string['http.route'] AS `http.route`, quantile(0.95)(duration_nano) / 1000000000 AS value FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_string_service$$name = 'svc-a' AND resources_string['deployment.environment'] = 'dev' AND attributes_string['http.route'] = '/v1' AND http_method = 'POST' AND timestamp BETWEEN toDateTime(1784601720) AND toDateTime(1784602620) AND ts_bucket_start BETWEEN 1784601720 - 1800 AND 1784602620 GROUP BY `service.name`, `http.route`, interval ORDER BY interval ASC"},
+		// Separating the two readings of INTERVAL needs backtracking as per the current implementation which could have performance regressions.
+		// https://github.com/AfterShip/clickhouse-sql-parser/pull/296#issuecomment-5150316367
+		{"UnquotedIntervalRepeatedThirtyTimes", "SELECT interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval AS total FROM t WHERE interval > 0 ORDER BY interval ASC"},
 		{"SignedLiteralAfterClosingParenUnspaced", "SELECT now() AS ts, toFloat64(count()) AS value FROM ( SELECT attributes_string['TableName'] AS T, attributes_string['MissingId'] AS M, max(fromUnixTimestamp64Nano(timestamp)) AS last_seen, dateDiff('minute', min(fromUnixTimestamp64Nano(timestamp)), max(fromUnixTimestamp64Nano(timestamp))) AS age_min FROM signoz_logs.distributed_logs_v2 WHERE body='missing_map_record' AND timestamp >= (toUnixTimestamp(now())-3600)*1000000000 GROUP BY T, M ) WHERE age_min >= 20 AND last_seen >= now() - toIntervalMinute(8)"},
 		{"SignedLiteralAfterClosingParenMinimal", "SELECT (1)-1"},
 		{"TrimFunction", "SELECT trimBoth('/api/endpoint/', '/');"},
+		// The SQL-standard keyword-separated argument forms, which took commas only. https://github.com/AfterShip/clickhouse-sql-parser/pull/290
+		{"StandardTrimSyntax", "SELECT trim(BOTH ' ' FROM body) FROM t"},
+		{"StandardSubstringSyntax", "SELECT substring(body FROM 2 FOR 3) FROM t"},
+		{"StandardOverlaySyntax", "SELECT overlay(body PLACING 'x' FROM 2) FROM t"},
+		// Row generators compute their rows from their arguments, so they read through nothing. This is the shape they get used for: a dense interval axis to CROSS JOIN a sparse series against.
+		{"NumbersTableFunction", "SELECT intervals.interval AS interval, active.cluster AS cluster, toFloat64(if(ts_data.has_data = 0, 0, 1)) AS value FROM ( SELECT DISTINCT JSONExtractString(labels, 'k8s.cluster.name') AS cluster FROM signoz_metrics.distributed_time_series_v4 WHERE metric_name = 'my_metric' AND unix_milli >= toUnixTimestamp(now() - INTERVAL 30 DAY) * 1000 HAVING cluster != '' ) AS active CROSS JOIN ( SELECT toStartOfInterval( toDateTime(toUnixTimestamp(now() - INTERVAL 30 MINUTE) + number * 60), INTERVAL 1 MINUTE ) AS interval FROM numbers(31) ) AS intervals LEFT JOIN ( SELECT toStartOfInterval( toDateTime(intDiv(s.unix_milli, 1000)), INTERVAL 1 MINUTE ) AS interval, JSONExtractString(ts.labels, 'k8s.cluster.name') AS cluster, 1 AS has_data FROM signoz_metrics.distributed_samples_v4 s INNER JOIN ( SELECT DISTINCT fingerprint, labels FROM signoz_metrics.distributed_time_series_v4 WHERE metric_name = 'my_metric' ) AS ts ON s.fingerprint = ts.fingerprint WHERE s.metric_name = 'my_metric' AND s.unix_milli >= toUnixTimestamp(now() - INTERVAL 30 MINUTE) * 1000 GROUP BY interval, cluster ) AS ts_data ON active.cluster = ts_data.cluster AND intervals.interval = ts_data.interval ORDER BY interval ASC"},
+		{"NumbersMtTableFunction", "SELECT * FROM numbers_mt(31)"},
+		{"ZerosTableFunction", "SELECT * FROM zeros(31)"},
+		{"ZerosMtTableFunction", "SELECT * FROM zeros_mt(31)"},
+		{"GenerateSeriesTableFunction", "SELECT * FROM generateSeries(1, 10)"},
+		{"GenerateSeriesSnakeCaseTableFunction", "SELECT * FROM generate_series(1, 10)"},
+		{"GeneratorTableFunctionUppercase", "SELECT * FROM NUMBERS(31)"},
+		{"GeneratorTableFunctionParenthesisedArgument", "SELECT * FROM NUMBERS((31))"},
+		{"GeneratorTableFunctionInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l CROSS JOIN numbers(31) AS n"},
+		{"GeneratorTableFunctionInCommonTableExpression", "WITH axis AS (SELECT number FROM numbers(31)) SELECT * FROM axis"},
+		{"GeneratorTableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT number FROM numbers(31))"},
+		{"GeneratorTableFunctionInUnion", "SELECT number FROM numbers(31) UNION ALL SELECT number FROM zeros(31)"},
 	}

 	for _, testCase := range testCases {
 		t.Run(testCase.name, func(t *testing.T) {
-			err := ErrIfStatementIsNotValid(testCase.query)
-			assert.NoError(t, err)
+			// Bounded rather than called directly: a parser that backtracks without memoising
+			// hangs instead of returning. Every case here parses in well under a millisecond.
+			errC := make(chan error, 1)
+			go func() { errC <- ErrIfStatementIsNotValid(testCase.query) }()
+
+			select {
+			case err := <-errC:
+				assert.NoError(t, err)
+			case <-time.After(10 * time.Second):
+				assert.Fail(t, "timed out, which means the parser is no longer bounding its backtracking")
+			}
 		})
 	}
 }
@@ -70,6 +104,8 @@ func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
 		{"CreateTable", "CREATE TABLE evil (a Int) ENGINE = Memory", CodeClickHouseSQLNotSelect},
 		{"Grant", "GRANT ALL ON *.* TO admin", CodeClickHouseSQLNotSelect},
 		{"Set", "SET readonly = 0", CodeClickHouseSQLNotSelect},
+		// The parser still dereferences nil on a DEFAULT expression it cannot read, so the recover is what turns this into a rejection rather than a crash.
+		{"UnparseableDefaultExpression", "CREATE TABLE t (a String DEFAULT foo(b FROM 2)) ENGINE = Memory", CodeClickHouseSQLParserPanic},
 		// These the parser rejects outright rather than classifying.
 		{"ShowGrants", "SHOW GRANTS", CodeClickHouseSQLUnparseable},
 		{"IntoOutfile", "SELECT * FROM t INTO OUTFILE '/tmp/x.csv'", CodeClickHouseSQLUnparseable},
@@ -81,6 +117,20 @@ func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
 		{"TableFunctionInCommonTableExpression", "WITH c AS (SELECT * FROM url('http://x', CSV, 'a String')) SELECT * FROM c", CodeClickHouseSQLTableFunction},
 		{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM file('/etc/passwd', CSV, 'a String'))", CodeClickHouseSQLTableFunction},
 		{"TableFunctionInUnion", "SELECT * FROM t UNION ALL SELECT * FROM url('http://x', CSV, 'a String')", CodeClickHouseSQLTableFunction},
+		// These reach the internal databases without ever naming one, so the table-function rule is the only thing that sees them.
+		{"MergeTableFunction", "SELECT * FROM merge('system', '.*')", CodeClickHouseSQLTableFunction},
+		{"RemoteTableFunction", "SELECT * FROM remote('other-host', 'system.users')", CodeClickHouseSQLTableFunction},
+		{"ClusterTableFunction", "SELECT * FROM cluster('c', 'system.users')", CodeClickHouseSQLTableFunction},
+		// Pure, but excluded: generateRandom streams rows the arguments do not bound, and values has no use here that an array literal does not already cover.
+		{"GenerateRandomTableFunction", "SELECT * FROM generateRandom('a UInt64')", CodeClickHouseSQLTableFunction},
+		{"ValuesTableFunction", "SELECT * FROM values('a UInt64', 1, 2)", CodeClickHouseSQLTableFunction},
+		// Arguments are visited before the table function itself, so allowing a generator does not give anyone a wrapper to smuggle a read through.
+		{"InternalDatabaseInsideAllowedTableFunction", "SELECT * FROM numbers((SELECT count() FROM system.users))", CodeClickHouseSQLInternalDatabase},
+		{"InternalDatabaseJoinedOntoAllowedTableFunction", "SELECT * FROM numbers(31) AS n JOIN system.users AS u ON 1 = 1", CodeClickHouseSQLInternalDatabase},
+		{"InternalDatabaseUnionedWithAllowedTableFunction", "SELECT number FROM numbers(31) UNION ALL SELECT name FROM system.users", CodeClickHouseSQLInternalDatabase},
+		{"RefusedTableFunctionJoinedOntoAllowedTableFunction", "SELECT * FROM numbers(31) AS n JOIN url('http://x', CSV, 'a String') AS u ON 1 = 1", CodeClickHouseSQLTableFunction},
+		{"RefusedTableFunctionInsideAllowedTableFunction", "SELECT * FROM numbers((SELECT count() FROM file('/etc/passwd', CSV, 'a String')))", CodeClickHouseSQLTableFunction},
+		{"InternalDatabaseInsideAllowedTableFunctionCommonTableExpression", "WITH axis AS (SELECT * FROM numbers((SELECT count() FROM system.users))) SELECT * FROM axis", CodeClickHouseSQLInternalDatabase},
 		// Internal databases, which hold grants and server metadata rather than telemetry.
 		{"SystemUsers", "SELECT * FROM system.users", CodeClickHouseSQLInternalDatabase},
 		{"SystemUppercase", "SELECT * FROM SYSTEM.USERS", CodeClickHouseSQLInternalDatabase},
@@ -103,50 +153,3 @@ func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
 		})
 	}
 }
-
-// Queries the parser cannot read. ClickHouse runs all of them.
-func TestErrIfStatementIsNotValid_ShouldPassButFails(t *testing.T) {
-	testCases := []struct {
-		name  string
-		query string
-		// The construct the parser stops after, which is the one it cannot read.
-		expectedStopsAfter string
-		// The same construct written so the parser accepts it.
-		fix string
-	}{
-		{
-			name:               "IntervalAliasInOrderBy",
-			query:              "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, resource_string_service$$name AS `service.name`, attributes_string['http.route'] AS `http.route`, quantile(0.95)(duration_nano) / 1000000000 AS value FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_string_service$$name = 'svc-a' AND resources_string['deployment.environment'] = 'dev' AND attributes_string['http.route'] = '/v1' AND http_method = 'POST' AND timestamp BETWEEN toDateTime(1784601720) AND toDateTime(1784602620) AND ts_bucket_start BETWEEN 1784601720 - 1800 AND 1784602620 GROUP BY `service.name`, `http.route`, interval ORDER BY interval ASC",
-			expectedStopsAfter: "ORDER BY interval ASC",
-			fix:                "SELECT count() AS interval FROM t ORDER BY `interval` ASC",
-		},
-		{
-			name:               "IntervalAliasInOrderByDesc",
-			query:              "SELECT count() AS value, toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, serviceName, resourceTagsMap['deployment.environment'] AS environment, exceptionStacktrace FROM signoz_traces.distributed_signoz_error_index_v2 WHERE exceptionType != 'OSError' AND resourceTagsMap['deployment.environment'] = 'staging' AND timestamp BETWEEN toDateTime(1785186300) AND toDateTime(1785186600) GROUP BY serviceName, interval, environment, exceptionStacktrace ORDER BY interval DESC",
-			expectedStopsAfter: "ORDER BY interval DESC",
-			fix:                "SELECT count() AS interval FROM t ORDER BY `interval` DESC",
-		},
-		{
-			name:               "StandardTrimSyntax",
-			query:              "SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 5 MINUTE) AS interval, resources_string['host.name'] as host_name, toFloat64(countIf( lower(trim(BOTH ' ' FROM replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ))) IN ('unhealthy','starting','failing') )) as value FROM signoz_logs.distributed_logs_v2 WHERE timestamp BETWEEN 1784602320000000000 AND 1784602620000000000 AND ts_bucket_start BETWEEN 1784602320 - 300 AND 1784602620 AND JSONExtractString(body, 'Type') = 'container' AND JSONExtractString(body, 'Actor', 'Attributes', 'name') IS NOT NULL AND resources_string['host.name'] IS NOT NULL AND resources_string['host.name'] = 'aihub-nightly' GROUP BY interval, host_name ORDER BY interval, host_name",
-			expectedStopsAfter: "trim(BOTH '",
-			fix:                "SELECT trimBoth(replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ), ' ')",
-		},
-	}
-
-	for _, testCase := range testCases {
-		t.Run(testCase.name, func(t *testing.T) {
-			err := ErrIfStatementIsNotValid(testCase.query)
-
-			var parseErr *chparser.ParseError
-			require.ErrorAs(t, err, &parseErr, "expected a parser failure rather than a rule violation")
-
-			// The parser reports the offset it stopped at, which sits just past the construct
-			// it choked on, so the text leading up to it is what needs looking at.
-			consumed := testCase.query[:parseErr.Pos]
-			assert.Equal(t, testCase.expectedStopsAfter, consumed[max(0, len(consumed)-len(testCase.expectedStopsAfter)):])
-
-			assert.NoError(t, ErrIfStatementIsNotValid(testCase.fix))
-		})
-	}
-}
--- a/scripts/promqltestcorpus/go.mod
+++ b/scripts/promqltestcorpus/go.mod
@@ -76,14 +76,14 @@ require (
 	go.uber.org/atomic v1.11.0 // indirect
 	go.uber.org/goleak v1.3.0 // indirect
 	go.yaml.in/yaml/v2 v2.4.4 // indirect
-	golang.org/x/crypto v0.50.0 // indirect
+	golang.org/x/crypto v0.52.0 // indirect
 	golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa // indirect
-	golang.org/x/net v0.53.0 // indirect
+	golang.org/x/net v0.55.0 // indirect
 	golang.org/x/oauth2 v0.36.0 // indirect
 	golang.org/x/sync v0.20.0 // indirect
-	golang.org/x/sys v0.43.0 // indirect
-	golang.org/x/term v0.42.0 // indirect
-	golang.org/x/text v0.36.0 // indirect
+	golang.org/x/sys v0.45.0 // indirect
+	golang.org/x/term v0.43.0 // indirect
+	golang.org/x/text v0.37.0 // indirect
 	golang.org/x/time v0.15.0 // indirect
 	google.golang.org/api v0.272.0 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20260414002931-afd174a4e478 // indirect
--- a/scripts/promqltestcorpus/go.sum
+++ b/scripts/promqltestcorpus/go.sum
@@ -377,29 +377,29 @@ go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
 go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
 go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
-golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
-golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
+golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
+golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
 golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa h1:Zt3DZoOFFYkKhDT3v7Lm9FDMEV06GpzjG2jrqW+QTE0=
 golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa/go.mod h1:K79w1Vqn7PoiZn+TkNpx3BUWUQksGO3JcVX6qIjytmA=
-golang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI=
-golang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY=
-golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
-golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
+golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
+golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
+golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
+golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
 golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
 golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
 golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
 golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
 golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
-golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
-golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
-golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
-golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
-golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
+golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
+golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
+golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
+golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
+golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
 golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
 golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
-golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
-golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
+golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c=
+golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI=
 gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
 gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
 google.golang.org/api v0.272.0 h1:eLUQZGnAS3OHn31URRf9sAmRk3w2JjMx37d2k8AjJmA=
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,3 +1,5 @@
+import subprocess
+
 import pytest

 pytest_plugins = [
@@ -32,6 +34,22 @@ pytest_plugins = [
 ]


+def pytest_configure(config: pytest.Config):
+    if config.getoption("--rebuild"):
+        if not config.getoption("--reuse"):
+            raise pytest.UsageError("--rebuild requires --reuse: it replaces the signoz container within an environment that is being reused.")
+        if config.getoption("--teardown"):
+            raise pytest.UsageError("--rebuild cannot be combined with --teardown.")
+        if config.getoption("--clean"):
+            raise pytest.UsageError("--rebuild cannot be combined with --clean: --clean forces a cold build, which defeats the purpose of --rebuild.")
+
+
+def pytest_sessionstart(session: pytest.Session):
+    if session.config.getoption("--clean"):
+        # The type filter removes only cache mounts, leaving images and layer cache intact.
+        subprocess.run(["docker", "builder", "prune", "--force", "--filter", "type=exec.cachemount"], check=True)
+
+
 def pytest_addoption(parser: pytest.Parser):
    parser.addoption(
        "--reuse",
@@ -45,6 +63,18 @@ def pytest_addoption(parser: pytest.Parser):
        default=False,
        help="Teardown environment. Run pytest --basetemp=./tmp/ -vv --teardown src/bootstrap/setup::test_teardown to teardown your local dev environment.",
    )
+    parser.addoption(
+        "--rebuild",
+        action="store_true",
+        default=False,
+        help="Rebuild the signoz container from the current sources while reusing the rest of the stack (databases, mocks, migrations). Only meaningful together with --reuse: pytest --basetemp=./tmp/ -vv --reuse --rebuild integration/bootstrap/setup.py::test_setup.",
+    )
+    parser.addoption(
+        "--clean",
+        action="store_true",
+        default=False,
+        help="Prune the BuildKit cache mounts (go build and module caches) used by the signoz image build, forcing the next build to start cold. Combine with --teardown to reset everything: pytest --basetemp=./tmp/ -vv --teardown --clean integration/bootstrap/setup.py::test_teardown.",
+    )
    parser.addoption(
        "--with-web",
        action="store_true",
--- a/tests/fixtures/reuse.py
+++ b/tests/fixtures/reuse.py
@@ -41,6 +41,7 @@ def wrap(  # pylint: disable=too-many-arguments,too-many-positional-arguments
    create: Callable[[], T],
    delete: Callable[[T], None],
    restore: Callable[[dict], T],
+    rebuild: bool = False,
 ) -> T:
    """
    Wraps a resource creation and cleanup process with reuse and teardown options.
@@ -51,6 +52,7 @@ def wrap(  # pylint: disable=too-many-arguments,too-many-positional-arguments
    - create: function to create the resource
    - delete: function to delete the resource
    - restore: function to restore resource from cache
+    - rebuild: under --reuse, delete the cached resource and recreate it instead of restoring it
    """
    resource = empty()

@@ -58,8 +60,13 @@ def wrap(  # pylint: disable=too-many-arguments,too-many-positional-arguments
        existing_resource = pytestconfig.cache.get(key, None)
        if existing_resource:
            assert isinstance(existing_resource, dict)
-            logger.info("Reusing existing %s(%s)", key, existing_resource)
-            return restore(existing_resource)
+            if rebuild:
+                logger.info("Rebuilding %s(%s), removing the existing one", key, existing_resource)
+                delete(restore(existing_resource))
+                pytestconfig.cache.set(key, None)
+            else:
+                logger.info("Reusing existing %s(%s)", key, existing_resource)
+                return restore(existing_resource)

    if not teardown(request):
        resource = create()
--- a/tests/fixtures/signoz.py
+++ b/tests/fixtures/signoz.py
@@ -1,4 +1,6 @@
+import os
 import platform
+import subprocess
 import time
 from http import HTTPStatus
 from os import path
@@ -8,7 +10,6 @@ import docker.errors
 import pytest
 import requests
 from testcontainers.core.container import DockerContainer, Network
-from testcontainers.core.image import DockerImage

 from fixtures import reuse, types
 from fixtures.logger import setup_logger
@@ -50,17 +51,27 @@ def create_signoz(

        # Docker build context is the repo root — one up from pytest's
        # rootdir (tests/).
-        self = DockerImage(
-            path=str(pytestconfig.rootpath.parent),
-            dockerfile_path=dockerfile_path,
-            tag="signoz:integration",
-            buildargs={
-                "TARGETARCH": arch,
-                "ZEUSURL": zeus.container_configs["8080"].base(),
-            },
-        )
+        context = pytestconfig.rootpath.parent

-        self.build()
+        # The docker CLI is required: the Dockerfiles use BuildKit cache
+        # mounts, which docker-py does not support.
+        subprocess.run(
+            [
+                "docker",
+                "build",
+                "--file",
+                str(context / dockerfile_path),
+                "--tag",
+                "signoz:integration",
+                "--build-arg",
+                f"TARGETARCH={arch}",
+                "--build-arg",
+                f"ZEUSURL={zeus.container_configs['8080'].base()}",
+                str(context),
+            ],
+            check=True,
+            env=os.environ | {"DOCKER_BUILDKIT": "1"},
+        )

        env = (
            {
@@ -200,6 +211,7 @@ def create_signoz(
        create=create,
        delete=delete,
        restore=restore,
+        rebuild=pytestconfig.getoption("--rebuild"),
    )
Author	SHA1	Message	Date
grandwizard28	4b1b2ec59c	refactor(querybuilder): derive the allowed table function message from the map Values the map by the spelling to name back to the caller, so the message comes off the same data the lookup uses and cannot drift from it. Drops the test that existed only to catch that drift.	2026-08-02 17:32:19 +05:30
grandwizard28	c0a4ec6d41	test(querybuilder): address review on the generator allow list Names the allowed table functions in the rejection error, collapses the added comments, and covers the generators in JOIN, CTE, subquery and UNION position alongside the reads they must not be usable to smuggle.	2026-08-02 17:32:19 +05:30
grandwizard28	d744ff6d5c	feat(querybuilder): allow row-generator table functions The table-function rule refuses everything, which costs a false positive on queries that use numbers() or generateSeries() to build a dense interval axis to join a sparse series against. Neither reads through anything: they compute their rows from their arguments, open no file or socket, reach no other host, and name no table, database or dictionary. Everything else stays refused. Most table functions read through something, and merge('system', '.*'), remote() and cluster() reach the internal databases without producing a TableIdentifier for the database rule to catch, so this rule is all that sees them. generateRandom is pure but streams rows its arguments do not bound, so it stays out too. Arguments are visited before the table function itself, which is what keeps an allowed generator from being usable as a wrapper to smuggle a read through.	2026-08-02 17:32:18 +05:30
Pandey	b1b668925e	chore(deps): bump clickhouse-sql-parser to v0.5.4 (#12375 ) Some checks are pending build-staging / prepare (push) Waiting to run Details build-staging / js-build (push) Blocked by required conditions Details build-staging / go-build (push) Blocked by required conditions Details build-staging / staging (push) Blocked by required conditions Details cacheci / tests (push) Waiting to run Details Release Drafter / update_release_draft (push) Waiting to run Details * chore(deps): bump clickhouse-sql-parser to v0.5.4 v0.5.4 carries the four SigNoz-reported grammar fixes: unquoted `interval` as a column name, GLOBAL before a join type and GLOBAL NOT IN, arithmetic inside table function arguments, and the SQL-standard keyword-argument forms of trim/substring/overlay. Replaying the saved production corpus takes the v5 clickhouse_sql rejection rate from 11/198 to 1/198, and the one remaining rejection is a real rule hit rather than a parser gap. TestErrIfStatementIsNotValid_ShouldPassButFails therefore has nothing left to hold; its queries move to _Pass as regression canaries. * test(querybuilder): bound how long a valid statement may take to parse Telling an INTERVAL operator from a column named interval needs backtracking, and v0.5.4 keeps that from going exponential by remembering the offsets it has already failed at. Losing that memoisation would hang the parser rather than fail it, so a 30-term case covers the shape and the loop bounds how long any valid statement may take. * test(querybuilder): address review on the parser bump Collapse the added comments to fewer lines, and fail the timeout branch with assert rather than t.Fatal so a case that does hang reports alongside the rest. * test(querybuilder): drop the stale signed-literal comment * test(querybuilder): pin the numbers table function false positive Restores TestErrIfStatementIsNotValid_ShouldPassButFails around the one rejection the saved production corpus still produces on v0.5.4. numbers() generates rows rather than reading through anything, so this is the blanket table-function rule being stricter than the threat it exists for rather than a parser gap, and the table now asserts the error code. * test(querybuilder): trim the numbers case comment and query Collapses the query to a single line to match the other cases in the file and swaps the tenant's metric name for a placeholder.	2026-08-02 11:35:08 +00:00
dependabot[bot]	2d90a9f5eb	chore(deps): bump golang.org/x/net in /scripts/promqltestcorpus (#12373 ) Some checks failed build-staging / prepare (push) Has been cancelled Details build-staging / js-build (push) Has been cancelled Details build-staging / go-build (push) Has been cancelled Details build-staging / staging (push) Has been cancelled Details cacheci / tests (push) Has been cancelled Details Release Drafter / update_release_draft (push) Has been cancelled Details Bumps [golang.org/x/net](https://github.com/golang/net) from 0.54.0 to 0.55.0. - [Commits](https://github.com/golang/net/compare/v0.54.0...v0.55.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.55.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-08-02 09:28:15 +00:00
dependabot[bot]	f75cd0854f	chore(deps): bump golang.org/x/crypto in /scripts/promqltestcorpus (#12320 ) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.50.0 to 0.52.0. - [Commits](https://github.com/golang/crypto/compare/v0.50.0...v0.52.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-08-02 08:54:18 +00:00
Pandey	ab91995ee5	ci: restore shared go and pnpm caches in integration and e2e jobs (#12369 ) Some checks failed build-staging / prepare (push) Has been cancelled Details build-staging / js-build (push) Has been cancelled Details build-staging / go-build (push) Has been cancelled Details build-staging / staging (push) Has been cancelled Details cacheci / tests (push) Has been cancelled Details Release Drafter / update_release_draft (push) Has been cancelled Details * feat(tests): add --signoz-image option to run a prebuilt image * ci: collapse integrationci and e2eci into testsci with a shared image build * ci: rename flag to --zeus-network-aliases * ci: use singular --zeus-network-alias derived from a shared env var * revert: drop the testsci workflow and the --signoz-image option * ci: restore shared go and pnpm caches in integration and e2e jobs * ci: inject cache tarballs instead of directory trees	2026-08-01 18:58:17 +00:00
Pandey	fd2bd85e90	ci(cacheci): extract cache mounts via docker cp instead of the local exporter (#12370 ) * ci(cacheci): extract cache mounts via docker cp instead of the local exporter * ci(cacheci): make extracted cache writable for unprivileged untar * ci(cacheci): move cache contents as tarballs * ci(cacheci): drop temporary pr trigger and cancel superseded runs	2026-08-01 17:45:15 +00:00
Pandey	c110e14505	ci: add cacheci workflow to maintain shared go and pnpm caches (#12368 ) Some checks failed build-staging / prepare (push) Has been cancelled Details build-staging / js-build (push) Has been cancelled Details build-staging / go-build (push) Has been cancelled Details build-staging / staging (push) Has been cancelled Details cacheci / tests (push) Has been cancelled Details Release Drafter / update_release_draft (push) Has been cancelled Details	2026-08-01 16:11:09 +00:00
Pandey	5917f9fe31	perf(tests): cache go and pnpm stores across integration image builds (#12366 ) * perf(tests): cache go and pnpm stores across integration image builds Add BuildKit cache mounts for GOCACHE/GOMODCACHE and the pnpm store to the integration Dockerfiles, and build the image via the docker CLI (docker-py, used by testcontainers' DockerImage, does not support BuildKit). Embed the go build command directly so Makefile changes do not invalidate the build layer, and pin HOME/GOCACHE/GOMODCACHE/PNPM_HOME explicitly so cache-mount targets match tool defaults by contract. The with-web node stage fetches dependencies from the lockfile before the source copy, so frontend edits only re-run the offline install and build. * feat(tests): add --clean flag to prune buildkit cache mounts The go and pnpm caches introduced for the integration image build survive --teardown since they belong to the docker builder, not to any container. --clean runs docker builder prune with a type=exec.cachemount filter at session start, forcing the next image build to start cold. Documented in the integration testing guide. * feat(tests): add --rebuild flag to refresh the signoz container under --reuse --reuse keeps the running signoz container, so backend source changes are never picked up without tearing down the whole stack. --rebuild deletes the cached signoz container and recreates it from the current sources (an incremental image build), while databases, mocks and migrations stay reused. Requires --reuse; combining with --teardown or --clean is a usage error. * chore(tests): prune comments to non-obvious constraints * docs(tests): make py-test-setup rebuild signoz and audit the integration guide py-test-setup now passes --rebuild so re-running it after backend changes transparently swaps in a signoz container built from the current sources. The integration guide documents the iteration loop and fixes stale content: option defaults (clickhouse 25.12.5, schema migrator v0.144.6), the nonexistent --zookeeper-version option, Zookeeper vs ClickHouse Keeper, the e2e doc path, and the lint toolchain (ruff). * docs(tests): wire --rebuild into the e2e setup flow The e2e bootstrap shares the signoz fixture, so --rebuild already applies; with --with-web it also picks up frontend changes since the image bakes the built frontend in. The setup command now passes --rebuild, and the guide documents the iteration loop, the --rebuild/--clean flags, ClickHouse Keeper instead of Zookeeper, and the corrected integration doc path. * docs(tests): qualify --rebuild workflow for suites with custom signoz variants make py-test-setup only rebuilds the default signoz instance; suites that create their own via create_signoz(cache_key=...) keep a separately cached container. Passing --rebuild on the suite run itself rebuilds every variant that run instantiates. * docs(tests): describe --clean behaviour instead of its exact command Keeps the docs from drifting if the prune invocation behind --clean changes.	2026-08-01 13:52:19 +00:00