Compare commits

..

2 Commits

Author SHA1 Message Date
Claude
6cdc5dbb74 docs(frontend): add frontend-testing skill for unit/integration tests
Adds a repo skill at .claude/skills/frontend-testing that codifies how
this codebase writes and optimizes frontend Jest + React Testing Library
tests, so generated/updated tests match existing conventions instead of
drifting.

It covers the unit-vs-integration split, the custom tests/test-utils
render and its provider overrides, MSW mocking via mocks-server, the nuqs
testing adapter, renderHook, data-testid-first querying, and the concrete
speed/flake do's and don'ts (userEvent delay:null, findBy over fixed
setTimeout sleeps, awaiting portals by role), plus the pnpm run/verify
commands. Complements the existing Playwright E2E agents in .claude/agents.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kvj24L5mcezPkxoBYew4xa
2026-07-31 18:27:40 +00:00
Claude
b66317d025 test(frontend): remove dead print-only test in generateColorPair
The "prints the base→dark table for visual inspection" test only emitted
console.log output (a debug/diagnostic dump) with a throwaway sentinel
assertion. It exercised no behaviour the sibling assertions don't already
cover, so it was dead weight in the suite. The real coverage — deterministic
pairing, palette bounds, darken uniqueness — remains intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kvj24L5mcezPkxoBYew4xa
2026-07-31 18:27:29 +00:00
53 changed files with 506 additions and 1903 deletions

View File

@@ -0,0 +1,212 @@
---
name: frontend-testing
description: Guide for writing and updating SigNoz frontend unit and integration (component) tests with Jest + React Testing Library. Use whenever you are creating a `*.test.ts`/`*.test.tsx` file, adding coverage for a component, hook, util, or provider under `frontend/src`, mocking an API for a component test, or fixing/optimizing an existing frontend test. Covers the custom `tests/test-utils` render, the MSW mock server, the nuqs testing adapter, `renderHook`, data-testid conventions, and the speed/flake pitfalls specific to this codebase.
---
# SigNoz frontend testing
Write frontend tests the way this repo already does. **Match the nearest existing test** in the same folder before inventing a pattern — consistency beats cleverness. This guide encodes the conventions so a new test looks like it was always there.
Everything below is scoped to `frontend/` (package manager is **pnpm**; the E2E Playwright suite under `tests/e2e/` is a separate stack — see `docs/contributing/tests/e2e.md`, not this skill).
## The two kinds of test in this repo
| | **Unit** | **Integration (component)** |
|---|---|---|
| Target | pure functions, hooks, reducers, selectors | a component / container rendered with its providers |
| Render | none, or `renderHook` | the custom `render` from `tests/test-utils` |
| API | not involved | **MSW** mock server (`mocks-server/`) |
| Assert on | return value / state | rendered DOM + user interactions |
Both live beside the code they test, never in a separate top-level tree.
## Decision checklist (read before writing)
1. **Is there a sibling test?** `ls` the folder for `__tests__/`, `__test__/`, or `tests/` and a `*.test.tsx`. If one exists, copy its imports and structure. Folder naming is inconsistent across the repo — follow the folder you are in, don't rename it.
2. **Unit or integration?** Use the table above.
3. **Does it hit an API?** If yes, you need MSW (see [API mocking](#api-mocking-msw)). Never let a component fire a real request.
4. **Does it read/write the URL (`nuqs`)?** If yes, use the nuqs testing adapter (see [URL state](#url-state-nuqs)).
5. **What's the query?** Prefer `data-testid` (see [Queries](#queries--assertions)). If the component lacks one on a critical control, add it to the component — `frontend/CLAUDE.md` requires testids on inputs/buttons.
## File placement & naming
- Co-locate: `Foo.tsx``Foo.test.tsx` next to it, or under a `__tests__/` folder beside it. Match whatever the sibling files do.
- `testMatch` is `src/**/*(*.)test.(ts|js)?(x)` — the file **must** end in `.test.ts`/`.test.tsx`.
- Name integration specs that render a subtree `*.integration.test.tsx` when the folder already uses that suffix (e.g. `NewSelect/__test__/VariableItem.integration.test.tsx`); otherwise plain `.test.tsx`.
- Shared per-suite render/setup goes in a `_helpers.tsx` or `*.testUtils.tsx` in the same folder — not exported app-wide.
## Integration tests: the custom `render`
Import `render` (and `screen`, `userEvent`, matchers) from **`tests/test-utils`**, not from `@testing-library/react` directly. The custom render wraps the component in every provider the app needs — `MemoryRouter`, `NuqsAdapter`, react-query `QueryClientProvider`, a redux mock store, `AppContext`, `Timezone`, `QueryBuilderProvider`, etc. — so a container renders exactly as it does in the app.
```tsx
import { render, screen, userEvent } from 'tests/test-utils';
import MyPanel from '../MyPanel';
describe('MyPanel', () => {
it('renders the empty state when there is no data', () => {
render(<MyPanel />);
expect(screen.getByTestId('my-panel-empty')).toBeInTheDocument();
});
});
```
### Provider overrides (third argument)
`render(ui, options?, providerProps?)`. The third arg drives the providers:
```tsx
render(<RolesSettings />, undefined, {
role: 'VIEWER', // 'ADMIN' (default) | 'EDITOR' | 'VIEWER'
initialRoute: '/settings/roles?tab=members', // seeds MemoryRouter + URL
appContextOverrides: { // deep-merge over getAppContextMock(role)
featureFlags: [{ name: FeatureKeys.GATEWAY, active: false, usage: 0, usage_limit: -1, route: '' }],
},
queryBuilderOverrides: { /* Partial<QueryBuilderContextType> */ },
});
```
- Use `role` to test permission gating (`hasEditPermission` is derived: ADMIN/EDITOR ⇒ true).
- Use `appContextOverrides` for licence state, feature flags, user/org, preferences — see `getAppContextMock` in `tests/test-utils.tsx` for the full shape. Import `getAppContextMock` directly when you build a bespoke wrapper (as `ListAlertRules/__tests__/_helpers.tsx` does).
- Don't hand-roll a provider stack unless a component needs something the custom render can't express (e.g. `VirtuosoMockContext` for a virtualised list, or a bespoke provider). When you do, model it on `_helpers.tsx` and still reuse `getAppContextMock`.
### What's already handled for you (don't re-mock)
The global `jest.setup.ts` and `test-utils` already provide these — reusing them keeps tests fast and consistent:
- **`react-i18next`** — `t` returns the key verbatim. Assert on keys, not translated copy.
- **MSW server** — `server.listen()` / `resetHandlers()` / `close()` run around every test automatically.
- **`ResizeObserver`, `IntersectionObserver`, `matchMedia`, `scrollIntoView`, Pointer Capture** — stubbed for jsdom + Radix/Ant.
- **System time** pinned to `2023-10-20` (`test-utils` sets it in `beforeEach`). Override per-suite with `jest.setSystemTime(new Date('...'))` when a test needs a specific clock.
- Module mocks in `frontend/__mocks__/` and `src/__tests__/` (`safeNavigateMock`, `logEventMock`, icons, css, uplot, motion). Import the shared mock rather than writing your own:
```ts
import { safeNavigateMock } from '__tests__/safeNavigateMock';
import { logEventMock } from '__tests__/logEventMock';
```
## API mocking (MSW)
APIs are mocked with **MSW**, not by hand-mocking `axios` or react-query. The server is started globally; default handlers live in `src/mocks-server/handlers.ts` with fixtures in `src/mocks-server/__mockdata__/`.
**Prefer overriding a handler per-test** over `jest.mock`-ing the api module — it exercises the real react-query + api layer:
```tsx
import { server } from 'mocks-server/server';
import { rest } from 'msw';
import { render, screen } from 'tests/test-utils';
it('shows an error when the rules API 500s', async () => {
server.use(
rest.get('*/api/v1/rules', (_req, res, ctx) => res(ctx.status(500))),
);
render(<ListAlertRules />);
expect(await screen.findByText(/something went wrong/i)).toBeInTheDocument();
});
```
- Add a reusable happy-path handler + fixture to `handlers.ts` / `__mockdata__/` when several tests need it; use `server.use(...)` for the one-off/error case. `resetHandlers()` after each test wipes per-test overrides automatically.
- For an API call the component makes, prefer an MSW handler over mocking the generated hook. Reach for `jest.mock('api/...')` only when the module has no HTTP surface to intercept (pure client-side helper) or the sibling tests already mock it that way.
## URL state (nuqs)
Components built on `nuqs` must use the **testing adapter**, not the real one (which throttles writes to `window.history`). Use the helpers in `tests/nuqs-helpers.ts`:
- `resetNuqsState(initialSearch)` in setup, `onNuqsUrlUpdate` as the adapter's `onUrlUpdate`, and assert with `getCurrentNuqsQueryString()` / `getCurrentNuqsSearchParams()`.
- `window.location.search` is **not** authoritative in tests — the adapter keeps state in memory. Assert via the helpers.
- The default `render` already installs `NuqsAdapter`; when you build a custom wrapper, use `NuqsTestingAdapter` with `rateLimitFactor={0}` and `hasMemory` (see `_helpers.tsx`).
## Hook & util tests
```tsx
import { act, renderHook } from '@testing-library/react';
import { useThing } from '../useThing';
it('increments', () => {
const { result } = renderHook(() => useThing());
act(() => result.current.increment());
expect(result.current.count).toBe(1);
});
```
- A hook that needs context/providers takes a `wrapper` — build a small local `createWrapper(props)` returning the minimal provider tree (see `QuerySearchV2` context tests). Don't pull in the full app render for a focused hook test.
- Pure utils: no render at all — call the function and assert. Keep these fast and exhaustive (edge cases, empty/undefined, boundaries).
## Queries & assertions
Locator priority (matches `frontend/CLAUDE.md`):
1. `getByTestId('...')` / `findByTestId` — **preferred**. If a critical control has no testid, add one to the component.
2. `getByRole('button', { name: /save/i })`
3. `getByLabelText`, `getByPlaceholderText`
4. `getByText` — last resort; brittle against copy edits (and i18n returns keys here).
Rules:
- **`findBy*` for anything async** (after a click that fires a request, or portal content). `getBy*` throws immediately — only for content already in the DOM.
- **`queryBy*` for absence**: `expect(screen.queryByRole('menu')).not.toBeInTheDocument()`.
- Radix/Ant menus and dialogs render in a **portal** — after opening, `await screen.findByRole('menu')`/`'dialog'`, then query items inside it.
- Use `@testing-library/jest-dom` matchers (`toBeInTheDocument`, `toBeDisabled`, `toHaveTextContent`) — they're globally available.
## Interactions
Prefer `userEvent` over `fireEvent` for real user flows. **Set it up with `delay: null`** so tests don't wait real time between keystrokes:
```tsx
const user = userEvent.setup({ delay: null });
await user.click(screen.getByTestId('open-menu'));
await user.type(screen.getByRole('textbox'), 'cpu');
```
`fireEvent` is fine for a single low-level event where `userEvent` is overkill.
## Performance & flake — do / don't
**Do**
- `userEvent.setup({ delay: null })` — the single biggest per-test speedup.
- `await expect(locator).toBeVisible()` / `await screen.findBy...` to wait on state.
- Reuse the shared render, MSW server, and `__mocks__/` — re-mocking heavy modules per file is slow.
- Keep pure logic in utils and unit-test it directly instead of asserting it through a full component render.
- Scope MSW overrides tightly and rely on the automatic `resetHandlers()`.
**Don't**
- ❌ `await new Promise(r => setTimeout(r, 500))` or `waitForTimeout` — poll with `findBy`/`waitFor` instead.
- ❌ `it.only` / `describe.only` / `fit` / `fdescribe` — CI and lint forbid `.only`; a stray one silently drops the rest of the suite.
- ❌ Leaving `it.skip`/`xit` behind — either fix and enable, or delete it and open an issue. A permanently-skipped test is dead coverage that reads as green.
- ❌ `console.log` in committed tests.
- ❌ Asserting on translated strings (i18n mock returns the key) or on real timers/dates (clock is pinned).
- ❌ Real network — every request must be an MSW handler.
## Running & verifying
From `frontend/`:
```bash
pnpm jest path/to/File.test.tsx # one file
pnpm jest -t "renders the empty state" # by test name
pnpm jest:watch # watch mode while iterating
pnpm test:changedsince # only tests changed vs main, with coverage
pnpm jest:coverage # full coverage run
```
Before calling a test task done (per `frontend/CLAUDE.md`):
```bash
pnpm tsgo --noEmit # types
pnpm oxlint <files> # lint the files you touched, fix all warnings
pnpm jest <files> # the tests pass
```
## Author checklist
- [ ] File ends in `.test.tsx`, co-located, matching the sibling folder's convention.
- [ ] Imports `render`/`screen`/`userEvent` from `tests/test-utils` (integration) or uses `renderHook` (hook).
- [ ] Every API call is an MSW handler; no real network, no ad-hoc axios mock.
- [ ] nuqs components use the testing adapter; URL asserted via `getCurrentNuqs*`.
- [ ] Queries use `data-testid` first; async waits use `findBy*`; portals awaited by role.
- [ ] `userEvent.setup({ delay: null })`; no `setTimeout`/`waitForTimeout`.
- [ ] No `.only`, no stray `.skip`, no `console.log`.
- [ ] Assertions cover success **and** the failure/empty/permission path.
- [ ] `pnpm tsgo --noEmit`, `pnpm oxlint`, and `pnpm jest` all green on the touched files.

View File

@@ -75,30 +75,6 @@ jobs:
docker rm pw
echo "PLAYWRIGHT_BROWSERS_PATH=$RUNNER_TEMP/ms-playwright" >> "$GITHUB_ENV"
cd tests/e2e && pnpm playwright install-deps ${{ matrix.project }}
# Restore-only: the cacheci workflow owns cache saves. Seeds the
# BuildKit cache mounts so the in-test image build is incremental.
- name: restore
id: restore
uses: actions/cache/restore@v4
with:
path: ${{ runner.temp }}/cacheci
key: tests-primary
restore-keys: |
tests-secondary
- name: inject
if: steps.restore.outputs.cache-matched-key != ''
run: |
cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
FROM busybox:1.37
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/pnpm/store \
--mount=type=bind,target=/restored \
tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
tar -xf /restored/pnpm-store.tar -C /pnpm/store
EOF
docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
- name: bring-up-stack
run: |
cd tests && \

View File

@@ -110,30 +110,6 @@ jobs:
sudo mv chromedriver-linux64/chromedriver /usr/local/bin/chromedriver
chromedriver -version
google-chrome-stable --version
# Restore-only: the cacheci workflow owns cache saves. Seeds the
# BuildKit cache mounts so the in-test image build is incremental.
- name: restore
id: restore
uses: actions/cache/restore@v4
with:
path: ${{ runner.temp }}/cacheci
key: tests-primary
restore-keys: |
tests-secondary
- name: inject
if: steps.restore.outputs.cache-matched-key != ''
run: |
cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
FROM busybox:1.37
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/pnpm/store \
--mount=type=bind,target=/restored \
tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
tar -xf /restored/pnpm-store.tar -C /pnpm/store
EOF
docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
- name: run
run: |
cd tests && \

View File

@@ -209,8 +209,8 @@ py-lint: ## Run ruff check across the shared tests project
@cd tests && uv run ruff check --fix .
.PHONY: py-test-setup
py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests, rebuilding signoz from the current sources
@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup
py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests
@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --capture=no integration/bootstrap/setup.py::test_setup
.PHONY: py-test-teardown
py-test-teardown: ## Tear down the shared SigNoz backend

View File

@@ -4,13 +4,9 @@ ARG OS="linux"
ARG TARGETARCH
ARG ZEUSURL
# HOME comes from the build user, not the image config; declare it so the
# /root paths below trace to it.
ENV HOME=/root
# This path is important for stacktraces
WORKDIR $GOPATH/src/github.com/signoz/signoz
WORKDIR $HOME
WORKDIR /root
RUN set -eux; \
apt-get update; \
@@ -18,36 +14,23 @@ RUN set -eux; \
g++ \
gcc \
libc6-dev \
make \
pkg-config \
; \
rm -rf /var/lib/apt/lists/*
# Keep the literal cache-mount targets below in sync with these. The caches
# are shared with Dockerfile.with-web.integration (same target paths).
ENV GOCACHE=$HOME/.cache/go-build
ENV GOMODCACHE=$GOPATH/pkg/mod
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
go mod download
RUN go mod download
COPY ./cmd/ ./cmd/
COPY ./ee/ ./ee/
COPY ./pkg/ ./pkg/
COPY ./templates /root/templates
# Invoked directly instead of via make so Makefile changes don't invalidate
# this layer; the Makefile's git-derived ldflags resolve to empty in here
# anyway (.git is dockerignored).
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
-ldflags "-s -w \
-X github.com/SigNoz/signoz/pkg/version.version=integration \
-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"
COPY Makefile Makefile
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
RUN chmod 755 /root /root/signoz

View File

@@ -1,23 +1,10 @@
FROM node:22-bookworm AS build
WORKDIR /opt/
# HOME comes from the build user, not the image config.
ENV HOME=/root
# pnpm's store lives at $PNPM_HOME/store — a dedicated directory pnpm
# manages. Keep the literal cache-mount targets below in sync.
ENV PNPM_HOME=/pnpm
ENV NODE_OPTIONS=--max-old-space-size=8192
RUN CI=1 npm i -g pnpm@10
# pnpm fetch resolves from the lockfile alone and runs no lifecycle scripts;
# the repo's postinstall needs source files that are not copied yet.
COPY ./frontend/package.json ./frontend/pnpm-lock.yaml ./frontend/pnpm-workspace.yaml ./
RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm fetch
COPY ./frontend/ ./
RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm install --offline
ENV NODE_OPTIONS=--max-old-space-size=8192
RUN CI=1 npm i -g pnpm@10
RUN CI=1 pnpm install
RUN CI=1 pnpm build
FROM golang:1.25-bookworm
@@ -26,13 +13,9 @@ ARG OS="linux"
ARG TARGETARCH
ARG ZEUSURL
# HOME comes from the build user, not the image config; declare it so the
# /root paths below trace to it.
ENV HOME=/root
# This path is important for stacktraces
WORKDIR $GOPATH/src/github.com/signoz/signoz
WORKDIR $HOME
WORKDIR /root
RUN set -eux; \
apt-get update; \
@@ -40,36 +23,23 @@ RUN set -eux; \
g++ \
gcc \
libc6-dev \
make \
pkg-config \
; \
rm -rf /var/lib/apt/lists/*
# Keep the literal cache-mount targets below in sync with these. The caches
# are shared with Dockerfile.integration (same target paths).
ENV GOCACHE=$HOME/.cache/go-build
ENV GOMODCACHE=$GOPATH/pkg/mod
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
go mod download
RUN go mod download
COPY ./cmd/ ./cmd/
COPY ./ee/ ./ee/
COPY ./pkg/ ./pkg/
COPY ./templates /root/templates
# Invoked directly instead of via make so Makefile changes don't invalidate
# this layer; the Makefile's git-derived ldflags resolve to empty in here
# anyway (.git is dockerignored).
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
-ldflags "-s -w \
-X github.com/SigNoz/signoz/pkg/version.version=integration \
-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"
COPY Makefile Makefile
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
COPY --from=build /opt/build ./web/

View File

@@ -7430,8 +7430,6 @@ components:
- below
- equal
- not_equal
- above_or_equal
- below_or_equal
- outside_bounds
type: string
RuletypesCumulativeSchedule:

View File

@@ -1,123 +0,0 @@
# PromQL Serving — clickhouseprometheusv2
This document is the subsystem context for `pkg/prometheus/clickhouseprometheusv2`,
the second-generation ClickHouse-backed Prometheus provider. It explains why the
package exists, the correctness constraints that shaped it, and how each fetch
reduction is proven not to change results. Any change to the provider must keep
these invariants; if a change would violate one, it must be flagged and
discussed.
---
## Why a second provider
The v1 provider (`pkg/prometheus/clickhouseprometheus`) serves the promql engine
through the remote-read protobuf adapter: every raw sample of a query's union
window is fetched, serialized, and handed to the engine. The cost is a function
of ingested data, not of the question asked — which is how a dashboard of PromQL
panels can take an instance down.
In v2 the stock promql engine evaluates over a native `storage.Querier`: no
translation layer, per-selector fetch windows, and fetch reductions that are
provably invisible to the engine.
**The core constraint: every reduction either preserves engine semantics exactly
or is not performed.** A PromQL result that differs from upstream Prometheus is
a lost user. The conformance suite
(`tests/integration/tests/promqlconformance/`) replays Prometheus' own test
corpus against both providers and is the arbiter.
---
## Series lookup
Matchers resolve to series once per selector (`selectSeries`) against the series
tables, which hold one row per (fingerprint, bucket) at 1h/6h/1d/1w
granularities. Table selection and window rounding delegate to the shared
metrics schema package (`pkg/telemetryschema/metricstelemetryschema`); the
window start rounds down to the bucket boundary so a window beginning mid-bucket
still matches the bucket's row.
How matchers become SQL is documented at `applySeriesConditions`. The rules that
carry semantics:
- `__name__` matchers (all four types) translate to the `metric_name` column.
- Every other matcher becomes a `JSONExtractString` condition on the labels
column. An equality matcher against `""` matches series *without* the label,
mirroring PromQL, because `JSONExtractString` returns `""` for missing keys.
- Regexes are anchored (`^(?:...)$`) before they reach `match()`: PromQL
matchers match the whole value, ClickHouse `match()` searches for a
substring.
- The series-lookup upper bound is inclusive (`unix_milli <= end`) because the
exporter floors registration rows to the bucket start: a series first
registered in the bucket beginning exactly at `end` would otherwise be
invisible while its samples are in range.
Empty-valued labels come off at this boundary: an empty value means "label
absent" in Prometheus, but stored attribute JSON can carry them.
---
## Sample fetch
Samples are fetched per selector using the engine's per-selector hints, not the
query-wide union window — `foo / foo offset 1d` reads two narrow windows
instead of the widest one twice.
**Last-sample-per-step reduction.** Instant selectors of subquery-free queries
fetch only the last sample per step bucket. The engine resolves an instant
selector at each grid timestamp `t` to the latest sample in the left-open
lookback window `(t lookback, t]`. Buckets anchor at the selector's first
evaluation timestamp — recovered from the hints as
`hints.Start + lookback 1ms`, the inverse of how the engine derives
`hints.Start` — so bucket boundaries coincide with evaluation timestamps, and a
non-final sample of a bucket can never be the latest sample in
`(t lookback, t]` for any grid `t`. Real timestamps are preserved, so the
engine's own lookback and staleness handling stay exact.
Range selectors always fetch raw — every sample feeds the range function. The
subquery-free proof travels in the context as `prometheus.QueryTraits`, because
subquery selectors evaluate at the subquery's step while the hints carry the
top-level step; call sites that do not attach traits get the conservative raw
fetch.
**Row assembly** maps stale flags to the engine's `StaleNaN` and merges series
with identical label sets (`sortAndMerge`) — the engine assumes storages never
emit duplicates. Duplicate timestamps pass through as stored: uniqueness is
ingest's job, and v1 feeds them to the engine as-is over the same data.
**The fingerprint filter is a shard-local semi-join.** The samples query
restricts to the matched series by re-running the series predicates as an
`IN (SELECT fingerprint FROM <local series table> ...)` subquery, not a GLOBAL
broadcast of the matched set. ClickHouse materializes the subquery's set per
shard before the scan, so it still engages the fingerprint primary-key column.
Because the subquery re-executes the predicates after the lookup ran, it can
match series registered in between; sample rows whose fingerprint the lookup
never saw are skipped — the lookup is the read snapshot.
---
## Sharding
`samples_v4` and `time_series_v4` (and all their rollups) shard on the same key
`cityHash64(env, temporality, metric_name, fingerprint)` — so a series'
samples and catalog rows live on the same shard. The semi-join above exploits
that: each shard filters by its own series rows, which are exactly the series
of that shard's samples.
The temporality filter on every samples statement
(`temporality IN ['Cumulative', 'Unspecified']`) is a semantic no-op — the
matched fingerprints already come from those temporalities — that engages the
leading samples primary-key column.
Delta-temporality series stay invisible to PromQL exactly as they are in v1:
the rollout gate is parity with v1, and a Delta stream fed to `rate()`
as-if-cumulative would be wrong, not just new.
---
## Observability
Every statement carries a `log_comment` with
`code.namespace=clickhouse-prometheus-v2` and `code.function.name` naming the
call site, so this provider's work is attributable in `system.query_log`.

View File

@@ -30,11 +30,11 @@ yarn install:browsers # one-time Playwright browser install
### Starting the Test Environment
To spin up the backend stack (SigNoz, ClickHouse, Postgres, ClickHouse Keeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
To spin up the backend stack (SigNoz, ClickHouse, Postgres, Zookeeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
```bash
cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --with-web \
uv run pytest --basetemp=./tmp/ -vv --reuse --with-web \
e2e/bootstrap/setup.py::test_setup
```
@@ -45,13 +45,8 @@ This command will:
- Start the HTTP seeder container (`tests/seeder/` — exposing `/telemetry/{traces,logs,metrics}` POST + DELETE)
- Write backend coordinates to `tests/e2e/.env.local` (loaded by `playwright.config.ts` via dotenv)
- Keep containers running via the `--reuse` flag
- Rebuild the SigNoz container from the current sources via the `--rebuild` flag
The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start; later builds are incremental.
### Rebuilding After Source Changes
The `--with-web` image bakes the built frontend in, so neither backend nor frontend changes are picked up while `--reuse` keeps the container running. `--rebuild` fixes that for both: it kills the SigNoz container, rebuilds the image incrementally (go build cache + pnpm store — a frontend-only change rebuilds in about a minute), and starts a fresh one while databases, mocks, migrations, and the seeder stay reused. The setup command above passes it, so the iteration loop is: change code → re-run the setup command → re-run your specs. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.
The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start.
### Stopping the Test Environment
@@ -286,16 +281,13 @@ The full `playwright.config.ts` is the source of truth. Common things to tweak:
The same pytest flags integration tests expose work here, since E2E reuses the shared fixture graph:
- `--reuse` — keep containers warm between runs (required for all iteration).
- `--rebuild` — recreate the SigNoz container from the current sources (backend and, with `--with-web`, frontend) while the rest of the stack stays up. Requires `--reuse`.
- `--teardown` — tear everything down.
- `--clean` — prune the docker build caches, forcing the next image build to start cold.
- `--with-web` — build the frontend into the SigNoz container. **Required for E2E**; integration tests don't need it.
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/tests/integration.md`.
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/integration.md`.
## What should I remember?
- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build on a cold start; later builds are incremental.
- **Changed backend or frontend code? Re-run the setup command** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build; you only want to pay that once.
- **Don't teardown before setup.** `--reuse` correctly handles partially-set-up state, so chaining teardown → setup wastes time.
- **Prefer UI-driven flows.** Playwright captures BE requests in the trace; a parallel `fetch` probe is almost always redundant. Drop to `page.request.*` only when the UI can't reach what you need.
- **Use `page.waitForResponse` on UI clicks** to assert BE contracts — it still exercises the UI trigger path.

View File

@@ -37,34 +37,13 @@ make py-test-setup
Under the hood this runs, from `tests/`:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup
uv run pytest --basetemp=./tmp/ -vv --reuse integration/bootstrap/setup.py::test_setup
```
This command will:
- Start all required services (ClickHouse, PostgreSQL, ClickHouse Keeper, SigNoz, Zeus mock, gateway mock)
- Start all required services (ClickHouse, PostgreSQL, Zookeeper, SigNoz, Zeus mock, gateway mock)
- Register an admin user
- Keep containers running via the `--reuse` flag
- Rebuild the SigNoz container from the current sources via the `--rebuild` flag
### Rebuilding After Source Changes
`--reuse` keeps the running SigNoz container, which means backend source changes are not picked up. `--rebuild` fixes exactly that: it kills the existing SigNoz container, rebuilds the image (incremental — only changed packages recompile thanks to the build cache), and starts a fresh one, while everything else (databases, mocks, migrations) stays reused. `make py-test-setup` passes it by default, so the iteration loop is simply:
```bash
make py-test-setup # (re)build signoz from your current sources
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
# ... edit backend code or tests ...
make py-test-setup # pick up the backend changes
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
```
The same applies to the e2e stack. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.
Some suites define their own SigNoz variant in a suite-local `conftest.py` (`create_signoz(..., cache_key=...)` — e.g. `basepath`, `metricreduction`, `querier_json_body`). Those containers are not touched by `make py-test-setup`, which only rebuilds the default instance. For such suites, pass `--rebuild` on the suite run itself — it rebuilds every SigNoz variant the run instantiates:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild integration/tests/<suite>/
```
### Stopping the Test Environment
@@ -77,21 +56,11 @@ make py-test-teardown
Which runs:
```bash
uv run pytest --basetemp=./tmp/ -vv --teardown --capture=no integration/bootstrap/setup.py::test_teardown
uv run pytest --basetemp=./tmp/ -vv --teardown integration/bootstrap/setup.py::test_teardown
```
This destroys the running integration test setup and cleans up resources.
### Cleaning the Image Build Cache
The `signoz:integration` image build keeps its Go build and module caches in BuildKit cache mounts, so rebuilds only recompile what changed. These caches survive `--teardown` (they belong to the Docker builder, not to any container). If a cache ever needs to be nuked — suspected corruption, disk pressure, or to force a genuinely cold build — pass the `--clean` flag:
```bash
uv run pytest --basetemp=./tmp/ -vv --teardown --clean integration/bootstrap/setup.py::test_teardown
```
`--clean` prunes the docker build artifacts backing the incremental image build at session start, so the next build starts from a clean slate. Images and regular layer cache stay intact, but note the pruning is host-wide — it clears build caches for other projects too, not just SigNoz's. The flag composes with any invocation — passing it on a normal `--reuse` run simply makes the next image build start cold (~34 minutes instead of seconds).
## Understanding the Integration Test Framework
Python and pytest form the foundation of the integration testing framework. Testcontainers are used to spin up disposable integration environments. WireMock is used to spin up **test doubles** of external services (Zeus cloud API, gateway, etc.).
@@ -130,7 +99,7 @@ tests/
│ ├── passwordauthn/
│ ├── querier/
│ └── ...
└── e2e/ # Playwright suite (see docs/contributing/tests/e2e.md)
└── e2e/ # Playwright suite (see docs/contributing/e2e.md)
```
Each test suite follows these principles:
@@ -255,9 +224,9 @@ Tests can be configured using pytest options:
- `--sqlstore-provider` — Choose the SQL store provider (default: `postgres`)
- `--sqlite-mode` — SQLite journal mode: `delete` or `wal` (default: `delete`). Only relevant when `--sqlstore-provider=sqlite`.
- `--postgres-version` — PostgreSQL version (default: `15`)
- `--clickhouse-version` — ClickHouse version, also used for ClickHouse Keeper (default: `25.12.5`)
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.6`)
- `--with-web` — Build the frontend into the SigNoz image (required for e2e)
- `--clickhouse-version` — ClickHouse version (default: `25.5.6`)
- `--zookeeper-version` — Zookeeper version (default: `3.7.1`)
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.2`)
Example:
@@ -270,7 +239,6 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
## What should I remember?
- **Always use the `--reuse` flag** when setting up the environment or running tests to keep containers warm. Without it every run rebuilds the stack (~4 mins).
- **Changed backend code? Re-run `make py-test-setup`** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
- **Use the `--teardown` flag** only when cleaning up — mixing `--teardown` with `--reuse` is a contradiction.
- **Do not pre-emptively teardown before setup.** If the stack is partially up, `--reuse` picks up from wherever it is. `make py-test-teardown` then `make py-test-setup` wastes minutes.
- **Follow the naming convention** with two-digit numeric prefixes (`01_`, `02_`) for ordered test execution within a suite.
@@ -279,5 +247,5 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
- **Use descriptive test names** that clearly indicate what is being tested.
- **Leverage fixtures** for common setup. The shared fixture package is at `tests/fixtures/` — reuse before adding new ones.
- **Test both success and failure scenarios** (4xx / 5xx paths) to ensure robust functionality.
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — ruff format + ruff check.
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — black + isort + autoflake + pylint.
- **`--sqlite-mode=wal` does not work on macOS.** The integration test environment runs SigNoz inside a Linux container with the SQLite database file mounted from the macOS host. WAL mode requires shared memory between connections, and connections crossing the VM boundary (macOS host ↔ Linux container) cannot share the WAL index, resulting in `SQLITE_IOERR_SHORT_READ`. WAL mode is tested in CI on Linux only.

View File

@@ -8479,8 +8479,6 @@ export enum RuletypesCompareOperatorDTO {
below = 'below',
equal = 'equal',
not_equal = 'not_equal',
above_or_equal = 'above_or_equal',
below_or_equal = 'below_or_equal',
outside_bounds = 'outside_bounds',
}
export interface RuletypesBasicRuleThresholdDTO {

View File

@@ -66,10 +66,6 @@ function ThresholdItem({
return '=';
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
return '!=';
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
return '>=';
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
return '<=';
default:
return '';
}

View File

@@ -83,10 +83,6 @@ const getOperatorWord = (op: AlertThresholdOperator): string => {
return 'equal';
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
return 'not equal';
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
return 'equal or exceed';
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
return 'equal or fall below';
default:
return 'exceed';
}
@@ -102,10 +98,6 @@ const getThresholdValue = (op: AlertThresholdOperator): number => {
return 100;
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
return 0;
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
return 80;
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
return 50;
default:
return 80;
}
@@ -124,8 +116,6 @@ const getDataPoints = (
[AlertThresholdOperator.IS_EQUAL_TO]: [95, 100, 105, 90, 100],
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [5, 0, 10, 15, 0],
[AlertThresholdOperator.IS_ABOVE]: [75, 85, 90, 78, 95],
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [75, 80, 90, 78, 95],
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [60, 50, 40, 55, 35],
[AlertThresholdOperator.ABOVE_BELOW]: [75, 85, 90, 78, 95],
},
[AlertThresholdMatchType.ALL_THE_TIME]: {
@@ -133,8 +123,6 @@ const getDataPoints = (
[AlertThresholdOperator.IS_EQUAL_TO]: [100, 100, 100, 100, 100],
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [5, 10, 15, 8, 12],
[AlertThresholdOperator.IS_ABOVE]: [85, 87, 90, 88, 95],
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [80, 87, 90, 88, 95],
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [50, 40, 35, 42, 38],
[AlertThresholdOperator.ABOVE_BELOW]: [85, 87, 90, 88, 95],
},
[AlertThresholdMatchType.ON_AVERAGE]: {
@@ -142,8 +130,6 @@ const getDataPoints = (
[AlertThresholdOperator.IS_EQUAL_TO]: [95, 105, 100, 95, 105],
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [5, 10, 15, 8, 12],
[AlertThresholdOperator.IS_ABOVE]: [75, 85, 90, 78, 95],
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [70, 85, 90, 75, 80],
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [60, 40, 55, 45, 50],
[AlertThresholdOperator.ABOVE_BELOW]: [75, 85, 90, 78, 95],
},
[AlertThresholdMatchType.IN_TOTAL]: {
@@ -151,8 +137,6 @@ const getDataPoints = (
[AlertThresholdOperator.IS_EQUAL_TO]: [20, 20, 20, 20, 20],
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [10, 15, 25, 5, 30],
[AlertThresholdOperator.IS_ABOVE]: [10, 15, 25, 5, 30],
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [10, 15, 25, 5, 25],
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [8, 5, 10, 12, 15],
[AlertThresholdOperator.ABOVE_BELOW]: [10, 15, 25, 5, 30],
},
[AlertThresholdMatchType.LAST]: {
@@ -160,8 +144,6 @@ const getDataPoints = (
[AlertThresholdOperator.IS_EQUAL_TO]: [75, 85, 90, 78, 100],
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [75, 85, 90, 78, 25],
[AlertThresholdOperator.IS_ABOVE]: [75, 85, 90, 78, 95],
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [75, 85, 90, 78, 80],
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [75, 85, 90, 78, 50],
[AlertThresholdOperator.ABOVE_BELOW]: [75, 85, 90, 78, 95],
},
};
@@ -175,8 +157,6 @@ const getTooltipOperatorSymbol = (op: AlertThresholdOperator): string => {
[AlertThresholdOperator.IS_BELOW]: '<',
[AlertThresholdOperator.IS_EQUAL_TO]: '=',
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: '!=',
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: '>=',
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: '<=',
[AlertThresholdOperator.ABOVE_BELOW]: '>',
};
return symbolMap[op] || '>';
@@ -272,10 +252,6 @@ export const getMatchTypeTooltip = (
return p === thresholdValue;
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
return p !== thresholdValue;
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
return p >= thresholdValue;
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
return p <= thresholdValue;
default:
return p > thresholdValue;
}
@@ -318,8 +294,7 @@ export const getMatchTypeTooltip = (
matchType={matchType}
>
Alert triggers (all points {operatorWord} {thresholdValue})<br />
If any point didn&apos;t {operatorWord} {thresholdValue}, no alert would
fire
If any point was {thresholdValue}, no alert would fire
</TooltipExample>
<TooltipLink />
</TooltipContent>

View File

@@ -532,7 +532,7 @@ describe('Footer utils', () => {
['symbol', '>', 'at_least_once'],
['literal', 'above', 'at_least_once'],
['short', 'eq', 'avg'],
['inclusive', 'above_or_equal', 'at_least_once'],
['UI-unexposed', 'above_or_equal', 'at_least_once'],
])(
'round-trips %s op/matchType unchanged through the submit payload (%s / %s)',
(_desc, op, matchType) => {

View File

@@ -332,20 +332,25 @@ describe('CreateAlertV2 utils', () => {
['not_equal', AlertThresholdOperator.IS_NOT_EQUAL_TO],
['not_eq', AlertThresholdOperator.IS_NOT_EQUAL_TO],
['!=', AlertThresholdOperator.IS_NOT_EQUAL_TO],
['5', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
['above_or_equal', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
['above_or_eq', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
['>=', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
['6', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
['below_or_equal', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
['below_or_eq', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
['<=', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
['7', AlertThresholdOperator.ABOVE_BELOW],
['outside_bounds', AlertThresholdOperator.ABOVE_BELOW],
])('maps backend alias %s to canonical enum', (alias, expected) => {
expect(normalizeOperator(alias)).toBe(expected);
});
it.each([
['5', 'above_or_equal'],
['above_or_equal', 'above_or_equal'],
['above_or_eq', 'above_or_equal'],
['>=', 'above_or_equal'],
['6', 'below_or_equal'],
['below_or_equal', 'below_or_equal'],
['below_or_eq', 'below_or_equal'],
['<=', 'below_or_equal'],
])('returns undefined for UI-unexposed alias %s (%s family)', (alias) => {
expect(normalizeOperator(alias)).toBeUndefined();
});
it('returns undefined for unknown values', () => {
expect(normalizeOperator('gibberish')).toBeUndefined();
expect(normalizeOperator(undefined)).toBeUndefined();
@@ -408,8 +413,8 @@ describe('CreateAlertV2 utils', () => {
['symbol', '>', 'at_least_once'],
['short form', 'eq', 'avg'],
['mixed numeric and literal', '7', 'last'],
['inclusive literal operator', 'above_or_equal', 'at_least_once'],
['inclusive numeric operator', '5', 'at_least_once'],
['UI-unexposed operator', 'above_or_equal', 'at_least_once'],
['UI-unexposed numeric operator', '5', 'at_least_once'],
])('preserves %s op/matchType verbatim (%s / %s)', (_desc, op, matchType) => {
const state = getThresholdStateFromAlertDef(buildDef(op, matchType));
expect(state.operator).toBe(op);

View File

@@ -2,8 +2,9 @@ import { AlertThresholdMatchType, AlertThresholdOperator } from './types';
// Mirrors the backend's CompareOperator.Normalize() in
// pkg/types/ruletypes/compare.go. Maps any accepted alias to the enum value
// the dropdown understands. Returns undefined for unknown values so callers
// can keep the raw value on screen instead of silently rewriting it.
// the dropdown understands. Returns undefined for aliases the UI does not
// expose (e.g. above_or_equal, below_or_equal) so callers can keep the raw
// value on screen instead of silently rewriting it.
export function normalizeOperator(
raw: string | undefined,
): AlertThresholdOperator | undefined {
@@ -26,16 +27,6 @@ export function normalizeOperator(
case 'not_eq':
case '!=':
return AlertThresholdOperator.IS_NOT_EQUAL_TO;
case '5':
case 'above_or_equal':
case 'above_or_eq':
case '>=':
return AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO;
case '6':
case 'below_or_equal':
case 'below_or_eq':
case '<=':
return AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO;
case '7':
case 'outside_bounds':
return AlertThresholdOperator.ABOVE_BELOW;

View File

@@ -125,14 +125,6 @@ export const THRESHOLD_OPERATOR_OPTIONS = [
{ value: AlertThresholdOperator.IS_BELOW, label: 'BELOW' },
{ value: AlertThresholdOperator.IS_EQUAL_TO, label: 'EQUAL TO' },
{ value: AlertThresholdOperator.IS_NOT_EQUAL_TO, label: 'NOT EQUAL TO' },
{
value: AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO,
label: 'ABOVE OR EQUAL TO',
},
{
value: AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO,
label: 'BELOW OR EQUAL TO',
},
];
export const ANOMALY_THRESHOLD_OPERATOR_OPTIONS = [

View File

@@ -99,8 +99,6 @@ export enum AlertThresholdOperator {
IS_BELOW = 'below',
IS_EQUAL_TO = 'equal',
IS_NOT_EQUAL_TO = 'not_equal',
IS_ABOVE_OR_EQUAL_TO = 'above_or_equal',
IS_BELOW_OR_EQUAL_TO = 'below_or_equal',
ABOVE_BELOW = 'outside_bounds',
}

View File

@@ -167,9 +167,9 @@ describe('deriveAlertPrefill', () => {
it.each([
['above', AlertThresholdOperator.IS_ABOVE],
['above_or_equal', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
['above_or_equal', AlertThresholdOperator.IS_ABOVE],
['below', AlertThresholdOperator.IS_BELOW],
['below_or_equal', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
['below_or_equal', AlertThresholdOperator.IS_BELOW],
['equal', AlertThresholdOperator.IS_EQUAL_TO],
['not_equal', AlertThresholdOperator.IS_NOT_EQUAL_TO],
])('maps panel operator %s → %s', (op, expected) => {

View File

@@ -104,6 +104,20 @@ function pickHighestDanger(
)[0];
}
// The alert UI has no inclusive operator; collapse "or equal" onto its strict variant.
function panelOperatorToAlertOperator(
operator: DashboardtypesComparisonOperatorDTO | undefined,
): AlertThresholdOperator | undefined {
switch (operator) {
case 'above_or_equal':
return normalizeOperator('above');
case 'below_or_equal':
return normalizeOperator('below');
default:
return normalizeOperator(operator);
}
}
export function deriveAlertPrefill(
panel: DashboardtypesPanelDTO,
query: Query,
@@ -121,7 +135,7 @@ export function deriveAlertPrefill(
const top = pickHighestDanger(readPanelThresholds(panel.spec.plugin));
if (top) {
prefill.operator = normalizeOperator(top.operator);
prefill.operator = panelOperatorToAlertOperator(top.operator);
prefill.threshold = {
id: uuid(),
label: 'critical',

View File

@@ -107,20 +107,4 @@ describe('PALETTE_V3 darken-pair table', () => {
const unique = new Set(darks);
expect(unique.size).toBe(PALETTE_V3.length);
});
it('prints the base→dark table for visual inspection', () => {
// eslint-disable-next-line no-console
console.log('\nPALETTE_V3 base → darkenHex(0.22) pairs:');
// eslint-disable-next-line no-console
console.log('idx name base dark');
PALETTE_V3.forEach((hex, i) => {
const dark = darkenHex(hex, 0.22);
const name = (PALETTE_NAMES[i] ?? '').padEnd(13);
const idx = String(i).padStart(2, ' ');
// eslint-disable-next-line no-console
console.log(`${idx} ${name} ${hex} ${dark}`);
});
// Sentinel assertion so the test is not flagged as having none.
expect(PALETTE_V3).toHaveLength(28);
});
});

4
pkg/config/doc.go Normal file
View File

@@ -0,0 +1,4 @@
// Package config provides the configuration management for the Signoz application.
// It includes functionality to define, load, and validate the application's configuration
// using various providers and formats.
package config

3
pkg/errors/doc.go Normal file
View File

@@ -0,0 +1,3 @@
// package error contains error related utilities. Use this package when
// a well-defined error has to be shown.
package errors

3
pkg/http/doc.go Normal file
View File

@@ -0,0 +1,3 @@
// package http contains all http related functions such
// as servers, middlewares, routers and renders.
package http

2
pkg/http/server/doc.go Normal file
View File

@@ -0,0 +1,2 @@
// package server contains an implementation of the http server.
package server

View File

@@ -0,0 +1,5 @@
// Package instrumentation provides utilities for initializing and managing
// OpenTelemetry resources, logging, tracing, and metering within the application. It
// leverages the OpenTelemetry SDK to facilitate the collection and
// export of telemetry data, to an OTLP (OpenTelemetry Protocol) endpoint.
package instrumentation

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/sync/errgroup"
)
// buildClusterRecords assembles the page records. Node condition counts and
@@ -84,36 +83,16 @@ func buildClusterRecords(
return records
}
func (m *module) getTopClusterGroupsAndMetadata(
func (m *module) getTopClusterGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableClusters,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getClustersTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.ClusterNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ClusterNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ClusterNameAttrKey), nil
}
queryNamesForOrderBy := orderByToClustersQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -147,20 +126,13 @@ func (m *module) getTopClusterGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getClustersTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableClusters) (map[string]map[string]string, error) {

View File

@@ -12,7 +12,6 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/sync/errgroup"
)
// buildContainerRecords assembles the page records, merging kubeletstats
@@ -139,36 +138,16 @@ func buildContainerRecords(
return records
}
func (m *module) getTopContainerGroupsAndMetadata(
func (m *module) getTopContainerGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableContainers,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getContainersTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.ContainerNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ContainerNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ContainerNameAttrKey), nil
}
queryNamesForOrderBy := orderByToContainersQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -202,20 +181,13 @@ func (m *module) getTopContainerGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getContainersTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableContainers) (map[string]map[string]string, error) {

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/sync/errgroup"
)
// buildDaemonSetRecords assembles the page records. Pod status counts come from
@@ -90,36 +89,16 @@ func buildDaemonSetRecords(
return records
}
func (m *module) getTopDaemonSetGroupsAndMetadata(
func (m *module) getTopDaemonSetGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableDaemonSets,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getDaemonSetsTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.DaemonSetNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DaemonSetNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DaemonSetNameAttrKey), nil
}
queryNamesForOrderBy := orderByToDaemonSetsQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -153,20 +132,13 @@ func (m *module) getTopDaemonSetGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getDaemonSetsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableDaemonSets) (map[string]map[string]string, error) {

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/sync/errgroup"
)
// buildDeploymentRecords assembles the page records. Pod status counts come from
@@ -82,36 +81,16 @@ func buildDeploymentRecords(
return records
}
func (m *module) getTopDeploymentGroupsAndMetadata(
func (m *module) getTopDeploymentGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableDeployments,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getDeploymentsTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.DeploymentNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DeploymentNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DeploymentNameAttrKey), nil
}
queryNamesForOrderBy := orderByToDeploymentsQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -145,20 +124,13 @@ func (m *module) getTopDeploymentGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getDeploymentsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableDeployments) (map[string]map[string]string, error) {

View File

@@ -13,7 +13,6 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/sync/errgroup"
)
// getPerGroupHostStatusCounts computes the number of active and inactive hosts per group
@@ -249,41 +248,19 @@ func buildHostRecords(
return records
}
// getTopHostGroupsAndMetadata fetches the group metadata and the ordering-metric
// ranking concurrently, then sorts the ranked results, paginates, and backfills
// from metadataMap when the page extends past the metric-ranked groups. Returns
// the page of groups and the metadata map (the caller needs it for Total and
// records). Callers must apply any req.Filter mutation before calling.
func (m *module) getTopHostGroupsAndMetadata(
// getTopHostGroups runs a ranking query for the ordering metric, sorts the
// results, paginates, and backfills from metadataMap when the page extends
// past the metric-ranked groups.
func (m *module) getTopHostGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableHosts,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getHostsTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.HostNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.HostNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.HostNameAttrKey), nil
}
queryNamesForOrderBy := orderByToHostsQueryNames[orderByKey]
// The last entry is the formula/query whose value we sort by.
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -318,20 +295,13 @@ func (m *module) getTopHostGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
// applyHostsActiveStatusFilter MODIFIES req.Filter.Expression to include an IN/NOT IN

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/sync/errgroup"
)
// buildJobRecords assembles the page records. Pod status counts come from
@@ -90,36 +89,16 @@ func buildJobRecords(
return records
}
func (m *module) getTopJobGroupsAndMetadata(
func (m *module) getTopJobGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableJobs,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getJobsTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.JobNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.JobNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.JobNameAttrKey), nil
}
queryNamesForOrderBy := orderByToJobsQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -153,20 +132,13 @@ func (m *module) getTopJobGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getJobsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableJobs) (map[string]map[string]string, error) {

View File

@@ -191,13 +191,18 @@ func (m *module) ListHosts(ctx context.Context, orgID valuer.UUID, req *inframon
return resp, nil
}
pageGroups, metadataMap, err := m.getTopHostGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getHostsTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopHostGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.HostRecord{}
return resp, nil
@@ -286,13 +291,18 @@ func (m *module) ListPods(ctx context.Context, orgID valuer.UUID, req *inframoni
return resp, nil
}
pageGroups, metadataMap, err := m.getTopPodGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getPodsTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopPodGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.PodRecord{}
return resp, nil
@@ -379,13 +389,18 @@ func (m *module) ListContainers(ctx context.Context, orgID valuer.UUID, req *inf
return resp, nil
}
pageGroups, metadataMap, err := m.getTopContainerGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getContainersTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopContainerGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.ContainerRecord{}
return resp, nil
@@ -478,13 +493,18 @@ func (m *module) ListNodes(ctx context.Context, orgID valuer.UUID, req *inframon
return resp, nil
}
pageGroups, metadataMap, err := m.getTopNodeGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getNodesTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopNodeGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.NodeRecord{}
return resp, nil
@@ -571,13 +591,18 @@ func (m *module) ListNamespaces(ctx context.Context, orgID valuer.UUID, req *inf
return resp, nil
}
pageGroups, metadataMap, err := m.getTopNamespaceGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getNamespacesTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopNamespaceGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.NamespaceRecord{}
return resp, nil
@@ -663,13 +688,18 @@ func (m *module) ListClusters(ctx context.Context, orgID valuer.UUID, req *infra
return resp, nil
}
pageGroups, metadataMap, err := m.getTopClusterGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getClustersTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopClusterGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.ClusterRecord{}
return resp, nil
@@ -769,13 +799,18 @@ func (m *module) ListVolumes(ctx context.Context, orgID valuer.UUID, req *infram
return resp, nil
}
pageGroups, metadataMap, err := m.getTopVolumeGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getVolumesTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopVolumeGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.VolumeRecord{}
return resp, nil
@@ -842,13 +877,18 @@ func (m *module) ListDeployments(ctx context.Context, orgID valuer.UUID, req *in
return resp, nil
}
pageGroups, metadataMap, err := m.getTopDeploymentGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getDeploymentsTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopDeploymentGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.DeploymentRecord{}
return resp, nil
@@ -934,13 +974,18 @@ func (m *module) ListStatefulSets(ctx context.Context, orgID valuer.UUID, req *i
return resp, nil
}
pageGroups, metadataMap, err := m.getTopStatefulSetGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getStatefulSetsTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopStatefulSetGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.StatefulSetRecord{}
return resp, nil
@@ -1028,13 +1073,18 @@ func (m *module) ListJobs(ctx context.Context, orgID valuer.UUID, req *inframoni
return resp, nil
}
pageGroups, metadataMap, err := m.getTopJobGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getJobsTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopJobGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.JobRecord{}
return resp, nil
@@ -1122,13 +1172,18 @@ func (m *module) ListDaemonSets(ctx context.Context, orgID valuer.UUID, req *inf
return resp, nil
}
pageGroups, metadataMap, err := m.getTopDaemonSetGroupsAndMetadata(ctx, orgID, req)
metadataMap, err := m.getDaemonSetsTableMetadata(ctx, orgID, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopDaemonSetGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.DaemonSetRecord{}
return resp, nil

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/sync/errgroup"
)
// buildNamespaceRecords assembles the page records. Pod status counts come from
@@ -65,36 +64,16 @@ func buildNamespaceRecords(
return records
}
func (m *module) getTopNamespaceGroupsAndMetadata(
func (m *module) getTopNamespaceGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableNamespaces,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getNamespacesTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.NamespaceNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NamespaceNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NamespaceNameAttrKey), nil
}
queryNamesForOrderBy := orderByToNamespacesQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -128,20 +107,13 @@ func (m *module) getTopNamespaceGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getNamespacesTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableNamespaces) (map[string]map[string]string, error) {

View File

@@ -12,7 +12,6 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/sync/errgroup"
)
// buildNodeRecords assembles the page records. Condition counts come from
@@ -92,36 +91,16 @@ func buildNodeRecords(
return records
}
func (m *module) getTopNodeGroupsAndMetadata(
func (m *module) getTopNodeGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableNodes,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getNodesTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.NodeNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NodeNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NodeNameAttrKey), nil
}
queryNamesForOrderBy := orderByToNodesQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -155,20 +134,13 @@ func (m *module) getTopNodeGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getNodesTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableNodes) (map[string]map[string]string, error) {

View File

@@ -13,7 +13,6 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/sync/errgroup"
)
// buildPodRecords assembles the page records. Status counts come from
@@ -146,40 +145,16 @@ func buildPodRecords(
return records
}
// getTopPodGroupsAndMetadata fetches the group metadata and the ordering-metric
// ranking concurrently, then pages the ranked groups, backfilling from metadata
// when the page extends past the metric-ranked groups. Returns the page of
// groups and the metadata map (needed by the caller for Total and records).
func (m *module) getTopPodGroupsAndMetadata(
func (m *module) getTopPodGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostablePods,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getPodsTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.PodNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PodNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PodNameAttrKey), nil
}
queryNamesForOrderBy := orderByToPodsQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -213,20 +188,13 @@ func (m *module) getTopPodGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getPodsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostablePods) (map[string]map[string]string, error) {

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/sync/errgroup"
)
// buildStatefulSetRecords assembles the page records. Pod status counts come from
@@ -82,36 +81,16 @@ func buildStatefulSetRecords(
return records
}
func (m *module) getTopStatefulSetGroupsAndMetadata(
func (m *module) getTopStatefulSetGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableStatefulSets,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getStatefulSetsTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.StatefulSetNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.StatefulSetNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.StatefulSetNameAttrKey), nil
}
queryNamesForOrderBy := orderByToStatefulSetsQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -145,20 +124,13 @@ func (m *module) getTopStatefulSetGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getStatefulSetsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableStatefulSets) (map[string]map[string]string, error) {

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/sync/errgroup"
)
// buildVolumeRecords assembles the page records. VolumeUsage is taken from the
@@ -69,36 +68,16 @@ func buildVolumeRecords(
return records
}
func (m *module) getTopVolumeGroupsAndMetadata(
func (m *module) getTopVolumeGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostableVolumes,
) ([]map[string]string, map[string]map[string]string, error) {
var (
orderByKey string
metadataMap map[string]map[string]string
allMetricGroups []rankedGroup
)
orderByKey = req.OrderBy.Key.Name
g, gCtx := errgroup.WithContext(ctx)
g.Go(func() error {
var err error
metadataMap, err = m.getVolumesTableMetadata(gCtx, orgID, req)
return err
})
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
if orderByKey == inframonitoringtypes.PersistentVolumeClaimNameAttrKey {
if err := g.Wait(); err != nil {
return nil, nil, err
}
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PersistentVolumeClaimNameAttrKey)
return pageGroups, metadataMap, nil
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PersistentVolumeClaimNameAttrKey), nil
}
queryNamesForOrderBy := orderByToVolumesQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
@@ -132,20 +111,13 @@ func (m *module) getTopVolumeGroupsAndMetadata(
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
g.Go(func() error {
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
if err != nil {
return err
}
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return nil
})
if err := g.Wait(); err != nil {
return nil, nil, err
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getVolumesTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableVolumes) (map[string]map[string]string, error) {

View File

@@ -1,85 +0,0 @@
package clickhouseprometheusv2
import (
"context"
"sync"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/storage"
"github.com/prometheus/prometheus/util/annotations"
)
// statementRecorder collects the statements a PromQL evaluation would run.
// Safe for concurrent use: the engine may Select selectors concurrently.
type statementRecorder struct {
mu sync.Mutex
statements []prometheus.CapturedStatement
}
func (r *statementRecorder) record(query string, args []any) {
r.mu.Lock()
defer r.mu.Unlock()
r.statements = append(r.statements, prometheus.CapturedStatement{Query: query, Args: args})
}
func (r *statementRecorder) Statements() []prometheus.CapturedStatement {
r.mu.Lock()
defer r.mu.Unlock()
out := make([]prometheus.CapturedStatement, len(r.statements))
copy(out, r.statements)
return out
}
type captureQueryable struct {
client *client
recorder *statementRecorder
}
func (c *captureQueryable) Querier(mint, maxt int64) (storage.Querier, error) {
return &captureQuerier{
querier: querier{mint: mint, maxt: maxt, client: c.client},
recorder: c.recorder,
}, nil
}
// captureQuerier builds the same SQL as the live querier but records it and
// returns no data. The fingerprint filter always takes the subquery form:
// without executing the series lookup, the inline literal set is unknown.
type captureQuerier struct {
querier
recorder *statementRecorder
}
func (c *captureQuerier) Select(ctx context.Context, _ bool, hints *storage.SelectHints, matchers ...*labels.Matcher) storage.SeriesSet {
start, end := c.window(hints)
samplesQuery, args, err := buildSamplesQuery(start, end, metricNamesFromMatchers(matchers), matchers, c.lastSamplePerStepFor(ctx, hints))
if err != nil {
return storage.ErrSeriesSet(err)
}
c.recorder.record(samplesQuery, args)
return storage.EmptySeriesSet()
}
func (c *captureQuerier) LabelValues(context.Context, string, *storage.LabelHints, ...*labels.Matcher) ([]string, annotations.Annotations, error) {
return nil, nil, nil
}
func (c *captureQuerier) LabelNames(context.Context, *storage.LabelHints, ...*labels.Matcher) ([]string, annotations.Annotations, error) {
return nil, nil, nil
}
// metricNamesFromMatchers extracts the statically known metric name, if any.
// The live path derives names from the matched series; the capture path has
// no execution results, so only a __name__ equality contributes.
func metricNamesFromMatchers(matchers []*labels.Matcher) []string {
for _, m := range matchers {
if m.Name == metricNameLabel && m.Type == labels.MatchEqual && m.Value != "" {
return []string{m.Value}
}
}
return nil
}

View File

@@ -1,180 +0,0 @@
package clickhouseprometheusv2
import (
"context"
"encoding/json"
"log/slog"
"math"
"slices"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/prometheus/prometheus/model/labels"
promValue "github.com/prometheus/prometheus/model/value"
)
// seriesLookup holds a series lookup's result: matched fingerprints with
// their labels, and the distinct metric names seen on them.
type seriesLookup struct {
fingerprints map[uint64]labels.Labels
metricNames []string
}
// client executes the series, samples and raw queries against ClickHouse.
type client struct {
settings factory.ScopedProviderSettings
telemetryStore telemetrystore.TelemetryStore
lookbackMs int64
}
func newClient(settings factory.ScopedProviderSettings, telemetryStore telemetrystore.TelemetryStore, cfg prometheus.Config) *client {
lookback := cfg.LookbackDelta
if lookback <= 0 {
// Mirror the engine: promql defaults an unset lookback to 5m.
lookback = defaultLookbackDelta
}
return &client{
settings: settings,
telemetryStore: telemetryStore,
lookbackMs: lookback.Milliseconds(),
}
}
func (c *client) withContext(ctx context.Context, functionName string) context.Context {
return ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalMetrics.StringValue(),
instrumentationtypes.CodeNamespace: "clickhouse-prometheus-v2",
instrumentationtypes.CodeFunctionName: functionName,
})
}
func (c *client) selectSeries(ctx context.Context, query string, args []any) (*seriesLookup, error) {
ctx = c.withContext(ctx, "selectSeries")
rows, err := c.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
lookup := &seriesLookup{fingerprints: make(map[uint64]labels.Labels)}
names := make(map[string]struct{})
var fingerprint uint64
var labelsJSON string
for rows.Next() {
if err := rows.Scan(&fingerprint, &labelsJSON); err != nil {
return nil, err
}
lset, err := unmarshalLabels(labelsJSON)
if err != nil {
return nil, err
}
lookup.fingerprints[fingerprint] = lset
if name := lset.Get(metricNameLabel); name != "" {
names[name] = struct{}{}
}
}
if err := rows.Err(); err != nil {
return nil, err
}
for name := range names {
lookup.metricNames = append(lookup.metricNames, name)
}
slices.Sort(lookup.metricNames)
return lookup, nil
}
// unmarshalLabels parses the labels JSON column, dropping empty-valued
// labels: empty means "absent" in Prometheus, but stored attribute JSON can
// carry them.
func unmarshalLabels(s string) (labels.Labels, error) {
m := make(map[string]string)
if err := json.Unmarshal([]byte(s), &m); err != nil {
return labels.EmptyLabels(), err
}
builder := labels.NewScratchBuilder(len(m))
for k, v := range m {
if v == "" {
continue
}
builder.Add(k, v)
}
builder.Sort()
return builder.Labels(), nil
}
// selectSamples assembles per-series sample slices from a samples query (raw
// or last-sample-per-step; same column shape), whose rows arrive ordered by
// (fingerprint, unix_milli). Fingerprints missing from the lookup are skipped:
// the samples query's semi-join re-runs the series predicates and can match
// series registered after the lookup ran — the lookup is the read snapshot.
// Stale flags map to the engine's StaleNaN. Duplicate
// timestamps pass through: uniqueness is ingest's job, and v1 feeds them to
// the engine as-is over the same dirty data.
func (c *client) selectSamples(ctx context.Context, query string, args []any, lookup *seriesLookup) ([]*series, error) {
ctx = c.withContext(ctx, "selectSamples")
rows, err := c.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var (
result []*series
current *series
fingerprint uint64
prevFp uint64
timestampMs int64
val float64
flags uint32
first = true
haveCurrent bool
staleMarker = math.Float64frombits(promValue.StaleNaN)
unknownCount int
)
for rows.Next() {
if err := rows.Scan(&fingerprint, &timestampMs, &val, &flags); err != nil {
return nil, err
}
if first || fingerprint != prevFp {
first = false
prevFp = fingerprint
lset, ok := lookup.fingerprints[fingerprint]
if !ok {
unknownCount++
haveCurrent = false
continue
}
current = &series{lset: lset}
result = append(result, current)
haveCurrent = true
}
if !haveCurrent {
continue
}
if flags&1 == 1 {
val = staleMarker
}
current.ts = append(current.ts, timestampMs)
current.vs = append(current.vs, val)
}
if err := rows.Err(); err != nil {
return nil, err
}
if unknownCount > 0 {
c.settings.Logger().DebugContext(ctx, "skipped samples of fingerprints missing from series lookup",
slog.Int("unknown_fingerprints", unknownCount))
}
return result, nil
}

View File

@@ -1,71 +0,0 @@
package clickhouseprometheusv2
import (
"context"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/prometheus/prometheus/storage"
)
// provider ties the package together: its own engine and parser, and the
// ClickHouse client behind the native storage.Querier. It stays unexported:
// callers hold the prometheus.Prometheus interface, which is the boundary
// between the two provider implementations.
type provider struct {
settings factory.ScopedProviderSettings
engine *prometheus.Engine
parser prometheus.Parser
client *client
}
var (
_ prometheus.Prometheus = (*provider)(nil)
_ prometheus.StatementCapturer = (*provider)(nil)
)
func NewFactory(telemetryStore telemetrystore.TelemetryStore) factory.ProviderFactory[prometheus.Prometheus, prometheus.Config] {
return factory.NewProviderFactory(factory.MustNewName("clickhousev2"), func(ctx context.Context, providerSettings factory.ProviderSettings, config prometheus.Config) (prometheus.Prometheus, error) {
return New(ctx, providerSettings, config, telemetryStore)
})
}
func New(_ context.Context, providerSettings factory.ProviderSettings, config prometheus.Config, telemetryStore telemetrystore.TelemetryStore) (prometheus.Prometheus, error) {
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/prometheus/clickhouseprometheusv2")
engine := prometheus.NewEngine(settings.Logger(), config)
parser := prometheus.NewParser()
client := newClient(settings, telemetryStore, config)
return &provider{
settings: settings,
engine: engine,
parser: parser,
client: client,
}, nil
}
func (p *provider) Engine() *prometheus.Engine {
return p.engine
}
func (p *provider) Parser() prometheus.Parser {
return p.parser
}
func (p *provider) Storage() storage.Queryable {
return p
}
func (p *provider) Querier(mint, maxt int64) (storage.Querier, error) {
return &querier{mint: mint, maxt: maxt, client: p.client}, nil
}
// CapturingStorage implements prometheus.StatementCapturer: a storage that
// records each selector's SQL without executing it, for the preview path.
// A fresh recorder per call keeps concurrent dry-runs isolated.
func (p *provider) CapturingStorage() (storage.Queryable, prometheus.StatementRecorder) {
recorder := &statementRecorder{}
return &captureQueryable{client: p.client, recorder: recorder}, recorder
}

View File

@@ -1,171 +0,0 @@
package clickhouseprometheusv2
import (
"context"
"fmt"
"slices"
"time"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/huandu/go-sqlbuilder"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/storage"
"github.com/prometheus/prometheus/util/annotations"
)
// defaultLookbackDelta mirrors promql's default when the config leaves the
// lookback unset; the engine and the storage must agree on it for
// last-sample-per-step bucket anchoring.
const defaultLookbackDelta = 5 * time.Minute
// querier is a native storage.Querier over ClickHouse: Select builds SQL
// directly from the matchers and hints, with no remote-read protobuf layer.
type querier struct {
mint, maxt int64
client *client
}
var _ storage.Querier = (*querier)(nil)
func (q *querier) Select(ctx context.Context, sortSeries bool, hints *storage.SelectHints, matchers ...*labels.Matcher) storage.SeriesSet {
start, end := q.window(hints)
seriesQuery, seriesArgs, err := buildSeriesQuery(start, end, matchers)
if err != nil {
return storage.ErrSeriesSet(err)
}
lookup, err := q.client.selectSeries(ctx, seriesQuery, seriesArgs)
if err != nil {
return storage.ErrSeriesSet(err)
}
if len(lookup.fingerprints) == 0 {
return storage.EmptySeriesSet()
}
list, err := q.fetchSamples(ctx, start, end, matchers, lookup, q.lastSamplePerStepFor(ctx, hints))
if err != nil {
return storage.ErrSeriesSet(err)
}
// The engine assumes storages never emit duplicate label sets.
list = sortAndMerge(list)
return newSeriesSet(list)
}
func (q *querier) LabelValues(ctx context.Context, name string, hints *storage.LabelHints, matchers ...*labels.Matcher) ([]string, annotations.Annotations, error) {
sb := sqlbuilder.NewSelectBuilder()
if name == metricNameLabel {
sb.Select("DISTINCT metric_name AS value")
} else {
sb.Select(fmt.Sprintf("DISTINCT JSONExtractString(labels, %s) AS value", sb.Var(name)))
}
adjustedStart, _, table, _ := metricstelemetryschema.WhichTSTableToUse(uint64(q.mint), uint64(q.maxt), false, nil)
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, table))
if err := applySeriesConditions(sb, int64(adjustedStart), q.maxt, matchers); err != nil {
return nil, nil, err
}
sb.Where("value != ''")
if hints != nil && hints.Limit > 0 {
sb.Limit(hints.Limit)
}
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
values, err := q.selectStrings(ctx, "LabelValues", query, args)
if err != nil {
return nil, nil, err
}
slices.Sort(values)
return values, nil, nil
}
func (q *querier) LabelNames(ctx context.Context, hints *storage.LabelHints, matchers ...*labels.Matcher) ([]string, annotations.Annotations, error) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("DISTINCT arrayJoin(JSONExtractKeys(labels)) AS name")
adjustedStart, _, table, _ := metricstelemetryschema.WhichTSTableToUse(uint64(q.mint), uint64(q.maxt), false, nil)
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, table))
if err := applySeriesConditions(sb, int64(adjustedStart), q.maxt, matchers); err != nil {
return nil, nil, err
}
if hints != nil && hints.Limit > 0 {
sb.Limit(hints.Limit)
}
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
names, err := q.selectStrings(ctx, "LabelNames", query, args)
if err != nil {
return nil, nil, err
}
slices.Sort(names)
return names, nil, nil
}
func (q *querier) Close() error {
return nil
}
// window returns the per-selector fetch window from the hints (already
// adjusted for offset, @, range and lookback); mint/maxt span the union of
// all selectors and are the fallback.
func (q *querier) window(hints *storage.SelectHints) (int64, int64) {
if hints != nil && hints.Start != 0 && hints.End != 0 && hints.Start <= hints.End {
return hints.Start, hints.End
}
return q.mint, q.maxt
}
// lastSamplePerStepFor decides whether the fetch can keep only the last
// sample per step bucket (see lastSamplePerStep). Only instant selectors
// (hints.Range == 0) of subquery-free queries qualify: range selectors need
// every raw sample, and subquery selectors evaluate at the subquery's own
// step while hints carry the top-level step — the subquery-free proof
// arrives as QueryTraits in the context. The first evaluation timestamp is
// recovered as hints.Start + lookback - 1ms, inverting how the engine
// derives hints.Start.
func (q *querier) lastSamplePerStepFor(ctx context.Context, hints *storage.SelectHints) *lastSamplePerStep {
if hints == nil || hints.Range != 0 || hints.Start <= 0 {
return nil
}
traits, ok := prometheus.QueryTraitsFromContext(ctx)
if !ok || !traits.SubqueryFree {
return nil
}
firstEval := hints.Start + q.client.lookbackMs - 1
if firstEval > hints.End {
// Defensive: never anchor a bucket past the window.
firstEval = hints.End
}
return &lastSamplePerStep{firstEvalMs: firstEval, stepMs: hints.Step}
}
// fetchSamples runs the samples query for the matched series (see
// buildSamplesQuery).
func (q *querier) fetchSamples(ctx context.Context, start, end int64, matchers []*labels.Matcher, lookup *seriesLookup, lastPerStep *lastSamplePerStep) ([]*series, error) {
query, args, err := buildSamplesQuery(start, end, lookup.metricNames, matchers, lastPerStep)
if err != nil {
return nil, err
}
return q.client.selectSamples(ctx, query, args, lookup)
}
func (q *querier) selectStrings(ctx context.Context, fn, query string, args []any) ([]string, error) {
ctx = q.client.withContext(ctx, fn)
rows, err := q.client.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var out []string
var v string
for rows.Next() {
if err := rows.Scan(&v); err != nil {
return nil, err
}
out = append(out, v)
}
if err := rows.Err(); err != nil {
return nil, err
}
return out, nil
}

View File

@@ -1,184 +0,0 @@
package clickhouseprometheusv2
import (
"sort"
"github.com/prometheus/prometheus/model/histogram"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/storage"
"github.com/prometheus/prometheus/tsdb/chunkenc"
"github.com/prometheus/prometheus/util/annotations"
)
// series is one time series with samples as parallel slices ordered by
// timestamp. Deliberately not storage.NewListSeries: that boxes every sample
// as an interface value, a per-sample allocation this fetch path exists to
// avoid.
type series struct {
lset labels.Labels
ts []int64
vs []float64
}
var _ storage.Series = (*series)(nil)
func (s *series) Labels() labels.Labels {
return s.lset
}
func (s *series) Iterator(it chunkenc.Iterator) chunkenc.Iterator {
if fit, ok := it.(*floatIterator); ok {
fit.reset(s)
return fit
}
fit := &floatIterator{}
fit.reset(s)
return fit
}
// floatIterator implements chunkenc.Iterator over a series' sample slices.
type floatIterator struct {
s *series
i int
}
var _ chunkenc.Iterator = (*floatIterator)(nil)
func (it *floatIterator) reset(s *series) {
it.s = s
it.i = -1
}
func (it *floatIterator) Next() chunkenc.ValueType {
it.i++
if it.i >= len(it.s.ts) {
return chunkenc.ValNone
}
return chunkenc.ValFloat
}
func (it *floatIterator) Seek(t int64) chunkenc.ValueType { //nolint:govet // stdmethods flags io.Seeker; this is chunkenc.Iterator's Seek
if it.i < 0 {
it.i = 0
}
if it.i >= len(it.s.ts) {
return chunkenc.ValNone
}
// The current position, once valid, must not move backwards.
if it.s.ts[it.i] >= t {
return chunkenc.ValFloat
}
it.i += sort.Search(len(it.s.ts)-it.i, func(j int) bool {
return it.s.ts[it.i+j] >= t
})
if it.i >= len(it.s.ts) {
return chunkenc.ValNone
}
return chunkenc.ValFloat
}
func (it *floatIterator) At() (int64, float64) {
return it.s.ts[it.i], it.s.vs[it.i]
}
func (it *floatIterator) AtHistogram(*histogram.Histogram) (int64, *histogram.Histogram) {
return 0, nil
}
func (it *floatIterator) AtFloatHistogram(*histogram.FloatHistogram) (int64, *histogram.FloatHistogram) {
return 0, nil
}
func (it *floatIterator) AtT() int64 {
return it.s.ts[it.i]
}
// AtST returns the current start timestamp; not tracked by this storage.
func (it *floatIterator) AtST() int64 {
return 0
}
func (it *floatIterator) Err() error {
return nil
}
// seriesSet iterates a fully materialized, label-sorted list of series.
type seriesSet struct {
series []*series
i int
}
var _ storage.SeriesSet = (*seriesSet)(nil)
func newSeriesSet(list []*series) *seriesSet {
return &seriesSet{series: list, i: -1}
}
func (s *seriesSet) Next() bool {
s.i++
return s.i < len(s.series)
}
func (s *seriesSet) At() storage.Series {
return s.series[s.i]
}
func (s *seriesSet) Err() error {
return nil
}
func (s *seriesSet) Warnings() annotations.Annotations {
return nil
}
// sortAndMerge orders series by label set and merges identical label sets
// by timestamp (first sample wins ties): distinct fingerprints can carry
// identical label sets, and the engine assumes storages never emit
// duplicates.
func sortAndMerge(list []*series) []*series {
if len(list) < 2 {
return list
}
sort.Slice(list, func(i, j int) bool {
return labels.Compare(list[i].lset, list[j].lset) < 0
})
out := list[:1]
for _, s := range list[1:] {
last := out[len(out)-1]
if labels.Compare(last.lset, s.lset) != 0 {
out = append(out, s)
continue
}
merged := mergeSamples(last, s)
out[len(out)-1] = merged
}
return out
}
func mergeSamples(a, b *series) *series {
ts := make([]int64, 0, len(a.ts)+len(b.ts))
vs := make([]float64, 0, len(a.ts)+len(b.ts))
i, j := 0, 0
for i < len(a.ts) && j < len(b.ts) {
switch {
case a.ts[i] < b.ts[j]:
ts = append(ts, a.ts[i])
vs = append(vs, a.vs[i])
i++
case a.ts[i] > b.ts[j]:
ts = append(ts, b.ts[j])
vs = append(vs, b.vs[j])
j++
default:
ts = append(ts, a.ts[i])
vs = append(vs, a.vs[i])
i++
j++
}
}
ts = append(ts, a.ts[i:]...)
vs = append(vs, a.vs[i:]...)
ts = append(ts, b.ts[j:]...)
vs = append(vs, b.vs[j:]...)
return &series{lset: a.lset, ts: ts, vs: vs}
}

View File

@@ -1,172 +0,0 @@
package clickhouseprometheusv2
import (
"fmt"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/huandu/go-sqlbuilder"
"github.com/prometheus/prometheus/model/labels"
)
// metricNameLabel is the reserved PromQL label holding the metric name.
const metricNameLabel = "__name__"
// buildSeriesQuery renders the series lookup: one row per matched fingerprint
// with its labels.
func buildSeriesQuery(start, end int64, matchers []*labels.Matcher) (string, []any, error) {
// The series tables hold one row per (fingerprint, bucket) at 1h/6h/1d/1w
// granularities; the schema package picks the table whose bucket fits the
// window and rounds the start down to the bucket boundary, so a window
// beginning mid-bucket still matches the bucket's row.
adjustedStart, _, table, _ := metricstelemetryschema.WhichTSTableToUse(uint64(start), uint64(end), false, nil)
sb := sqlbuilder.NewSelectBuilder()
sb.Select("fingerprint", "any(labels)")
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, table))
if err := applySeriesConditions(sb, int64(adjustedStart), end, matchers); err != nil {
return "", nil, err
}
sb.GroupBy("fingerprint")
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return query, args, nil
}
// buildSamplesQuery renders the samples fetch for the matched series: a
// semi-join re-runs the series predicates against the shard-local series
// table (complete by fingerprint co-locality, see localTimeSeriesTable — a
// GLOBAL broadcast would ship the matched set to every shard, and ClickHouse
// materializes the subquery's set per shard before the scan, so it still
// engages the fingerprint primary-key column). metricNames (observed on the
// matched series when the selector had no __name__ equality) narrows the
// primary-key scan. A non-nil lastPerStep groups to the last sample per
// step bucket.
func buildSamplesQuery(start, end int64, metricNames []string, matchers []*labels.Matcher, lastPerStep *lastSamplePerStep) (string, []any, error) {
sb := sqlbuilder.NewSelectBuilder()
if lastPerStep != nil {
// Aliases must not shadow source columns: ClickHouse resolves aliases
// in WHERE too, and "max(unix_milli) AS unix_milli" would put an
// aggregate into the WHERE clause (error 184).
sb.Select("fingerprint", "max(unix_milli) AS ts", "argMax(value, unix_milli) AS val", "argMax(flags, unix_milli) AS fl")
} else {
sb.Select("fingerprint", "unix_milli", "value", "flags")
}
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName))
switch len(metricNames) {
case 0:
// No name constraint derivable; the primary-key prefix goes unused.
case 1:
sb.Where(sb.EQ("metric_name", metricNames[0]))
default:
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
}
// Semantically redundant (the fingerprints already come from these
// temporalities) but engages the leading primary-key column.
sb.Where("temporality IN ['Cumulative', 'Unspecified']")
sub := sqlbuilder.NewSelectBuilder()
sub.Select("fingerprint")
adjustedStart, _, _, localTable := metricstelemetryschema.WhichTSTableToUse(uint64(start), uint64(end), false, nil)
sub.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTable))
if err := applySeriesConditions(sub, int64(adjustedStart), end, matchers); err != nil {
return "", nil, err
}
sb.Where(sb.In("fingerprint", sub))
sb.Where(sb.GTE("unix_milli", start), sb.LTE("unix_milli", end))
if lastPerStep != nil {
sb.GroupBy("fingerprint")
if expr := lastPerStep.bucketExpr(); expr != "" {
sb.GroupBy(expr)
}
sb.OrderBy("fingerprint", "ts")
} else {
sb.OrderBy("fingerprint", "unix_milli")
}
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
return query, args, nil
}
// applySeriesConditions adds the WHERE conditions of a series table scan:
// __name__ matchers (all four types) translate to the metric_name column,
// every other matcher to a JSONExtractString condition on the labels column.
// An equality matcher against "" matches series without the label, mirroring
// PromQL, because JSONExtractString returns "" for missing keys. Regexes are
// anchored: PromQL matchers match the whole value, ClickHouse match()
// searches for a substring.
func applySeriesConditions(sb *sqlbuilder.SelectBuilder, start, end int64, matchers []*labels.Matcher) error {
for _, m := range matchers {
if m.Name != metricNameLabel {
continue
}
switch m.Type {
case labels.MatchEqual:
sb.Where(sb.EQ("metric_name", m.Value))
case labels.MatchNotEqual:
sb.Where(sb.NE("metric_name", m.Value))
case labels.MatchRegexp:
sb.Where(fmt.Sprintf("match(metric_name, %s)", sb.Var(anchorRegex(m.Value))))
case labels.MatchNotRegexp:
sb.Where(fmt.Sprintf("NOT match(metric_name, %s)", sb.Var(anchorRegex(m.Value))))
default:
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported matcher type %q for __name__", m.Type)
}
}
sb.Where("temporality IN ['Cumulative', 'Unspecified']")
// Inclusive upper bound: registration rows are hour-floored (and 6h/1d/1w
// for the rollup tables) by the exporter, so a series first registered in
// the bucket starting exactly at `end` would otherwise be invisible while
// its samples (<= end) are in range.
sb.Where(sb.GTE("unix_milli", start), sb.LTE("unix_milli", end))
for _, m := range matchers {
if m.Name == metricNameLabel {
continue
}
switch m.Type {
case labels.MatchEqual:
sb.Where(fmt.Sprintf("JSONExtractString(labels, %s) = %s", sb.Var(m.Name), sb.Var(m.Value)))
case labels.MatchNotEqual:
sb.Where(fmt.Sprintf("JSONExtractString(labels, %s) != %s", sb.Var(m.Name), sb.Var(m.Value)))
case labels.MatchRegexp:
sb.Where(fmt.Sprintf("match(JSONExtractString(labels, %s), %s)", sb.Var(m.Name), sb.Var(anchorRegex(m.Value))))
case labels.MatchNotRegexp:
sb.Where(fmt.Sprintf("NOT match(JSONExtractString(labels, %s), %s)", sb.Var(m.Name), sb.Var(anchorRegex(m.Value))))
default:
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported matcher type %q", m.Type)
}
}
return nil
}
// anchorRegex turns a PromQL regex into its fully-anchored form (see
// applySeriesConditions).
func anchorRegex(v string) string {
return "^(?:" + v + ")$"
}
// lastSamplePerStep reduces an instant-selector fetch to the last sample of
// each step bucket: bucket 0 is (start, firstEval], bucket i is
// (firstEval+(i-1)·step, firstEval+i·step]. Anchoring buckets at the first
// evaluation timestamp makes every bucket boundary an evaluation timestamp,
// so a non-final sample of a bucket can never be the latest sample in any
// (t-lookback, t] the engine resolves — the reduction is lossless. Real
// timestamps are preserved, so the engine's own lookback and staleness
// handling remain exact.
type lastSamplePerStep struct {
firstEvalMs int64
stepMs int64
}
func (t *lastSamplePerStep) bucketExpr() string {
if t.stepMs <= 0 {
// Instant query: a single evaluation at firstEval; one bucket.
return ""
}
return fmt.Sprintf(
"if(unix_milli <= %d, 0, intDiv(unix_milli - %d - 1, %d) + 1)",
t.firstEvalMs, t.firstEvalMs, t.stepMs,
)
}

View File

@@ -1,115 +0,0 @@
package clickhouseprometheusv2
import (
"testing"
"time"
"github.com/prometheus/prometheus/model/labels"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func mustMatcher(t *testing.T, mt labels.MatchType, name, value string) *labels.Matcher {
t.Helper()
m, err := labels.NewMatcher(mt, name, value)
require.NoError(t, err)
return m
}
func TestBuildSeriesQuery(t *testing.T) {
start := int64(1_700_000_000_000)
end := start + time.Hour.Milliseconds()
// The series table window rounds down to the table's bucket boundary.
adjustedStart := start - (start % time.Hour.Milliseconds())
t.Run("equality name and label matchers", func(t *testing.T) {
query, args, err := buildSeriesQuery(start, end, []*labels.Matcher{
mustMatcher(t, labels.MatchEqual, "__name__", "http_requests_total"),
mustMatcher(t, labels.MatchEqual, "job", "api"),
})
require.NoError(t, err)
assert.Equal(t,
"SELECT fingerprint, any(labels) FROM signoz_metrics.distributed_time_series_v4 WHERE metric_name = ? AND temporality IN ['Cumulative', 'Unspecified'] AND unix_milli >= ? AND unix_milli <= ? AND JSONExtractString(labels, ?) = ? GROUP BY fingerprint",
query,
)
assert.Equal(t, []any{"http_requests_total", adjustedStart, end, "job", "api"}, args)
})
t.Run("regex matchers are anchored", func(t *testing.T) {
_, args, err := buildSeriesQuery(start, end, []*labels.Matcher{
mustMatcher(t, labels.MatchEqual, "__name__", "up"),
mustMatcher(t, labels.MatchRegexp, "instance", "prod.*"),
mustMatcher(t, labels.MatchNotRegexp, "env", "dev|test"),
})
require.NoError(t, err)
assert.Equal(t, []any{"up", adjustedStart, end, "instance", "^(?:prod.*)$", "env", "^(?:dev|test)$"}, args)
})
t.Run("regex name matcher uses metric_name column", func(t *testing.T) {
query, args, err := buildSeriesQuery(start, end, []*labels.Matcher{
mustMatcher(t, labels.MatchRegexp, "__name__", "node_cpu.*|node_memory.*"),
})
require.NoError(t, err)
assert.Contains(t, query, "match(metric_name, ?)")
assert.NotContains(t, query, "JSONExtractString")
assert.Equal(t, []any{"^(?:node_cpu.*|node_memory.*)$", adjustedStart, end}, args)
})
t.Run("no name matcher omits metric_name condition", func(t *testing.T) {
query, _, err := buildSeriesQuery(start, end, []*labels.Matcher{
mustMatcher(t, labels.MatchEqual, "job", "api"),
})
require.NoError(t, err)
assert.NotContains(t, query, "metric_name")
})
}
func TestBuildSamplesQuery(t *testing.T) {
start := int64(1_700_000_000_000)
end := start + time.Hour.Milliseconds()
adjustedStart := start - (start % time.Hour.Milliseconds())
matchers := []*labels.Matcher{
mustMatcher(t, labels.MatchEqual, "__name__", "up"),
mustMatcher(t, labels.MatchEqual, "job", "api"),
}
t.Run("raw fetch filters by a shard-local semi-join", func(t *testing.T) {
query, args, err := buildSamplesQuery(start, end, []string{"up"}, matchers, nil)
require.NoError(t, err)
assert.Contains(t, query, "fingerprint IN (SELECT fingerprint FROM signoz_metrics.time_series_v4 WHERE ")
assert.NotContains(t, query, "GLOBAL IN")
assert.Contains(t, query, "ORDER BY fingerprint, unix_milli")
// Args follow placeholder order: samples metric name, the semi-join's
// series predicates, then the samples window bounds.
assert.Equal(t, []any{"up", "up", adjustedStart, end, "job", "api", start, end}, args)
})
t.Run("last-sample-per-step groups by step bucket anchored at first eval", func(t *testing.T) {
lastPerStep := &lastSamplePerStep{firstEvalMs: start + 299_999, stepMs: 60_000}
query, _, err := buildSamplesQuery(start, end, []string{"up"}, matchers, lastPerStep)
require.NoError(t, err)
assert.Contains(t, query, "argMax(value, unix_milli) AS val")
assert.Contains(t, query, "argMax(flags, unix_milli) AS fl")
assert.Contains(t, query, "GROUP BY fingerprint, if(unix_milli <= 1700000299999, 0, intDiv(unix_milli - 1700000299999 - 1, 60000) + 1)")
assert.Contains(t, query, "ORDER BY fingerprint, ts")
// Aliases must not shadow the source columns referenced in WHERE.
assert.NotContains(t, query, "AS unix_milli")
assert.NotContains(t, query, "AS value")
assert.NotContains(t, query, "AS flags")
})
t.Run("instant query keeps one bucket", func(t *testing.T) {
lastPerStep := &lastSamplePerStep{firstEvalMs: end, stepMs: 0}
query, _, err := buildSamplesQuery(start, end, []string{"up"}, matchers, lastPerStep)
require.NoError(t, err)
assert.Contains(t, query, "GROUP BY fingerprint ORDER BY fingerprint, ts")
assert.NotContains(t, query, "intDiv")
})
t.Run("multiple metric names from regex selector", func(t *testing.T) {
query, args, err := buildSamplesQuery(start, end, []string{"node_cpu", "node_memory"}, matchers, nil)
require.NoError(t, err)
assert.Contains(t, query, "metric_name IN (?, ?)")
assert.Equal(t, []any{"node_cpu", "node_memory", "up", adjustedStart, end, "job", "api", start, end}, args)
})
}

View File

@@ -1,49 +0,0 @@
package prometheus
import (
"context"
"github.com/prometheus/prometheus/promql/parser"
)
type queryTraitsKey struct{}
// QueryTraits carries per-query facts a storage implementation cannot derive
// from SelectHints alone. Call sites that parse the PromQL expression attach
// traits to the context before handing it to the engine; storages treat a
// missing traits value as "unknown" and stay conservative.
type QueryTraits struct {
// SubqueryFree is true when the query contains no subquery expression.
// Subquery selectors are evaluated at the subquery's own step, but
// SelectHints.Step always carries the top-level step, so step-aligned
// storage optimizations (e.g. keeping only the last sample per step
// bucket) are safe only when this is true.
SubqueryFree bool
}
// DetectQueryTraits derives QueryTraits from a parsed PromQL expression.
func DetectQueryTraits(expr parser.Expr) QueryTraits {
subqueryFree := true
parser.Inspect(expr, func(node parser.Node, _ []parser.Node) error {
if _, ok := node.(*parser.SubqueryExpr); ok {
subqueryFree = false
}
return nil
})
return QueryTraits{SubqueryFree: subqueryFree}
}
// NewContextWithQueryTraits returns a context carrying the given traits.
func NewContextWithQueryTraits(ctx context.Context, traits QueryTraits) context.Context {
return context.WithValue(ctx, queryTraitsKey{}, traits)
}
// QueryTraitsFromContext returns the traits attached to ctx, if any.
//
// Context is used here, unlike for backend selection, because traits must
// cross the promql engine to reach storage.Querier.Select, and the engine's
// interfaces offer no other channel; the alternative is a Prometheus fork.
func QueryTraitsFromContext(ctx context.Context) (QueryTraits, bool) {
traits, ok := ctx.Value(queryTraitsKey{}).(QueryTraits)
return traits, ok
}

View File

@@ -0,0 +1,9 @@
// Package blockkit provides a goldmark extension that renders markdown as
// Slack Block Kit JSON (an array of blocks suitable for the Slack API's
// `blocks` payload field).
//
// The current Slack notifier uses the sibling mrkdwn package instead, because
// Block Kit's hard limits (one table per message, 50 blocks per message,
// 12k-character total text) would silently truncate or reject notifications
// with rich bodies. This package is kept in tree for future use.
package blockkit

View File

@@ -0,0 +1,3 @@
// Package markdownrenderer renders markdown to the formats that alert
// notifications are delivered in: HTML, Slack Block Kit JSON, and Slack mrkdwn.
package markdownrenderer

View File

@@ -0,0 +1,4 @@
// Package mrkdwn provides a goldmark extension that renders markdown as
// Slack's "mrkdwn" text format (the legacy text field used in attachments
// and message payloads).
package mrkdwn

View File

@@ -48,8 +48,8 @@ func (CompareOperator) Enum() []any {
ValueIsBelowLiteral,
ValueIsEqLiteral,
ValueIsNotEqLiteral,
ValueAboveOrEqLiteral,
ValueBelowOrEqLiteral,
// ValueAboveOrEqLiteral,
// ValueBelowOrEqLiteral,
ValueOutsideBoundsLiteral,
}
}

View File

@@ -1,5 +1,3 @@
import subprocess
import pytest
pytest_plugins = [
@@ -34,22 +32,6 @@ pytest_plugins = [
]
def pytest_configure(config: pytest.Config):
if config.getoption("--rebuild"):
if not config.getoption("--reuse"):
raise pytest.UsageError("--rebuild requires --reuse: it replaces the signoz container within an environment that is being reused.")
if config.getoption("--teardown"):
raise pytest.UsageError("--rebuild cannot be combined with --teardown.")
if config.getoption("--clean"):
raise pytest.UsageError("--rebuild cannot be combined with --clean: --clean forces a cold build, which defeats the purpose of --rebuild.")
def pytest_sessionstart(session: pytest.Session):
if session.config.getoption("--clean"):
# The type filter removes only cache mounts, leaving images and layer cache intact.
subprocess.run(["docker", "builder", "prune", "--force", "--filter", "type=exec.cachemount"], check=True)
def pytest_addoption(parser: pytest.Parser):
parser.addoption(
"--reuse",
@@ -63,18 +45,6 @@ def pytest_addoption(parser: pytest.Parser):
default=False,
help="Teardown environment. Run pytest --basetemp=./tmp/ -vv --teardown src/bootstrap/setup::test_teardown to teardown your local dev environment.",
)
parser.addoption(
"--rebuild",
action="store_true",
default=False,
help="Rebuild the signoz container from the current sources while reusing the rest of the stack (databases, mocks, migrations). Only meaningful together with --reuse: pytest --basetemp=./tmp/ -vv --reuse --rebuild integration/bootstrap/setup.py::test_setup.",
)
parser.addoption(
"--clean",
action="store_true",
default=False,
help="Prune the BuildKit cache mounts (go build and module caches) used by the signoz image build, forcing the next build to start cold. Combine with --teardown to reset everything: pytest --basetemp=./tmp/ -vv --teardown --clean integration/bootstrap/setup.py::test_teardown.",
)
parser.addoption(
"--with-web",
action="store_true",

View File

@@ -41,7 +41,6 @@ def wrap( # pylint: disable=too-many-arguments,too-many-positional-arguments
create: Callable[[], T],
delete: Callable[[T], None],
restore: Callable[[dict], T],
rebuild: bool = False,
) -> T:
"""
Wraps a resource creation and cleanup process with reuse and teardown options.
@@ -52,7 +51,6 @@ def wrap( # pylint: disable=too-many-arguments,too-many-positional-arguments
- create: function to create the resource
- delete: function to delete the resource
- restore: function to restore resource from cache
- rebuild: under --reuse, delete the cached resource and recreate it instead of restoring it
"""
resource = empty()
@@ -60,13 +58,8 @@ def wrap( # pylint: disable=too-many-arguments,too-many-positional-arguments
existing_resource = pytestconfig.cache.get(key, None)
if existing_resource:
assert isinstance(existing_resource, dict)
if rebuild:
logger.info("Rebuilding %s(%s), removing the existing one", key, existing_resource)
delete(restore(existing_resource))
pytestconfig.cache.set(key, None)
else:
logger.info("Reusing existing %s(%s)", key, existing_resource)
return restore(existing_resource)
logger.info("Reusing existing %s(%s)", key, existing_resource)
return restore(existing_resource)
if not teardown(request):
resource = create()

View File

@@ -1,6 +1,4 @@
import os
import platform
import subprocess
import time
from http import HTTPStatus
from os import path
@@ -10,6 +8,7 @@ import docker.errors
import pytest
import requests
from testcontainers.core.container import DockerContainer, Network
from testcontainers.core.image import DockerImage
from fixtures import reuse, types
from fixtures.logger import setup_logger
@@ -51,28 +50,18 @@ def create_signoz(
# Docker build context is the repo root — one up from pytest's
# rootdir (tests/).
context = pytestconfig.rootpath.parent
# The docker CLI is required: the Dockerfiles use BuildKit cache
# mounts, which docker-py does not support.
subprocess.run(
[
"docker",
"build",
"--file",
str(context / dockerfile_path),
"--tag",
"signoz:integration",
"--build-arg",
f"TARGETARCH={arch}",
"--build-arg",
f"ZEUSURL={zeus.container_configs['8080'].base()}",
str(context),
],
check=True,
env=os.environ | {"DOCKER_BUILDKIT": "1"},
self = DockerImage(
path=str(pytestconfig.rootpath.parent),
dockerfile_path=dockerfile_path,
tag="signoz:integration",
buildargs={
"TARGETARCH": arch,
"ZEUSURL": zeus.container_configs["8080"].base(),
},
)
self.build()
env = (
{
"SIGNOZ_WEB_ENABLED": False,
@@ -211,7 +200,6 @@ def create_signoz(
create=create,
delete=delete,
restore=restore,
rebuild=pytestconfig.getoption("--rebuild"),
)