mirror of
https://github.com/SigNoz/signoz.git
synced 2026-08-01 18:50:35 +01:00
Compare commits
2 Commits
ci/testsci
...
claude/int
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6cdc5dbb74 | ||
|
|
b66317d025 |
212
.claude/skills/frontend-testing/SKILL.md
Normal file
212
.claude/skills/frontend-testing/SKILL.md
Normal file
@@ -0,0 +1,212 @@
|
||||
---
|
||||
name: frontend-testing
|
||||
description: Guide for writing and updating SigNoz frontend unit and integration (component) tests with Jest + React Testing Library. Use whenever you are creating a `*.test.ts`/`*.test.tsx` file, adding coverage for a component, hook, util, or provider under `frontend/src`, mocking an API for a component test, or fixing/optimizing an existing frontend test. Covers the custom `tests/test-utils` render, the MSW mock server, the nuqs testing adapter, `renderHook`, data-testid conventions, and the speed/flake pitfalls specific to this codebase.
|
||||
---
|
||||
|
||||
# SigNoz frontend testing
|
||||
|
||||
Write frontend tests the way this repo already does. **Match the nearest existing test** in the same folder before inventing a pattern — consistency beats cleverness. This guide encodes the conventions so a new test looks like it was always there.
|
||||
|
||||
Everything below is scoped to `frontend/` (package manager is **pnpm**; the E2E Playwright suite under `tests/e2e/` is a separate stack — see `docs/contributing/tests/e2e.md`, not this skill).
|
||||
|
||||
## The two kinds of test in this repo
|
||||
|
||||
| | **Unit** | **Integration (component)** |
|
||||
|---|---|---|
|
||||
| Target | pure functions, hooks, reducers, selectors | a component / container rendered with its providers |
|
||||
| Render | none, or `renderHook` | the custom `render` from `tests/test-utils` |
|
||||
| API | not involved | **MSW** mock server (`mocks-server/`) |
|
||||
| Assert on | return value / state | rendered DOM + user interactions |
|
||||
|
||||
Both live beside the code they test, never in a separate top-level tree.
|
||||
|
||||
## Decision checklist (read before writing)
|
||||
|
||||
1. **Is there a sibling test?** `ls` the folder for `__tests__/`, `__test__/`, or `tests/` and a `*.test.tsx`. If one exists, copy its imports and structure. Folder naming is inconsistent across the repo — follow the folder you are in, don't rename it.
|
||||
2. **Unit or integration?** Use the table above.
|
||||
3. **Does it hit an API?** If yes, you need MSW (see [API mocking](#api-mocking-msw)). Never let a component fire a real request.
|
||||
4. **Does it read/write the URL (`nuqs`)?** If yes, use the nuqs testing adapter (see [URL state](#url-state-nuqs)).
|
||||
5. **What's the query?** Prefer `data-testid` (see [Queries](#queries--assertions)). If the component lacks one on a critical control, add it to the component — `frontend/CLAUDE.md` requires testids on inputs/buttons.
|
||||
|
||||
## File placement & naming
|
||||
|
||||
- Co-locate: `Foo.tsx` → `Foo.test.tsx` next to it, or under a `__tests__/` folder beside it. Match whatever the sibling files do.
|
||||
- `testMatch` is `src/**/*(*.)test.(ts|js)?(x)` — the file **must** end in `.test.ts`/`.test.tsx`.
|
||||
- Name integration specs that render a subtree `*.integration.test.tsx` when the folder already uses that suffix (e.g. `NewSelect/__test__/VariableItem.integration.test.tsx`); otherwise plain `.test.tsx`.
|
||||
- Shared per-suite render/setup goes in a `_helpers.tsx` or `*.testUtils.tsx` in the same folder — not exported app-wide.
|
||||
|
||||
## Integration tests: the custom `render`
|
||||
|
||||
Import `render` (and `screen`, `userEvent`, matchers) from **`tests/test-utils`**, not from `@testing-library/react` directly. The custom render wraps the component in every provider the app needs — `MemoryRouter`, `NuqsAdapter`, react-query `QueryClientProvider`, a redux mock store, `AppContext`, `Timezone`, `QueryBuilderProvider`, etc. — so a container renders exactly as it does in the app.
|
||||
|
||||
```tsx
|
||||
import { render, screen, userEvent } from 'tests/test-utils';
|
||||
|
||||
import MyPanel from '../MyPanel';
|
||||
|
||||
describe('MyPanel', () => {
|
||||
it('renders the empty state when there is no data', () => {
|
||||
render(<MyPanel />);
|
||||
expect(screen.getByTestId('my-panel-empty')).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Provider overrides (third argument)
|
||||
|
||||
`render(ui, options?, providerProps?)`. The third arg drives the providers:
|
||||
|
||||
```tsx
|
||||
render(<RolesSettings />, undefined, {
|
||||
role: 'VIEWER', // 'ADMIN' (default) | 'EDITOR' | 'VIEWER'
|
||||
initialRoute: '/settings/roles?tab=members', // seeds MemoryRouter + URL
|
||||
appContextOverrides: { // deep-merge over getAppContextMock(role)
|
||||
featureFlags: [{ name: FeatureKeys.GATEWAY, active: false, usage: 0, usage_limit: -1, route: '' }],
|
||||
},
|
||||
queryBuilderOverrides: { /* Partial<QueryBuilderContextType> */ },
|
||||
});
|
||||
```
|
||||
|
||||
- Use `role` to test permission gating (`hasEditPermission` is derived: ADMIN/EDITOR ⇒ true).
|
||||
- Use `appContextOverrides` for licence state, feature flags, user/org, preferences — see `getAppContextMock` in `tests/test-utils.tsx` for the full shape. Import `getAppContextMock` directly when you build a bespoke wrapper (as `ListAlertRules/__tests__/_helpers.tsx` does).
|
||||
- Don't hand-roll a provider stack unless a component needs something the custom render can't express (e.g. `VirtuosoMockContext` for a virtualised list, or a bespoke provider). When you do, model it on `_helpers.tsx` and still reuse `getAppContextMock`.
|
||||
|
||||
### What's already handled for you (don't re-mock)
|
||||
|
||||
The global `jest.setup.ts` and `test-utils` already provide these — reusing them keeps tests fast and consistent:
|
||||
|
||||
- **`react-i18next`** — `t` returns the key verbatim. Assert on keys, not translated copy.
|
||||
- **MSW server** — `server.listen()` / `resetHandlers()` / `close()` run around every test automatically.
|
||||
- **`ResizeObserver`, `IntersectionObserver`, `matchMedia`, `scrollIntoView`, Pointer Capture** — stubbed for jsdom + Radix/Ant.
|
||||
- **System time** pinned to `2023-10-20` (`test-utils` sets it in `beforeEach`). Override per-suite with `jest.setSystemTime(new Date('...'))` when a test needs a specific clock.
|
||||
- Module mocks in `frontend/__mocks__/` and `src/__tests__/` (`safeNavigateMock`, `logEventMock`, icons, css, uplot, motion). Import the shared mock rather than writing your own:
|
||||
```ts
|
||||
import { safeNavigateMock } from '__tests__/safeNavigateMock';
|
||||
import { logEventMock } from '__tests__/logEventMock';
|
||||
```
|
||||
|
||||
## API mocking (MSW)
|
||||
|
||||
APIs are mocked with **MSW**, not by hand-mocking `axios` or react-query. The server is started globally; default handlers live in `src/mocks-server/handlers.ts` with fixtures in `src/mocks-server/__mockdata__/`.
|
||||
|
||||
**Prefer overriding a handler per-test** over `jest.mock`-ing the api module — it exercises the real react-query + api layer:
|
||||
|
||||
```tsx
|
||||
import { server } from 'mocks-server/server';
|
||||
import { rest } from 'msw';
|
||||
import { render, screen } from 'tests/test-utils';
|
||||
|
||||
it('shows an error when the rules API 500s', async () => {
|
||||
server.use(
|
||||
rest.get('*/api/v1/rules', (_req, res, ctx) => res(ctx.status(500))),
|
||||
);
|
||||
render(<ListAlertRules />);
|
||||
expect(await screen.findByText(/something went wrong/i)).toBeInTheDocument();
|
||||
});
|
||||
```
|
||||
|
||||
- Add a reusable happy-path handler + fixture to `handlers.ts` / `__mockdata__/` when several tests need it; use `server.use(...)` for the one-off/error case. `resetHandlers()` after each test wipes per-test overrides automatically.
|
||||
- For an API call the component makes, prefer an MSW handler over mocking the generated hook. Reach for `jest.mock('api/...')` only when the module has no HTTP surface to intercept (pure client-side helper) or the sibling tests already mock it that way.
|
||||
|
||||
## URL state (nuqs)
|
||||
|
||||
Components built on `nuqs` must use the **testing adapter**, not the real one (which throttles writes to `window.history`). Use the helpers in `tests/nuqs-helpers.ts`:
|
||||
|
||||
- `resetNuqsState(initialSearch)` in setup, `onNuqsUrlUpdate` as the adapter's `onUrlUpdate`, and assert with `getCurrentNuqsQueryString()` / `getCurrentNuqsSearchParams()`.
|
||||
- `window.location.search` is **not** authoritative in tests — the adapter keeps state in memory. Assert via the helpers.
|
||||
- The default `render` already installs `NuqsAdapter`; when you build a custom wrapper, use `NuqsTestingAdapter` with `rateLimitFactor={0}` and `hasMemory` (see `_helpers.tsx`).
|
||||
|
||||
## Hook & util tests
|
||||
|
||||
```tsx
|
||||
import { act, renderHook } from '@testing-library/react';
|
||||
import { useThing } from '../useThing';
|
||||
|
||||
it('increments', () => {
|
||||
const { result } = renderHook(() => useThing());
|
||||
act(() => result.current.increment());
|
||||
expect(result.current.count).toBe(1);
|
||||
});
|
||||
```
|
||||
|
||||
- A hook that needs context/providers takes a `wrapper` — build a small local `createWrapper(props)` returning the minimal provider tree (see `QuerySearchV2` context tests). Don't pull in the full app render for a focused hook test.
|
||||
- Pure utils: no render at all — call the function and assert. Keep these fast and exhaustive (edge cases, empty/undefined, boundaries).
|
||||
|
||||
## Queries & assertions
|
||||
|
||||
Locator priority (matches `frontend/CLAUDE.md`):
|
||||
|
||||
1. `getByTestId('...')` / `findByTestId` — **preferred**. If a critical control has no testid, add one to the component.
|
||||
2. `getByRole('button', { name: /save/i })`
|
||||
3. `getByLabelText`, `getByPlaceholderText`
|
||||
4. `getByText` — last resort; brittle against copy edits (and i18n returns keys here).
|
||||
|
||||
Rules:
|
||||
|
||||
- **`findBy*` for anything async** (after a click that fires a request, or portal content). `getBy*` throws immediately — only for content already in the DOM.
|
||||
- **`queryBy*` for absence**: `expect(screen.queryByRole('menu')).not.toBeInTheDocument()`.
|
||||
- Radix/Ant menus and dialogs render in a **portal** — after opening, `await screen.findByRole('menu')`/`'dialog'`, then query items inside it.
|
||||
- Use `@testing-library/jest-dom` matchers (`toBeInTheDocument`, `toBeDisabled`, `toHaveTextContent`) — they're globally available.
|
||||
|
||||
## Interactions
|
||||
|
||||
Prefer `userEvent` over `fireEvent` for real user flows. **Set it up with `delay: null`** so tests don't wait real time between keystrokes:
|
||||
|
||||
```tsx
|
||||
const user = userEvent.setup({ delay: null });
|
||||
await user.click(screen.getByTestId('open-menu'));
|
||||
await user.type(screen.getByRole('textbox'), 'cpu');
|
||||
```
|
||||
|
||||
`fireEvent` is fine for a single low-level event where `userEvent` is overkill.
|
||||
|
||||
## Performance & flake — do / don't
|
||||
|
||||
**Do**
|
||||
|
||||
- `userEvent.setup({ delay: null })` — the single biggest per-test speedup.
|
||||
- `await expect(locator).toBeVisible()` / `await screen.findBy...` to wait on state.
|
||||
- Reuse the shared render, MSW server, and `__mocks__/` — re-mocking heavy modules per file is slow.
|
||||
- Keep pure logic in utils and unit-test it directly instead of asserting it through a full component render.
|
||||
- Scope MSW overrides tightly and rely on the automatic `resetHandlers()`.
|
||||
|
||||
**Don't**
|
||||
|
||||
- ❌ `await new Promise(r => setTimeout(r, 500))` or `waitForTimeout` — poll with `findBy`/`waitFor` instead.
|
||||
- ❌ `it.only` / `describe.only` / `fit` / `fdescribe` — CI and lint forbid `.only`; a stray one silently drops the rest of the suite.
|
||||
- ❌ Leaving `it.skip`/`xit` behind — either fix and enable, or delete it and open an issue. A permanently-skipped test is dead coverage that reads as green.
|
||||
- ❌ `console.log` in committed tests.
|
||||
- ❌ Asserting on translated strings (i18n mock returns the key) or on real timers/dates (clock is pinned).
|
||||
- ❌ Real network — every request must be an MSW handler.
|
||||
|
||||
## Running & verifying
|
||||
|
||||
From `frontend/`:
|
||||
|
||||
```bash
|
||||
pnpm jest path/to/File.test.tsx # one file
|
||||
pnpm jest -t "renders the empty state" # by test name
|
||||
pnpm jest:watch # watch mode while iterating
|
||||
pnpm test:changedsince # only tests changed vs main, with coverage
|
||||
pnpm jest:coverage # full coverage run
|
||||
```
|
||||
|
||||
Before calling a test task done (per `frontend/CLAUDE.md`):
|
||||
|
||||
```bash
|
||||
pnpm tsgo --noEmit # types
|
||||
pnpm oxlint <files> # lint the files you touched, fix all warnings
|
||||
pnpm jest <files> # the tests pass
|
||||
```
|
||||
|
||||
## Author checklist
|
||||
|
||||
- [ ] File ends in `.test.tsx`, co-located, matching the sibling folder's convention.
|
||||
- [ ] Imports `render`/`screen`/`userEvent` from `tests/test-utils` (integration) or uses `renderHook` (hook).
|
||||
- [ ] Every API call is an MSW handler; no real network, no ad-hoc axios mock.
|
||||
- [ ] nuqs components use the testing adapter; URL asserted via `getCurrentNuqs*`.
|
||||
- [ ] Queries use `data-testid` first; async waits use `findBy*`; portals awaited by role.
|
||||
- [ ] `userEvent.setup({ delay: null })`; no `setTimeout`/`waitForTimeout`.
|
||||
- [ ] No `.only`, no stray `.skip`, no `console.log`.
|
||||
- [ ] Assertions cover success **and** the failure/empty/permission path.
|
||||
- [ ] `pnpm tsgo --noEmit`, `pnpm oxlint`, and `pnpm jest` all green on the touched files.
|
||||
24
.github/workflows/e2eci.yaml
vendored
24
.github/workflows/e2eci.yaml
vendored
@@ -75,30 +75,6 @@ jobs:
|
||||
docker rm pw
|
||||
echo "PLAYWRIGHT_BROWSERS_PATH=$RUNNER_TEMP/ms-playwright" >> "$GITHUB_ENV"
|
||||
cd tests/e2e && pnpm playwright install-deps ${{ matrix.project }}
|
||||
# Restore-only: the cacheci workflow owns cache saves. Seeds the
|
||||
# BuildKit cache mounts so the in-test image build is incremental.
|
||||
- name: restore
|
||||
id: restore
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: ${{ runner.temp }}/cacheci
|
||||
key: tests-primary
|
||||
restore-keys: |
|
||||
tests-secondary
|
||||
- name: inject
|
||||
if: steps.restore.outputs.cache-matched-key != ''
|
||||
run: |
|
||||
cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
|
||||
FROM busybox:1.37
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
--mount=type=cache,target=/go/pkg/mod \
|
||||
--mount=type=cache,target=/pnpm/store \
|
||||
--mount=type=bind,target=/restored \
|
||||
tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
|
||||
tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
|
||||
tar -xf /restored/pnpm-store.tar -C /pnpm/store
|
||||
EOF
|
||||
docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
|
||||
- name: bring-up-stack
|
||||
run: |
|
||||
cd tests && \
|
||||
|
||||
24
.github/workflows/integrationci.yaml
vendored
24
.github/workflows/integrationci.yaml
vendored
@@ -110,30 +110,6 @@ jobs:
|
||||
sudo mv chromedriver-linux64/chromedriver /usr/local/bin/chromedriver
|
||||
chromedriver -version
|
||||
google-chrome-stable --version
|
||||
# Restore-only: the cacheci workflow owns cache saves. Seeds the
|
||||
# BuildKit cache mounts so the in-test image build is incremental.
|
||||
- name: restore
|
||||
id: restore
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: ${{ runner.temp }}/cacheci
|
||||
key: tests-primary
|
||||
restore-keys: |
|
||||
tests-secondary
|
||||
- name: inject
|
||||
if: steps.restore.outputs.cache-matched-key != ''
|
||||
run: |
|
||||
cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
|
||||
FROM busybox:1.37
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
--mount=type=cache,target=/go/pkg/mod \
|
||||
--mount=type=cache,target=/pnpm/store \
|
||||
--mount=type=bind,target=/restored \
|
||||
tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
|
||||
tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
|
||||
tar -xf /restored/pnpm-store.tar -C /pnpm/store
|
||||
EOF
|
||||
docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
|
||||
- name: run
|
||||
run: |
|
||||
cd tests && \
|
||||
|
||||
4
Makefile
4
Makefile
@@ -209,8 +209,8 @@ py-lint: ## Run ruff check across the shared tests project
|
||||
@cd tests && uv run ruff check --fix .
|
||||
|
||||
.PHONY: py-test-setup
|
||||
py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests, rebuilding signoz from the current sources
|
||||
@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup
|
||||
py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests
|
||||
@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --capture=no integration/bootstrap/setup.py::test_setup
|
||||
|
||||
.PHONY: py-test-teardown
|
||||
py-test-teardown: ## Tear down the shared SigNoz backend
|
||||
|
||||
@@ -4,13 +4,9 @@ ARG OS="linux"
|
||||
ARG TARGETARCH
|
||||
ARG ZEUSURL
|
||||
|
||||
# HOME comes from the build user, not the image config; declare it so the
|
||||
# /root paths below trace to it.
|
||||
ENV HOME=/root
|
||||
|
||||
# This path is important for stacktraces
|
||||
WORKDIR $GOPATH/src/github.com/signoz/signoz
|
||||
WORKDIR $HOME
|
||||
WORKDIR /root
|
||||
|
||||
RUN set -eux; \
|
||||
apt-get update; \
|
||||
@@ -18,36 +14,23 @@ RUN set -eux; \
|
||||
g++ \
|
||||
gcc \
|
||||
libc6-dev \
|
||||
make \
|
||||
pkg-config \
|
||||
; \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Keep the literal cache-mount targets below in sync with these. The caches
|
||||
# are shared with Dockerfile.with-web.integration (same target paths).
|
||||
ENV GOCACHE=$HOME/.cache/go-build
|
||||
ENV GOMODCACHE=$GOPATH/pkg/mod
|
||||
|
||||
COPY go.mod go.sum ./
|
||||
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
go mod download
|
||||
RUN go mod download
|
||||
|
||||
COPY ./cmd/ ./cmd/
|
||||
COPY ./ee/ ./ee/
|
||||
COPY ./pkg/ ./pkg/
|
||||
COPY ./templates /root/templates
|
||||
|
||||
# Invoked directly instead of via make so Makefile changes don't invalidate
|
||||
# this layer; the Makefile's git-derived ldflags resolve to empty in here
|
||||
# anyway (.git is dockerignored).
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
--mount=type=cache,target=/root/.cache/go-build \
|
||||
GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
|
||||
-ldflags "-s -w \
|
||||
-X github.com/SigNoz/signoz/pkg/version.version=integration \
|
||||
-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
|
||||
-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
|
||||
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"
|
||||
COPY Makefile Makefile
|
||||
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
|
||||
RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
|
||||
|
||||
RUN chmod 755 /root /root/signoz
|
||||
|
||||
|
||||
@@ -1,23 +1,10 @@
|
||||
FROM node:22-bookworm AS build
|
||||
|
||||
WORKDIR /opt/
|
||||
|
||||
# HOME comes from the build user, not the image config.
|
||||
ENV HOME=/root
|
||||
# pnpm's store lives at $PNPM_HOME/store — a dedicated directory pnpm
|
||||
# manages. Keep the literal cache-mount targets below in sync.
|
||||
ENV PNPM_HOME=/pnpm
|
||||
ENV NODE_OPTIONS=--max-old-space-size=8192
|
||||
|
||||
RUN CI=1 npm i -g pnpm@10
|
||||
|
||||
# pnpm fetch resolves from the lockfile alone and runs no lifecycle scripts;
|
||||
# the repo's postinstall needs source files that are not copied yet.
|
||||
COPY ./frontend/package.json ./frontend/pnpm-lock.yaml ./frontend/pnpm-workspace.yaml ./
|
||||
RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm fetch
|
||||
|
||||
COPY ./frontend/ ./
|
||||
RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm install --offline
|
||||
ENV NODE_OPTIONS=--max-old-space-size=8192
|
||||
RUN CI=1 npm i -g pnpm@10
|
||||
RUN CI=1 pnpm install
|
||||
RUN CI=1 pnpm build
|
||||
|
||||
FROM golang:1.25-bookworm
|
||||
@@ -26,13 +13,9 @@ ARG OS="linux"
|
||||
ARG TARGETARCH
|
||||
ARG ZEUSURL
|
||||
|
||||
# HOME comes from the build user, not the image config; declare it so the
|
||||
# /root paths below trace to it.
|
||||
ENV HOME=/root
|
||||
|
||||
# This path is important for stacktraces
|
||||
WORKDIR $GOPATH/src/github.com/signoz/signoz
|
||||
WORKDIR $HOME
|
||||
WORKDIR /root
|
||||
|
||||
RUN set -eux; \
|
||||
apt-get update; \
|
||||
@@ -40,36 +23,23 @@ RUN set -eux; \
|
||||
g++ \
|
||||
gcc \
|
||||
libc6-dev \
|
||||
make \
|
||||
pkg-config \
|
||||
; \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Keep the literal cache-mount targets below in sync with these. The caches
|
||||
# are shared with Dockerfile.integration (same target paths).
|
||||
ENV GOCACHE=$HOME/.cache/go-build
|
||||
ENV GOMODCACHE=$GOPATH/pkg/mod
|
||||
|
||||
COPY go.mod go.sum ./
|
||||
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
go mod download
|
||||
RUN go mod download
|
||||
|
||||
COPY ./cmd/ ./cmd/
|
||||
COPY ./ee/ ./ee/
|
||||
COPY ./pkg/ ./pkg/
|
||||
COPY ./templates /root/templates
|
||||
|
||||
# Invoked directly instead of via make so Makefile changes don't invalidate
|
||||
# this layer; the Makefile's git-derived ldflags resolve to empty in here
|
||||
# anyway (.git is dockerignored).
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
--mount=type=cache,target=/root/.cache/go-build \
|
||||
GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
|
||||
-ldflags "-s -w \
|
||||
-X github.com/SigNoz/signoz/pkg/version.version=integration \
|
||||
-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
|
||||
-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
|
||||
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"
|
||||
COPY Makefile Makefile
|
||||
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
|
||||
RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
|
||||
|
||||
COPY --from=build /opt/build ./web/
|
||||
|
||||
|
||||
@@ -7430,8 +7430,6 @@ components:
|
||||
- below
|
||||
- equal
|
||||
- not_equal
|
||||
- above_or_equal
|
||||
- below_or_equal
|
||||
- outside_bounds
|
||||
type: string
|
||||
RuletypesCumulativeSchedule:
|
||||
|
||||
@@ -1,123 +0,0 @@
|
||||
# PromQL Serving — clickhouseprometheusv2
|
||||
|
||||
This document is the subsystem context for `pkg/prometheus/clickhouseprometheusv2`,
|
||||
the second-generation ClickHouse-backed Prometheus provider. It explains why the
|
||||
package exists, the correctness constraints that shaped it, and how each fetch
|
||||
reduction is proven not to change results. Any change to the provider must keep
|
||||
these invariants; if a change would violate one, it must be flagged and
|
||||
discussed.
|
||||
|
||||
---
|
||||
|
||||
## Why a second provider
|
||||
|
||||
The v1 provider (`pkg/prometheus/clickhouseprometheus`) serves the promql engine
|
||||
through the remote-read protobuf adapter: every raw sample of a query's union
|
||||
window is fetched, serialized, and handed to the engine. The cost is a function
|
||||
of ingested data, not of the question asked — which is how a dashboard of PromQL
|
||||
panels can take an instance down.
|
||||
|
||||
In v2 the stock promql engine evaluates over a native `storage.Querier`: no
|
||||
translation layer, per-selector fetch windows, and fetch reductions that are
|
||||
provably invisible to the engine.
|
||||
|
||||
**The core constraint: every reduction either preserves engine semantics exactly
|
||||
or is not performed.** A PromQL result that differs from upstream Prometheus is
|
||||
a lost user. The conformance suite
|
||||
(`tests/integration/tests/promqlconformance/`) replays Prometheus' own test
|
||||
corpus against both providers and is the arbiter.
|
||||
|
||||
---
|
||||
|
||||
## Series lookup
|
||||
|
||||
Matchers resolve to series once per selector (`selectSeries`) against the series
|
||||
tables, which hold one row per (fingerprint, bucket) at 1h/6h/1d/1w
|
||||
granularities. Table selection and window rounding delegate to the shared
|
||||
metrics schema package (`pkg/telemetryschema/metricstelemetryschema`); the
|
||||
window start rounds down to the bucket boundary so a window beginning mid-bucket
|
||||
still matches the bucket's row.
|
||||
|
||||
How matchers become SQL is documented at `applySeriesConditions`. The rules that
|
||||
carry semantics:
|
||||
|
||||
- `__name__` matchers (all four types) translate to the `metric_name` column.
|
||||
- Every other matcher becomes a `JSONExtractString` condition on the labels
|
||||
column. An equality matcher against `""` matches series *without* the label,
|
||||
mirroring PromQL, because `JSONExtractString` returns `""` for missing keys.
|
||||
- Regexes are anchored (`^(?:...)$`) before they reach `match()`: PromQL
|
||||
matchers match the whole value, ClickHouse `match()` searches for a
|
||||
substring.
|
||||
- The series-lookup upper bound is inclusive (`unix_milli <= end`) because the
|
||||
exporter floors registration rows to the bucket start: a series first
|
||||
registered in the bucket beginning exactly at `end` would otherwise be
|
||||
invisible while its samples are in range.
|
||||
|
||||
Empty-valued labels come off at this boundary: an empty value means "label
|
||||
absent" in Prometheus, but stored attribute JSON can carry them.
|
||||
|
||||
---
|
||||
|
||||
## Sample fetch
|
||||
|
||||
Samples are fetched per selector using the engine's per-selector hints, not the
|
||||
query-wide union window — `foo / foo offset 1d` reads two narrow windows
|
||||
instead of the widest one twice.
|
||||
|
||||
**Last-sample-per-step reduction.** Instant selectors of subquery-free queries
|
||||
fetch only the last sample per step bucket. The engine resolves an instant
|
||||
selector at each grid timestamp `t` to the latest sample in the left-open
|
||||
lookback window `(t − lookback, t]`. Buckets anchor at the selector's first
|
||||
evaluation timestamp — recovered from the hints as
|
||||
`hints.Start + lookback − 1ms`, the inverse of how the engine derives
|
||||
`hints.Start` — so bucket boundaries coincide with evaluation timestamps, and a
|
||||
non-final sample of a bucket can never be the latest sample in
|
||||
`(t − lookback, t]` for any grid `t`. Real timestamps are preserved, so the
|
||||
engine's own lookback and staleness handling stay exact.
|
||||
|
||||
Range selectors always fetch raw — every sample feeds the range function. The
|
||||
subquery-free proof travels in the context as `prometheus.QueryTraits`, because
|
||||
subquery selectors evaluate at the subquery's step while the hints carry the
|
||||
top-level step; call sites that do not attach traits get the conservative raw
|
||||
fetch.
|
||||
|
||||
**Row assembly** maps stale flags to the engine's `StaleNaN` and merges series
|
||||
with identical label sets (`sortAndMerge`) — the engine assumes storages never
|
||||
emit duplicates. Duplicate timestamps pass through as stored: uniqueness is
|
||||
ingest's job, and v1 feeds them to the engine as-is over the same data.
|
||||
|
||||
**The fingerprint filter is a shard-local semi-join.** The samples query
|
||||
restricts to the matched series by re-running the series predicates as an
|
||||
`IN (SELECT fingerprint FROM <local series table> ...)` subquery, not a GLOBAL
|
||||
broadcast of the matched set. ClickHouse materializes the subquery's set per
|
||||
shard before the scan, so it still engages the fingerprint primary-key column.
|
||||
Because the subquery re-executes the predicates after the lookup ran, it can
|
||||
match series registered in between; sample rows whose fingerprint the lookup
|
||||
never saw are skipped — the lookup is the read snapshot.
|
||||
|
||||
---
|
||||
|
||||
## Sharding
|
||||
|
||||
`samples_v4` and `time_series_v4` (and all their rollups) shard on the same key
|
||||
— `cityHash64(env, temporality, metric_name, fingerprint)` — so a series'
|
||||
samples and catalog rows live on the same shard. The semi-join above exploits
|
||||
that: each shard filters by its own series rows, which are exactly the series
|
||||
of that shard's samples.
|
||||
|
||||
The temporality filter on every samples statement
|
||||
(`temporality IN ['Cumulative', 'Unspecified']`) is a semantic no-op — the
|
||||
matched fingerprints already come from those temporalities — that engages the
|
||||
leading samples primary-key column.
|
||||
|
||||
Delta-temporality series stay invisible to PromQL exactly as they are in v1:
|
||||
the rollout gate is parity with v1, and a Delta stream fed to `rate()`
|
||||
as-if-cumulative would be wrong, not just new.
|
||||
|
||||
---
|
||||
|
||||
## Observability
|
||||
|
||||
Every statement carries a `log_comment` with
|
||||
`code.namespace=clickhouse-prometheus-v2` and `code.function.name` naming the
|
||||
call site, so this provider's work is attributable in `system.query_log`.
|
||||
@@ -30,11 +30,11 @@ yarn install:browsers # one-time Playwright browser install
|
||||
|
||||
### Starting the Test Environment
|
||||
|
||||
To spin up the backend stack (SigNoz, ClickHouse, Postgres, ClickHouse Keeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
|
||||
To spin up the backend stack (SigNoz, ClickHouse, Postgres, Zookeeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
|
||||
|
||||
```bash
|
||||
cd tests
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --with-web \
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse --with-web \
|
||||
e2e/bootstrap/setup.py::test_setup
|
||||
```
|
||||
|
||||
@@ -45,13 +45,8 @@ This command will:
|
||||
- Start the HTTP seeder container (`tests/seeder/` — exposing `/telemetry/{traces,logs,metrics}` POST + DELETE)
|
||||
- Write backend coordinates to `tests/e2e/.env.local` (loaded by `playwright.config.ts` via dotenv)
|
||||
- Keep containers running via the `--reuse` flag
|
||||
- Rebuild the SigNoz container from the current sources via the `--rebuild` flag
|
||||
|
||||
The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start; later builds are incremental.
|
||||
|
||||
### Rebuilding After Source Changes
|
||||
|
||||
The `--with-web` image bakes the built frontend in, so neither backend nor frontend changes are picked up while `--reuse` keeps the container running. `--rebuild` fixes that for both: it kills the SigNoz container, rebuilds the image incrementally (go build cache + pnpm store — a frontend-only change rebuilds in about a minute), and starts a fresh one while databases, mocks, migrations, and the seeder stay reused. The setup command above passes it, so the iteration loop is: change code → re-run the setup command → re-run your specs. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.
|
||||
The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start.
|
||||
|
||||
### Stopping the Test Environment
|
||||
|
||||
@@ -286,16 +281,13 @@ The full `playwright.config.ts` is the source of truth. Common things to tweak:
|
||||
The same pytest flags integration tests expose work here, since E2E reuses the shared fixture graph:
|
||||
|
||||
- `--reuse` — keep containers warm between runs (required for all iteration).
|
||||
- `--rebuild` — recreate the SigNoz container from the current sources (backend and, with `--with-web`, frontend) while the rest of the stack stays up. Requires `--reuse`.
|
||||
- `--teardown` — tear everything down.
|
||||
- `--clean` — prune the docker build caches, forcing the next image build to start cold.
|
||||
- `--with-web` — build the frontend into the SigNoz container. **Required for E2E**; integration tests don't need it.
|
||||
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/tests/integration.md`.
|
||||
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/integration.md`.
|
||||
|
||||
## What should I remember?
|
||||
|
||||
- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build on a cold start; later builds are incremental.
|
||||
- **Changed backend or frontend code? Re-run the setup command** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
|
||||
- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build; you only want to pay that once.
|
||||
- **Don't teardown before setup.** `--reuse` correctly handles partially-set-up state, so chaining teardown → setup wastes time.
|
||||
- **Prefer UI-driven flows.** Playwright captures BE requests in the trace; a parallel `fetch` probe is almost always redundant. Drop to `page.request.*` only when the UI can't reach what you need.
|
||||
- **Use `page.waitForResponse` on UI clicks** to assert BE contracts — it still exercises the UI trigger path.
|
||||
|
||||
@@ -37,34 +37,13 @@ make py-test-setup
|
||||
Under the hood this runs, from `tests/`:
|
||||
|
||||
```bash
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse integration/bootstrap/setup.py::test_setup
|
||||
```
|
||||
|
||||
This command will:
|
||||
- Start all required services (ClickHouse, PostgreSQL, ClickHouse Keeper, SigNoz, Zeus mock, gateway mock)
|
||||
- Start all required services (ClickHouse, PostgreSQL, Zookeeper, SigNoz, Zeus mock, gateway mock)
|
||||
- Register an admin user
|
||||
- Keep containers running via the `--reuse` flag
|
||||
- Rebuild the SigNoz container from the current sources via the `--rebuild` flag
|
||||
|
||||
### Rebuilding After Source Changes
|
||||
|
||||
`--reuse` keeps the running SigNoz container, which means backend source changes are not picked up. `--rebuild` fixes exactly that: it kills the existing SigNoz container, rebuilds the image (incremental — only changed packages recompile thanks to the build cache), and starts a fresh one, while everything else (databases, mocks, migrations) stays reused. `make py-test-setup` passes it by default, so the iteration loop is simply:
|
||||
|
||||
```bash
|
||||
make py-test-setup # (re)build signoz from your current sources
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
|
||||
# ... edit backend code or tests ...
|
||||
make py-test-setup # pick up the backend changes
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
|
||||
```
|
||||
|
||||
The same applies to the e2e stack. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.
|
||||
|
||||
Some suites define their own SigNoz variant in a suite-local `conftest.py` (`create_signoz(..., cache_key=...)` — e.g. `basepath`, `metricreduction`, `querier_json_body`). Those containers are not touched by `make py-test-setup`, which only rebuilds the default instance. For such suites, pass `--rebuild` on the suite run itself — it rebuilds every SigNoz variant the run instantiates:
|
||||
|
||||
```bash
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild integration/tests/<suite>/
|
||||
```
|
||||
|
||||
### Stopping the Test Environment
|
||||
|
||||
@@ -77,21 +56,11 @@ make py-test-teardown
|
||||
Which runs:
|
||||
|
||||
```bash
|
||||
uv run pytest --basetemp=./tmp/ -vv --teardown --capture=no integration/bootstrap/setup.py::test_teardown
|
||||
uv run pytest --basetemp=./tmp/ -vv --teardown integration/bootstrap/setup.py::test_teardown
|
||||
```
|
||||
|
||||
This destroys the running integration test setup and cleans up resources.
|
||||
|
||||
### Cleaning the Image Build Cache
|
||||
|
||||
The `signoz:integration` image build keeps its Go build and module caches in BuildKit cache mounts, so rebuilds only recompile what changed. These caches survive `--teardown` (they belong to the Docker builder, not to any container). If a cache ever needs to be nuked — suspected corruption, disk pressure, or to force a genuinely cold build — pass the `--clean` flag:
|
||||
|
||||
```bash
|
||||
uv run pytest --basetemp=./tmp/ -vv --teardown --clean integration/bootstrap/setup.py::test_teardown
|
||||
```
|
||||
|
||||
`--clean` prunes the docker build artifacts backing the incremental image build at session start, so the next build starts from a clean slate. Images and regular layer cache stay intact, but note the pruning is host-wide — it clears build caches for other projects too, not just SigNoz's. The flag composes with any invocation — passing it on a normal `--reuse` run simply makes the next image build start cold (~3–4 minutes instead of seconds).
|
||||
|
||||
## Understanding the Integration Test Framework
|
||||
|
||||
Python and pytest form the foundation of the integration testing framework. Testcontainers are used to spin up disposable integration environments. WireMock is used to spin up **test doubles** of external services (Zeus cloud API, gateway, etc.).
|
||||
@@ -130,7 +99,7 @@ tests/
|
||||
│ ├── passwordauthn/
|
||||
│ ├── querier/
|
||||
│ └── ...
|
||||
└── e2e/ # Playwright suite (see docs/contributing/tests/e2e.md)
|
||||
└── e2e/ # Playwright suite (see docs/contributing/e2e.md)
|
||||
```
|
||||
|
||||
Each test suite follows these principles:
|
||||
@@ -255,9 +224,9 @@ Tests can be configured using pytest options:
|
||||
- `--sqlstore-provider` — Choose the SQL store provider (default: `postgres`)
|
||||
- `--sqlite-mode` — SQLite journal mode: `delete` or `wal` (default: `delete`). Only relevant when `--sqlstore-provider=sqlite`.
|
||||
- `--postgres-version` — PostgreSQL version (default: `15`)
|
||||
- `--clickhouse-version` — ClickHouse version, also used for ClickHouse Keeper (default: `25.12.5`)
|
||||
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.6`)
|
||||
- `--with-web` — Build the frontend into the SigNoz image (required for e2e)
|
||||
- `--clickhouse-version` — ClickHouse version (default: `25.5.6`)
|
||||
- `--zookeeper-version` — Zookeeper version (default: `3.7.1`)
|
||||
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.2`)
|
||||
|
||||
Example:
|
||||
|
||||
@@ -270,7 +239,6 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
|
||||
## What should I remember?
|
||||
|
||||
- **Always use the `--reuse` flag** when setting up the environment or running tests to keep containers warm. Without it every run rebuilds the stack (~4 mins).
|
||||
- **Changed backend code? Re-run `make py-test-setup`** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
|
||||
- **Use the `--teardown` flag** only when cleaning up — mixing `--teardown` with `--reuse` is a contradiction.
|
||||
- **Do not pre-emptively teardown before setup.** If the stack is partially up, `--reuse` picks up from wherever it is. `make py-test-teardown` then `make py-test-setup` wastes minutes.
|
||||
- **Follow the naming convention** with two-digit numeric prefixes (`01_`, `02_`) for ordered test execution within a suite.
|
||||
@@ -279,5 +247,5 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
|
||||
- **Use descriptive test names** that clearly indicate what is being tested.
|
||||
- **Leverage fixtures** for common setup. The shared fixture package is at `tests/fixtures/` — reuse before adding new ones.
|
||||
- **Test both success and failure scenarios** (4xx / 5xx paths) to ensure robust functionality.
|
||||
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — ruff format + ruff check.
|
||||
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — black + isort + autoflake + pylint.
|
||||
- **`--sqlite-mode=wal` does not work on macOS.** The integration test environment runs SigNoz inside a Linux container with the SQLite database file mounted from the macOS host. WAL mode requires shared memory between connections, and connections crossing the VM boundary (macOS host ↔ Linux container) cannot share the WAL index, resulting in `SQLITE_IOERR_SHORT_READ`. WAL mode is tested in CI on Linux only.
|
||||
|
||||
@@ -8479,8 +8479,6 @@ export enum RuletypesCompareOperatorDTO {
|
||||
below = 'below',
|
||||
equal = 'equal',
|
||||
not_equal = 'not_equal',
|
||||
above_or_equal = 'above_or_equal',
|
||||
below_or_equal = 'below_or_equal',
|
||||
outside_bounds = 'outside_bounds',
|
||||
}
|
||||
export interface RuletypesBasicRuleThresholdDTO {
|
||||
|
||||
@@ -66,10 +66,6 @@ function ThresholdItem({
|
||||
return '=';
|
||||
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
|
||||
return '!=';
|
||||
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
|
||||
return '>=';
|
||||
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
|
||||
return '<=';
|
||||
default:
|
||||
return '';
|
||||
}
|
||||
|
||||
@@ -83,10 +83,6 @@ const getOperatorWord = (op: AlertThresholdOperator): string => {
|
||||
return 'equal';
|
||||
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
|
||||
return 'not equal';
|
||||
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
|
||||
return 'equal or exceed';
|
||||
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
|
||||
return 'equal or fall below';
|
||||
default:
|
||||
return 'exceed';
|
||||
}
|
||||
@@ -102,10 +98,6 @@ const getThresholdValue = (op: AlertThresholdOperator): number => {
|
||||
return 100;
|
||||
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
|
||||
return 0;
|
||||
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
|
||||
return 80;
|
||||
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
|
||||
return 50;
|
||||
default:
|
||||
return 80;
|
||||
}
|
||||
@@ -124,8 +116,6 @@ const getDataPoints = (
|
||||
[AlertThresholdOperator.IS_EQUAL_TO]: [95, 100, 105, 90, 100],
|
||||
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [5, 0, 10, 15, 0],
|
||||
[AlertThresholdOperator.IS_ABOVE]: [75, 85, 90, 78, 95],
|
||||
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [75, 80, 90, 78, 95],
|
||||
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [60, 50, 40, 55, 35],
|
||||
[AlertThresholdOperator.ABOVE_BELOW]: [75, 85, 90, 78, 95],
|
||||
},
|
||||
[AlertThresholdMatchType.ALL_THE_TIME]: {
|
||||
@@ -133,8 +123,6 @@ const getDataPoints = (
|
||||
[AlertThresholdOperator.IS_EQUAL_TO]: [100, 100, 100, 100, 100],
|
||||
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [5, 10, 15, 8, 12],
|
||||
[AlertThresholdOperator.IS_ABOVE]: [85, 87, 90, 88, 95],
|
||||
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [80, 87, 90, 88, 95],
|
||||
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [50, 40, 35, 42, 38],
|
||||
[AlertThresholdOperator.ABOVE_BELOW]: [85, 87, 90, 88, 95],
|
||||
},
|
||||
[AlertThresholdMatchType.ON_AVERAGE]: {
|
||||
@@ -142,8 +130,6 @@ const getDataPoints = (
|
||||
[AlertThresholdOperator.IS_EQUAL_TO]: [95, 105, 100, 95, 105],
|
||||
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [5, 10, 15, 8, 12],
|
||||
[AlertThresholdOperator.IS_ABOVE]: [75, 85, 90, 78, 95],
|
||||
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [70, 85, 90, 75, 80],
|
||||
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [60, 40, 55, 45, 50],
|
||||
[AlertThresholdOperator.ABOVE_BELOW]: [75, 85, 90, 78, 95],
|
||||
},
|
||||
[AlertThresholdMatchType.IN_TOTAL]: {
|
||||
@@ -151,8 +137,6 @@ const getDataPoints = (
|
||||
[AlertThresholdOperator.IS_EQUAL_TO]: [20, 20, 20, 20, 20],
|
||||
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [10, 15, 25, 5, 30],
|
||||
[AlertThresholdOperator.IS_ABOVE]: [10, 15, 25, 5, 30],
|
||||
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [10, 15, 25, 5, 25],
|
||||
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [8, 5, 10, 12, 15],
|
||||
[AlertThresholdOperator.ABOVE_BELOW]: [10, 15, 25, 5, 30],
|
||||
},
|
||||
[AlertThresholdMatchType.LAST]: {
|
||||
@@ -160,8 +144,6 @@ const getDataPoints = (
|
||||
[AlertThresholdOperator.IS_EQUAL_TO]: [75, 85, 90, 78, 100],
|
||||
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: [75, 85, 90, 78, 25],
|
||||
[AlertThresholdOperator.IS_ABOVE]: [75, 85, 90, 78, 95],
|
||||
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: [75, 85, 90, 78, 80],
|
||||
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: [75, 85, 90, 78, 50],
|
||||
[AlertThresholdOperator.ABOVE_BELOW]: [75, 85, 90, 78, 95],
|
||||
},
|
||||
};
|
||||
@@ -175,8 +157,6 @@ const getTooltipOperatorSymbol = (op: AlertThresholdOperator): string => {
|
||||
[AlertThresholdOperator.IS_BELOW]: '<',
|
||||
[AlertThresholdOperator.IS_EQUAL_TO]: '=',
|
||||
[AlertThresholdOperator.IS_NOT_EQUAL_TO]: '!=',
|
||||
[AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO]: '>=',
|
||||
[AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO]: '<=',
|
||||
[AlertThresholdOperator.ABOVE_BELOW]: '>',
|
||||
};
|
||||
return symbolMap[op] || '>';
|
||||
@@ -272,10 +252,6 @@ export const getMatchTypeTooltip = (
|
||||
return p === thresholdValue;
|
||||
case AlertThresholdOperator.IS_NOT_EQUAL_TO:
|
||||
return p !== thresholdValue;
|
||||
case AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO:
|
||||
return p >= thresholdValue;
|
||||
case AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO:
|
||||
return p <= thresholdValue;
|
||||
default:
|
||||
return p > thresholdValue;
|
||||
}
|
||||
@@ -318,8 +294,7 @@ export const getMatchTypeTooltip = (
|
||||
matchType={matchType}
|
||||
>
|
||||
Alert triggers (all points {operatorWord} {thresholdValue})<br />
|
||||
If any point didn't {operatorWord} {thresholdValue}, no alert would
|
||||
fire
|
||||
If any point was {thresholdValue}, no alert would fire
|
||||
</TooltipExample>
|
||||
<TooltipLink />
|
||||
</TooltipContent>
|
||||
|
||||
@@ -532,7 +532,7 @@ describe('Footer utils', () => {
|
||||
['symbol', '>', 'at_least_once'],
|
||||
['literal', 'above', 'at_least_once'],
|
||||
['short', 'eq', 'avg'],
|
||||
['inclusive', 'above_or_equal', 'at_least_once'],
|
||||
['UI-unexposed', 'above_or_equal', 'at_least_once'],
|
||||
])(
|
||||
'round-trips %s op/matchType unchanged through the submit payload (%s / %s)',
|
||||
(_desc, op, matchType) => {
|
||||
|
||||
@@ -332,20 +332,25 @@ describe('CreateAlertV2 utils', () => {
|
||||
['not_equal', AlertThresholdOperator.IS_NOT_EQUAL_TO],
|
||||
['not_eq', AlertThresholdOperator.IS_NOT_EQUAL_TO],
|
||||
['!=', AlertThresholdOperator.IS_NOT_EQUAL_TO],
|
||||
['5', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
|
||||
['above_or_equal', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
|
||||
['above_or_eq', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
|
||||
['>=', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
|
||||
['6', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
|
||||
['below_or_equal', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
|
||||
['below_or_eq', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
|
||||
['<=', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
|
||||
['7', AlertThresholdOperator.ABOVE_BELOW],
|
||||
['outside_bounds', AlertThresholdOperator.ABOVE_BELOW],
|
||||
])('maps backend alias %s to canonical enum', (alias, expected) => {
|
||||
expect(normalizeOperator(alias)).toBe(expected);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['5', 'above_or_equal'],
|
||||
['above_or_equal', 'above_or_equal'],
|
||||
['above_or_eq', 'above_or_equal'],
|
||||
['>=', 'above_or_equal'],
|
||||
['6', 'below_or_equal'],
|
||||
['below_or_equal', 'below_or_equal'],
|
||||
['below_or_eq', 'below_or_equal'],
|
||||
['<=', 'below_or_equal'],
|
||||
])('returns undefined for UI-unexposed alias %s (%s family)', (alias) => {
|
||||
expect(normalizeOperator(alias)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('returns undefined for unknown values', () => {
|
||||
expect(normalizeOperator('gibberish')).toBeUndefined();
|
||||
expect(normalizeOperator(undefined)).toBeUndefined();
|
||||
@@ -408,8 +413,8 @@ describe('CreateAlertV2 utils', () => {
|
||||
['symbol', '>', 'at_least_once'],
|
||||
['short form', 'eq', 'avg'],
|
||||
['mixed numeric and literal', '7', 'last'],
|
||||
['inclusive literal operator', 'above_or_equal', 'at_least_once'],
|
||||
['inclusive numeric operator', '5', 'at_least_once'],
|
||||
['UI-unexposed operator', 'above_or_equal', 'at_least_once'],
|
||||
['UI-unexposed numeric operator', '5', 'at_least_once'],
|
||||
])('preserves %s op/matchType verbatim (%s / %s)', (_desc, op, matchType) => {
|
||||
const state = getThresholdStateFromAlertDef(buildDef(op, matchType));
|
||||
expect(state.operator).toBe(op);
|
||||
|
||||
@@ -2,8 +2,9 @@ import { AlertThresholdMatchType, AlertThresholdOperator } from './types';
|
||||
|
||||
// Mirrors the backend's CompareOperator.Normalize() in
|
||||
// pkg/types/ruletypes/compare.go. Maps any accepted alias to the enum value
|
||||
// the dropdown understands. Returns undefined for unknown values so callers
|
||||
// can keep the raw value on screen instead of silently rewriting it.
|
||||
// the dropdown understands. Returns undefined for aliases the UI does not
|
||||
// expose (e.g. above_or_equal, below_or_equal) so callers can keep the raw
|
||||
// value on screen instead of silently rewriting it.
|
||||
export function normalizeOperator(
|
||||
raw: string | undefined,
|
||||
): AlertThresholdOperator | undefined {
|
||||
@@ -26,16 +27,6 @@ export function normalizeOperator(
|
||||
case 'not_eq':
|
||||
case '!=':
|
||||
return AlertThresholdOperator.IS_NOT_EQUAL_TO;
|
||||
case '5':
|
||||
case 'above_or_equal':
|
||||
case 'above_or_eq':
|
||||
case '>=':
|
||||
return AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO;
|
||||
case '6':
|
||||
case 'below_or_equal':
|
||||
case 'below_or_eq':
|
||||
case '<=':
|
||||
return AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO;
|
||||
case '7':
|
||||
case 'outside_bounds':
|
||||
return AlertThresholdOperator.ABOVE_BELOW;
|
||||
|
||||
@@ -125,14 +125,6 @@ export const THRESHOLD_OPERATOR_OPTIONS = [
|
||||
{ value: AlertThresholdOperator.IS_BELOW, label: 'BELOW' },
|
||||
{ value: AlertThresholdOperator.IS_EQUAL_TO, label: 'EQUAL TO' },
|
||||
{ value: AlertThresholdOperator.IS_NOT_EQUAL_TO, label: 'NOT EQUAL TO' },
|
||||
{
|
||||
value: AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO,
|
||||
label: 'ABOVE OR EQUAL TO',
|
||||
},
|
||||
{
|
||||
value: AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO,
|
||||
label: 'BELOW OR EQUAL TO',
|
||||
},
|
||||
];
|
||||
|
||||
export const ANOMALY_THRESHOLD_OPERATOR_OPTIONS = [
|
||||
|
||||
@@ -99,8 +99,6 @@ export enum AlertThresholdOperator {
|
||||
IS_BELOW = 'below',
|
||||
IS_EQUAL_TO = 'equal',
|
||||
IS_NOT_EQUAL_TO = 'not_equal',
|
||||
IS_ABOVE_OR_EQUAL_TO = 'above_or_equal',
|
||||
IS_BELOW_OR_EQUAL_TO = 'below_or_equal',
|
||||
ABOVE_BELOW = 'outside_bounds',
|
||||
}
|
||||
|
||||
|
||||
@@ -167,9 +167,9 @@ describe('deriveAlertPrefill', () => {
|
||||
|
||||
it.each([
|
||||
['above', AlertThresholdOperator.IS_ABOVE],
|
||||
['above_or_equal', AlertThresholdOperator.IS_ABOVE_OR_EQUAL_TO],
|
||||
['above_or_equal', AlertThresholdOperator.IS_ABOVE],
|
||||
['below', AlertThresholdOperator.IS_BELOW],
|
||||
['below_or_equal', AlertThresholdOperator.IS_BELOW_OR_EQUAL_TO],
|
||||
['below_or_equal', AlertThresholdOperator.IS_BELOW],
|
||||
['equal', AlertThresholdOperator.IS_EQUAL_TO],
|
||||
['not_equal', AlertThresholdOperator.IS_NOT_EQUAL_TO],
|
||||
])('maps panel operator %s → %s', (op, expected) => {
|
||||
|
||||
@@ -104,6 +104,20 @@ function pickHighestDanger(
|
||||
)[0];
|
||||
}
|
||||
|
||||
// The alert UI has no inclusive operator; collapse "or equal" onto its strict variant.
|
||||
function panelOperatorToAlertOperator(
|
||||
operator: DashboardtypesComparisonOperatorDTO | undefined,
|
||||
): AlertThresholdOperator | undefined {
|
||||
switch (operator) {
|
||||
case 'above_or_equal':
|
||||
return normalizeOperator('above');
|
||||
case 'below_or_equal':
|
||||
return normalizeOperator('below');
|
||||
default:
|
||||
return normalizeOperator(operator);
|
||||
}
|
||||
}
|
||||
|
||||
export function deriveAlertPrefill(
|
||||
panel: DashboardtypesPanelDTO,
|
||||
query: Query,
|
||||
@@ -121,7 +135,7 @@ export function deriveAlertPrefill(
|
||||
|
||||
const top = pickHighestDanger(readPanelThresholds(panel.spec.plugin));
|
||||
if (top) {
|
||||
prefill.operator = normalizeOperator(top.operator);
|
||||
prefill.operator = panelOperatorToAlertOperator(top.operator);
|
||||
prefill.threshold = {
|
||||
id: uuid(),
|
||||
label: 'critical',
|
||||
|
||||
@@ -107,20 +107,4 @@ describe('PALETTE_V3 darken-pair table', () => {
|
||||
const unique = new Set(darks);
|
||||
expect(unique.size).toBe(PALETTE_V3.length);
|
||||
});
|
||||
|
||||
it('prints the base→dark table for visual inspection', () => {
|
||||
// eslint-disable-next-line no-console
|
||||
console.log('\nPALETTE_V3 base → darkenHex(0.22) pairs:');
|
||||
// eslint-disable-next-line no-console
|
||||
console.log('idx name base dark');
|
||||
PALETTE_V3.forEach((hex, i) => {
|
||||
const dark = darkenHex(hex, 0.22);
|
||||
const name = (PALETTE_NAMES[i] ?? '').padEnd(13);
|
||||
const idx = String(i).padStart(2, ' ');
|
||||
// eslint-disable-next-line no-console
|
||||
console.log(`${idx} ${name} ${hex} ${dark}`);
|
||||
});
|
||||
// Sentinel assertion so the test is not flagged as having none.
|
||||
expect(PALETTE_V3).toHaveLength(28);
|
||||
});
|
||||
});
|
||||
|
||||
4
pkg/config/doc.go
Normal file
4
pkg/config/doc.go
Normal file
@@ -0,0 +1,4 @@
|
||||
// Package config provides the configuration management for the Signoz application.
|
||||
// It includes functionality to define, load, and validate the application's configuration
|
||||
// using various providers and formats.
|
||||
package config
|
||||
3
pkg/errors/doc.go
Normal file
3
pkg/errors/doc.go
Normal file
@@ -0,0 +1,3 @@
|
||||
// package error contains error related utilities. Use this package when
|
||||
// a well-defined error has to be shown.
|
||||
package errors
|
||||
3
pkg/http/doc.go
Normal file
3
pkg/http/doc.go
Normal file
@@ -0,0 +1,3 @@
|
||||
// package http contains all http related functions such
|
||||
// as servers, middlewares, routers and renders.
|
||||
package http
|
||||
2
pkg/http/server/doc.go
Normal file
2
pkg/http/server/doc.go
Normal file
@@ -0,0 +1,2 @@
|
||||
// package server contains an implementation of the http server.
|
||||
package server
|
||||
5
pkg/instrumentation/doc.go
Normal file
5
pkg/instrumentation/doc.go
Normal file
@@ -0,0 +1,5 @@
|
||||
// Package instrumentation provides utilities for initializing and managing
|
||||
// OpenTelemetry resources, logging, tracing, and metering within the application. It
|
||||
// leverages the OpenTelemetry SDK to facilitate the collection and
|
||||
// export of telemetry data, to an OTLP (OpenTelemetry Protocol) endpoint.
|
||||
package instrumentation
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildClusterRecords assembles the page records. Node condition counts and
|
||||
@@ -84,36 +83,16 @@ func buildClusterRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopClusterGroupsAndMetadata(
|
||||
func (m *module) getTopClusterGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableClusters,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getClustersTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.ClusterNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ClusterNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ClusterNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToClustersQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -147,20 +126,13 @@ func (m *module) getTopClusterGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getClustersTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableClusters) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -12,7 +12,6 @@ import (
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/huandu/go-sqlbuilder"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildContainerRecords assembles the page records, merging kubeletstats
|
||||
@@ -139,36 +138,16 @@ func buildContainerRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopContainerGroupsAndMetadata(
|
||||
func (m *module) getTopContainerGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableContainers,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getContainersTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.ContainerNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ContainerNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.ContainerNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToContainersQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -202,20 +181,13 @@ func (m *module) getTopContainerGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getContainersTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableContainers) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildDaemonSetRecords assembles the page records. Pod status counts come from
|
||||
@@ -90,36 +89,16 @@ func buildDaemonSetRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopDaemonSetGroupsAndMetadata(
|
||||
func (m *module) getTopDaemonSetGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableDaemonSets,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getDaemonSetsTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.DaemonSetNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DaemonSetNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DaemonSetNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToDaemonSetsQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -153,20 +132,13 @@ func (m *module) getTopDaemonSetGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getDaemonSetsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableDaemonSets) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildDeploymentRecords assembles the page records. Pod status counts come from
|
||||
@@ -82,36 +81,16 @@ func buildDeploymentRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopDeploymentGroupsAndMetadata(
|
||||
func (m *module) getTopDeploymentGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableDeployments,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getDeploymentsTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.DeploymentNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DeploymentNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.DeploymentNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToDeploymentsQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -145,20 +124,13 @@ func (m *module) getTopDeploymentGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getDeploymentsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableDeployments) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -13,7 +13,6 @@ import (
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/huandu/go-sqlbuilder"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// getPerGroupHostStatusCounts computes the number of active and inactive hosts per group
|
||||
@@ -249,41 +248,19 @@ func buildHostRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
// getTopHostGroupsAndMetadata fetches the group metadata and the ordering-metric
|
||||
// ranking concurrently, then sorts the ranked results, paginates, and backfills
|
||||
// from metadataMap when the page extends past the metric-ranked groups. Returns
|
||||
// the page of groups and the metadata map (the caller needs it for Total and
|
||||
// records). Callers must apply any req.Filter mutation before calling.
|
||||
func (m *module) getTopHostGroupsAndMetadata(
|
||||
// getTopHostGroups runs a ranking query for the ordering metric, sorts the
|
||||
// results, paginates, and backfills from metadataMap when the page extends
|
||||
// past the metric-ranked groups.
|
||||
func (m *module) getTopHostGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableHosts,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getHostsTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.HostNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.HostNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.HostNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToHostsQueryNames[orderByKey]
|
||||
// The last entry is the formula/query whose value we sort by.
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
@@ -318,20 +295,13 @@ func (m *module) getTopHostGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
// applyHostsActiveStatusFilter MODIFIES req.Filter.Expression to include an IN/NOT IN
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildJobRecords assembles the page records. Pod status counts come from
|
||||
@@ -90,36 +89,16 @@ func buildJobRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopJobGroupsAndMetadata(
|
||||
func (m *module) getTopJobGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableJobs,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getJobsTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.JobNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.JobNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.JobNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToJobsQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -153,20 +132,13 @@ func (m *module) getTopJobGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getJobsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableJobs) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -191,13 +191,18 @@ func (m *module) ListHosts(ctx context.Context, orgID valuer.UUID, req *inframon
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopHostGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getHostsTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopHostGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.HostRecord{}
|
||||
return resp, nil
|
||||
@@ -286,13 +291,18 @@ func (m *module) ListPods(ctx context.Context, orgID valuer.UUID, req *inframoni
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopPodGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getPodsTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopPodGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.PodRecord{}
|
||||
return resp, nil
|
||||
@@ -379,13 +389,18 @@ func (m *module) ListContainers(ctx context.Context, orgID valuer.UUID, req *inf
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopContainerGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getContainersTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopContainerGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.ContainerRecord{}
|
||||
return resp, nil
|
||||
@@ -478,13 +493,18 @@ func (m *module) ListNodes(ctx context.Context, orgID valuer.UUID, req *inframon
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopNodeGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getNodesTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopNodeGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.NodeRecord{}
|
||||
return resp, nil
|
||||
@@ -571,13 +591,18 @@ func (m *module) ListNamespaces(ctx context.Context, orgID valuer.UUID, req *inf
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopNamespaceGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getNamespacesTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopNamespaceGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.NamespaceRecord{}
|
||||
return resp, nil
|
||||
@@ -663,13 +688,18 @@ func (m *module) ListClusters(ctx context.Context, orgID valuer.UUID, req *infra
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopClusterGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getClustersTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopClusterGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.ClusterRecord{}
|
||||
return resp, nil
|
||||
@@ -769,13 +799,18 @@ func (m *module) ListVolumes(ctx context.Context, orgID valuer.UUID, req *infram
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopVolumeGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getVolumesTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopVolumeGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.VolumeRecord{}
|
||||
return resp, nil
|
||||
@@ -842,13 +877,18 @@ func (m *module) ListDeployments(ctx context.Context, orgID valuer.UUID, req *in
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopDeploymentGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getDeploymentsTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopDeploymentGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.DeploymentRecord{}
|
||||
return resp, nil
|
||||
@@ -934,13 +974,18 @@ func (m *module) ListStatefulSets(ctx context.Context, orgID valuer.UUID, req *i
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopStatefulSetGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getStatefulSetsTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopStatefulSetGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.StatefulSetRecord{}
|
||||
return resp, nil
|
||||
@@ -1028,13 +1073,18 @@ func (m *module) ListJobs(ctx context.Context, orgID valuer.UUID, req *inframoni
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopJobGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getJobsTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopJobGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.JobRecord{}
|
||||
return resp, nil
|
||||
@@ -1122,13 +1172,18 @@ func (m *module) ListDaemonSets(ctx context.Context, orgID valuer.UUID, req *inf
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
pageGroups, metadataMap, err := m.getTopDaemonSetGroupsAndMetadata(ctx, orgID, req)
|
||||
metadataMap, err := m.getDaemonSetsTableMetadata(ctx, orgID, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp.Total = len(metadataMap)
|
||||
|
||||
pageGroups, err := m.getTopDaemonSetGroups(ctx, orgID, req, metadataMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(pageGroups) == 0 {
|
||||
resp.Records = []inframonitoringtypes.DaemonSetRecord{}
|
||||
return resp, nil
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildNamespaceRecords assembles the page records. Pod status counts come from
|
||||
@@ -65,36 +64,16 @@ func buildNamespaceRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopNamespaceGroupsAndMetadata(
|
||||
func (m *module) getTopNamespaceGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableNamespaces,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getNamespacesTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.NamespaceNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NamespaceNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NamespaceNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToNamespacesQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -128,20 +107,13 @@ func (m *module) getTopNamespaceGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getNamespacesTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableNamespaces) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -12,7 +12,6 @@ import (
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/huandu/go-sqlbuilder"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildNodeRecords assembles the page records. Condition counts come from
|
||||
@@ -92,36 +91,16 @@ func buildNodeRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopNodeGroupsAndMetadata(
|
||||
func (m *module) getTopNodeGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableNodes,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getNodesTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.NodeNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NodeNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.NodeNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToNodesQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -155,20 +134,13 @@ func (m *module) getTopNodeGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getNodesTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableNodes) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -13,7 +13,6 @@ import (
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/huandu/go-sqlbuilder"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildPodRecords assembles the page records. Status counts come from
|
||||
@@ -146,40 +145,16 @@ func buildPodRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
// getTopPodGroupsAndMetadata fetches the group metadata and the ordering-metric
|
||||
// ranking concurrently, then pages the ranked groups, backfilling from metadata
|
||||
// when the page extends past the metric-ranked groups. Returns the page of
|
||||
// groups and the metadata map (needed by the caller for Total and records).
|
||||
func (m *module) getTopPodGroupsAndMetadata(
|
||||
func (m *module) getTopPodGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostablePods,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getPodsTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.PodNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PodNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PodNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToPodsQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -213,20 +188,13 @@ func (m *module) getTopPodGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getPodsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostablePods) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildStatefulSetRecords assembles the page records. Pod status counts come from
|
||||
@@ -82,36 +81,16 @@ func buildStatefulSetRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopStatefulSetGroupsAndMetadata(
|
||||
func (m *module) getTopStatefulSetGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableStatefulSets,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getStatefulSetsTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.StatefulSetNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.StatefulSetNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.StatefulSetNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToStatefulSetsQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -145,20 +124,13 @@ func (m *module) getTopStatefulSetGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getStatefulSetsTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableStatefulSets) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// buildVolumeRecords assembles the page records. VolumeUsage is taken from the
|
||||
@@ -69,36 +68,16 @@ func buildVolumeRecords(
|
||||
return records
|
||||
}
|
||||
|
||||
func (m *module) getTopVolumeGroupsAndMetadata(
|
||||
func (m *module) getTopVolumeGroups(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
req *inframonitoringtypes.PostableVolumes,
|
||||
) ([]map[string]string, map[string]map[string]string, error) {
|
||||
|
||||
var (
|
||||
orderByKey string
|
||||
metadataMap map[string]map[string]string
|
||||
allMetricGroups []rankedGroup
|
||||
)
|
||||
|
||||
orderByKey = req.OrderBy.Key.Name
|
||||
|
||||
g, gCtx := errgroup.WithContext(ctx)
|
||||
|
||||
g.Go(func() error {
|
||||
var err error
|
||||
metadataMap, err = m.getVolumesTableMetadata(gCtx, orgID, req)
|
||||
return err
|
||||
})
|
||||
|
||||
metadataMap map[string]map[string]string,
|
||||
) ([]map[string]string, error) {
|
||||
orderByKey := req.OrderBy.Key.Name
|
||||
if orderByKey == inframonitoringtypes.PersistentVolumeClaimNameAttrKey {
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
pageGroups := inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PersistentVolumeClaimNameAttrKey)
|
||||
return pageGroups, metadataMap, nil
|
||||
return inframonitoringtypes.PaginateMetadataByName(metadataMap, req.GroupBy, req.OrderBy.Direction, req.Offset, req.Limit, inframonitoringtypes.PersistentVolumeClaimNameAttrKey), nil
|
||||
}
|
||||
|
||||
queryNamesForOrderBy := orderByToVolumesQueryNames[orderByKey]
|
||||
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
|
||||
|
||||
@@ -132,20 +111,13 @@ func (m *module) getTopVolumeGroupsAndMetadata(
|
||||
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
resp, err := m.querier.QueryRange(gCtx, orgID, topReq)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allMetricGroups = parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, nil, err
|
||||
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), metadataMap, nil
|
||||
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
|
||||
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
|
||||
}
|
||||
|
||||
func (m *module) getVolumesTableMetadata(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableVolumes) (map[string]map[string]string, error) {
|
||||
|
||||
@@ -1,85 +0,0 @@
|
||||
package clickhouseprometheusv2
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/prometheus"
|
||||
"github.com/prometheus/prometheus/model/labels"
|
||||
"github.com/prometheus/prometheus/storage"
|
||||
"github.com/prometheus/prometheus/util/annotations"
|
||||
)
|
||||
|
||||
// statementRecorder collects the statements a PromQL evaluation would run.
|
||||
// Safe for concurrent use: the engine may Select selectors concurrently.
|
||||
type statementRecorder struct {
|
||||
mu sync.Mutex
|
||||
statements []prometheus.CapturedStatement
|
||||
}
|
||||
|
||||
func (r *statementRecorder) record(query string, args []any) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.statements = append(r.statements, prometheus.CapturedStatement{Query: query, Args: args})
|
||||
}
|
||||
|
||||
func (r *statementRecorder) Statements() []prometheus.CapturedStatement {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
out := make([]prometheus.CapturedStatement, len(r.statements))
|
||||
copy(out, r.statements)
|
||||
return out
|
||||
}
|
||||
|
||||
type captureQueryable struct {
|
||||
client *client
|
||||
recorder *statementRecorder
|
||||
}
|
||||
|
||||
func (c *captureQueryable) Querier(mint, maxt int64) (storage.Querier, error) {
|
||||
return &captureQuerier{
|
||||
querier: querier{mint: mint, maxt: maxt, client: c.client},
|
||||
recorder: c.recorder,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// captureQuerier builds the same SQL as the live querier but records it and
|
||||
// returns no data. The fingerprint filter always takes the subquery form:
|
||||
// without executing the series lookup, the inline literal set is unknown.
|
||||
type captureQuerier struct {
|
||||
querier
|
||||
recorder *statementRecorder
|
||||
}
|
||||
|
||||
func (c *captureQuerier) Select(ctx context.Context, _ bool, hints *storage.SelectHints, matchers ...*labels.Matcher) storage.SeriesSet {
|
||||
|
||||
start, end := c.window(hints)
|
||||
|
||||
samplesQuery, args, err := buildSamplesQuery(start, end, metricNamesFromMatchers(matchers), matchers, c.lastSamplePerStepFor(ctx, hints))
|
||||
if err != nil {
|
||||
return storage.ErrSeriesSet(err)
|
||||
}
|
||||
c.recorder.record(samplesQuery, args)
|
||||
|
||||
return storage.EmptySeriesSet()
|
||||
}
|
||||
|
||||
func (c *captureQuerier) LabelValues(context.Context, string, *storage.LabelHints, ...*labels.Matcher) ([]string, annotations.Annotations, error) {
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
func (c *captureQuerier) LabelNames(context.Context, *storage.LabelHints, ...*labels.Matcher) ([]string, annotations.Annotations, error) {
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
// metricNamesFromMatchers extracts the statically known metric name, if any.
|
||||
// The live path derives names from the matched series; the capture path has
|
||||
// no execution results, so only a __name__ equality contributes.
|
||||
func metricNamesFromMatchers(matchers []*labels.Matcher) []string {
|
||||
for _, m := range matchers {
|
||||
if m.Name == metricNameLabel && m.Type == labels.MatchEqual && m.Value != "" {
|
||||
return []string{m.Value}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,180 +0,0 @@
|
||||
package clickhouseprometheusv2
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"math"
|
||||
"slices"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/prometheus"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/prometheus/prometheus/model/labels"
|
||||
promValue "github.com/prometheus/prometheus/model/value"
|
||||
)
|
||||
|
||||
// seriesLookup holds a series lookup's result: matched fingerprints with
|
||||
// their labels, and the distinct metric names seen on them.
|
||||
type seriesLookup struct {
|
||||
fingerprints map[uint64]labels.Labels
|
||||
metricNames []string
|
||||
}
|
||||
|
||||
// client executes the series, samples and raw queries against ClickHouse.
|
||||
type client struct {
|
||||
settings factory.ScopedProviderSettings
|
||||
telemetryStore telemetrystore.TelemetryStore
|
||||
lookbackMs int64
|
||||
}
|
||||
|
||||
func newClient(settings factory.ScopedProviderSettings, telemetryStore telemetrystore.TelemetryStore, cfg prometheus.Config) *client {
|
||||
lookback := cfg.LookbackDelta
|
||||
if lookback <= 0 {
|
||||
// Mirror the engine: promql defaults an unset lookback to 5m.
|
||||
lookback = defaultLookbackDelta
|
||||
}
|
||||
return &client{
|
||||
settings: settings,
|
||||
telemetryStore: telemetryStore,
|
||||
lookbackMs: lookback.Milliseconds(),
|
||||
}
|
||||
}
|
||||
|
||||
func (c *client) withContext(ctx context.Context, functionName string) context.Context {
|
||||
return ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
|
||||
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalMetrics.StringValue(),
|
||||
instrumentationtypes.CodeNamespace: "clickhouse-prometheus-v2",
|
||||
instrumentationtypes.CodeFunctionName: functionName,
|
||||
})
|
||||
}
|
||||
|
||||
func (c *client) selectSeries(ctx context.Context, query string, args []any) (*seriesLookup, error) {
|
||||
ctx = c.withContext(ctx, "selectSeries")
|
||||
rows, err := c.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
lookup := &seriesLookup{fingerprints: make(map[uint64]labels.Labels)}
|
||||
names := make(map[string]struct{})
|
||||
|
||||
var fingerprint uint64
|
||||
var labelsJSON string
|
||||
for rows.Next() {
|
||||
if err := rows.Scan(&fingerprint, &labelsJSON); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lset, err := unmarshalLabels(labelsJSON)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lookup.fingerprints[fingerprint] = lset
|
||||
if name := lset.Get(metricNameLabel); name != "" {
|
||||
names[name] = struct{}{}
|
||||
}
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for name := range names {
|
||||
lookup.metricNames = append(lookup.metricNames, name)
|
||||
}
|
||||
slices.Sort(lookup.metricNames)
|
||||
|
||||
return lookup, nil
|
||||
}
|
||||
|
||||
// unmarshalLabels parses the labels JSON column, dropping empty-valued
|
||||
// labels: empty means "absent" in Prometheus, but stored attribute JSON can
|
||||
// carry them.
|
||||
func unmarshalLabels(s string) (labels.Labels, error) {
|
||||
m := make(map[string]string)
|
||||
if err := json.Unmarshal([]byte(s), &m); err != nil {
|
||||
return labels.EmptyLabels(), err
|
||||
}
|
||||
builder := labels.NewScratchBuilder(len(m))
|
||||
for k, v := range m {
|
||||
if v == "" {
|
||||
continue
|
||||
}
|
||||
builder.Add(k, v)
|
||||
}
|
||||
builder.Sort()
|
||||
return builder.Labels(), nil
|
||||
}
|
||||
|
||||
// selectSamples assembles per-series sample slices from a samples query (raw
|
||||
// or last-sample-per-step; same column shape), whose rows arrive ordered by
|
||||
// (fingerprint, unix_milli). Fingerprints missing from the lookup are skipped:
|
||||
// the samples query's semi-join re-runs the series predicates and can match
|
||||
// series registered after the lookup ran — the lookup is the read snapshot.
|
||||
// Stale flags map to the engine's StaleNaN. Duplicate
|
||||
// timestamps pass through: uniqueness is ingest's job, and v1 feeds them to
|
||||
// the engine as-is over the same dirty data.
|
||||
func (c *client) selectSamples(ctx context.Context, query string, args []any, lookup *seriesLookup) ([]*series, error) {
|
||||
ctx = c.withContext(ctx, "selectSamples")
|
||||
rows, err := c.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var (
|
||||
result []*series
|
||||
current *series
|
||||
fingerprint uint64
|
||||
prevFp uint64
|
||||
timestampMs int64
|
||||
val float64
|
||||
flags uint32
|
||||
first = true
|
||||
haveCurrent bool
|
||||
staleMarker = math.Float64frombits(promValue.StaleNaN)
|
||||
unknownCount int
|
||||
)
|
||||
|
||||
for rows.Next() {
|
||||
if err := rows.Scan(&fingerprint, ×tampMs, &val, &flags); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if first || fingerprint != prevFp {
|
||||
first = false
|
||||
prevFp = fingerprint
|
||||
lset, ok := lookup.fingerprints[fingerprint]
|
||||
if !ok {
|
||||
unknownCount++
|
||||
haveCurrent = false
|
||||
continue
|
||||
}
|
||||
current = &series{lset: lset}
|
||||
result = append(result, current)
|
||||
haveCurrent = true
|
||||
}
|
||||
if !haveCurrent {
|
||||
continue
|
||||
}
|
||||
|
||||
if flags&1 == 1 {
|
||||
val = staleMarker
|
||||
}
|
||||
current.ts = append(current.ts, timestampMs)
|
||||
current.vs = append(current.vs, val)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if unknownCount > 0 {
|
||||
c.settings.Logger().DebugContext(ctx, "skipped samples of fingerprints missing from series lookup",
|
||||
slog.Int("unknown_fingerprints", unknownCount))
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
@@ -1,71 +0,0 @@
|
||||
package clickhouseprometheusv2
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/prometheus"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/prometheus/prometheus/storage"
|
||||
)
|
||||
|
||||
// provider ties the package together: its own engine and parser, and the
|
||||
// ClickHouse client behind the native storage.Querier. It stays unexported:
|
||||
// callers hold the prometheus.Prometheus interface, which is the boundary
|
||||
// between the two provider implementations.
|
||||
type provider struct {
|
||||
settings factory.ScopedProviderSettings
|
||||
engine *prometheus.Engine
|
||||
parser prometheus.Parser
|
||||
client *client
|
||||
}
|
||||
|
||||
var (
|
||||
_ prometheus.Prometheus = (*provider)(nil)
|
||||
_ prometheus.StatementCapturer = (*provider)(nil)
|
||||
)
|
||||
|
||||
func NewFactory(telemetryStore telemetrystore.TelemetryStore) factory.ProviderFactory[prometheus.Prometheus, prometheus.Config] {
|
||||
return factory.NewProviderFactory(factory.MustNewName("clickhousev2"), func(ctx context.Context, providerSettings factory.ProviderSettings, config prometheus.Config) (prometheus.Prometheus, error) {
|
||||
return New(ctx, providerSettings, config, telemetryStore)
|
||||
})
|
||||
}
|
||||
|
||||
func New(_ context.Context, providerSettings factory.ProviderSettings, config prometheus.Config, telemetryStore telemetrystore.TelemetryStore) (prometheus.Prometheus, error) {
|
||||
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/prometheus/clickhouseprometheusv2")
|
||||
|
||||
engine := prometheus.NewEngine(settings.Logger(), config)
|
||||
parser := prometheus.NewParser()
|
||||
client := newClient(settings, telemetryStore, config)
|
||||
|
||||
return &provider{
|
||||
settings: settings,
|
||||
engine: engine,
|
||||
parser: parser,
|
||||
client: client,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (p *provider) Engine() *prometheus.Engine {
|
||||
return p.engine
|
||||
}
|
||||
|
||||
func (p *provider) Parser() prometheus.Parser {
|
||||
return p.parser
|
||||
}
|
||||
|
||||
func (p *provider) Storage() storage.Queryable {
|
||||
return p
|
||||
}
|
||||
|
||||
func (p *provider) Querier(mint, maxt int64) (storage.Querier, error) {
|
||||
return &querier{mint: mint, maxt: maxt, client: p.client}, nil
|
||||
}
|
||||
|
||||
// CapturingStorage implements prometheus.StatementCapturer: a storage that
|
||||
// records each selector's SQL without executing it, for the preview path.
|
||||
// A fresh recorder per call keeps concurrent dry-runs isolated.
|
||||
func (p *provider) CapturingStorage() (storage.Queryable, prometheus.StatementRecorder) {
|
||||
recorder := &statementRecorder{}
|
||||
return &captureQueryable{client: p.client, recorder: recorder}, recorder
|
||||
}
|
||||
@@ -1,171 +0,0 @@
|
||||
package clickhouseprometheusv2
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"slices"
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/prometheus"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/huandu/go-sqlbuilder"
|
||||
"github.com/prometheus/prometheus/model/labels"
|
||||
"github.com/prometheus/prometheus/storage"
|
||||
"github.com/prometheus/prometheus/util/annotations"
|
||||
)
|
||||
|
||||
// defaultLookbackDelta mirrors promql's default when the config leaves the
|
||||
// lookback unset; the engine and the storage must agree on it for
|
||||
// last-sample-per-step bucket anchoring.
|
||||
const defaultLookbackDelta = 5 * time.Minute
|
||||
|
||||
// querier is a native storage.Querier over ClickHouse: Select builds SQL
|
||||
// directly from the matchers and hints, with no remote-read protobuf layer.
|
||||
type querier struct {
|
||||
mint, maxt int64
|
||||
client *client
|
||||
}
|
||||
|
||||
var _ storage.Querier = (*querier)(nil)
|
||||
|
||||
func (q *querier) Select(ctx context.Context, sortSeries bool, hints *storage.SelectHints, matchers ...*labels.Matcher) storage.SeriesSet {
|
||||
start, end := q.window(hints)
|
||||
|
||||
seriesQuery, seriesArgs, err := buildSeriesQuery(start, end, matchers)
|
||||
if err != nil {
|
||||
return storage.ErrSeriesSet(err)
|
||||
}
|
||||
lookup, err := q.client.selectSeries(ctx, seriesQuery, seriesArgs)
|
||||
if err != nil {
|
||||
return storage.ErrSeriesSet(err)
|
||||
}
|
||||
if len(lookup.fingerprints) == 0 {
|
||||
return storage.EmptySeriesSet()
|
||||
}
|
||||
|
||||
list, err := q.fetchSamples(ctx, start, end, matchers, lookup, q.lastSamplePerStepFor(ctx, hints))
|
||||
if err != nil {
|
||||
return storage.ErrSeriesSet(err)
|
||||
}
|
||||
|
||||
// The engine assumes storages never emit duplicate label sets.
|
||||
list = sortAndMerge(list)
|
||||
return newSeriesSet(list)
|
||||
}
|
||||
|
||||
func (q *querier) LabelValues(ctx context.Context, name string, hints *storage.LabelHints, matchers ...*labels.Matcher) ([]string, annotations.Annotations, error) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
if name == metricNameLabel {
|
||||
sb.Select("DISTINCT metric_name AS value")
|
||||
} else {
|
||||
sb.Select(fmt.Sprintf("DISTINCT JSONExtractString(labels, %s) AS value", sb.Var(name)))
|
||||
}
|
||||
adjustedStart, _, table, _ := metricstelemetryschema.WhichTSTableToUse(uint64(q.mint), uint64(q.maxt), false, nil)
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, table))
|
||||
if err := applySeriesConditions(sb, int64(adjustedStart), q.maxt, matchers); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
sb.Where("value != ''")
|
||||
if hints != nil && hints.Limit > 0 {
|
||||
sb.Limit(hints.Limit)
|
||||
}
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
values, err := q.selectStrings(ctx, "LabelValues", query, args)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
slices.Sort(values)
|
||||
return values, nil, nil
|
||||
}
|
||||
|
||||
func (q *querier) LabelNames(ctx context.Context, hints *storage.LabelHints, matchers ...*labels.Matcher) ([]string, annotations.Annotations, error) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("DISTINCT arrayJoin(JSONExtractKeys(labels)) AS name")
|
||||
adjustedStart, _, table, _ := metricstelemetryschema.WhichTSTableToUse(uint64(q.mint), uint64(q.maxt), false, nil)
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, table))
|
||||
if err := applySeriesConditions(sb, int64(adjustedStart), q.maxt, matchers); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if hints != nil && hints.Limit > 0 {
|
||||
sb.Limit(hints.Limit)
|
||||
}
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
names, err := q.selectStrings(ctx, "LabelNames", query, args)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
slices.Sort(names)
|
||||
return names, nil, nil
|
||||
}
|
||||
|
||||
func (q *querier) Close() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// window returns the per-selector fetch window from the hints (already
|
||||
// adjusted for offset, @, range and lookback); mint/maxt span the union of
|
||||
// all selectors and are the fallback.
|
||||
func (q *querier) window(hints *storage.SelectHints) (int64, int64) {
|
||||
if hints != nil && hints.Start != 0 && hints.End != 0 && hints.Start <= hints.End {
|
||||
return hints.Start, hints.End
|
||||
}
|
||||
return q.mint, q.maxt
|
||||
}
|
||||
|
||||
// lastSamplePerStepFor decides whether the fetch can keep only the last
|
||||
// sample per step bucket (see lastSamplePerStep). Only instant selectors
|
||||
// (hints.Range == 0) of subquery-free queries qualify: range selectors need
|
||||
// every raw sample, and subquery selectors evaluate at the subquery's own
|
||||
// step while hints carry the top-level step — the subquery-free proof
|
||||
// arrives as QueryTraits in the context. The first evaluation timestamp is
|
||||
// recovered as hints.Start + lookback - 1ms, inverting how the engine
|
||||
// derives hints.Start.
|
||||
func (q *querier) lastSamplePerStepFor(ctx context.Context, hints *storage.SelectHints) *lastSamplePerStep {
|
||||
if hints == nil || hints.Range != 0 || hints.Start <= 0 {
|
||||
return nil
|
||||
}
|
||||
traits, ok := prometheus.QueryTraitsFromContext(ctx)
|
||||
if !ok || !traits.SubqueryFree {
|
||||
return nil
|
||||
}
|
||||
firstEval := hints.Start + q.client.lookbackMs - 1
|
||||
if firstEval > hints.End {
|
||||
// Defensive: never anchor a bucket past the window.
|
||||
firstEval = hints.End
|
||||
}
|
||||
return &lastSamplePerStep{firstEvalMs: firstEval, stepMs: hints.Step}
|
||||
}
|
||||
|
||||
// fetchSamples runs the samples query for the matched series (see
|
||||
// buildSamplesQuery).
|
||||
func (q *querier) fetchSamples(ctx context.Context, start, end int64, matchers []*labels.Matcher, lookup *seriesLookup, lastPerStep *lastSamplePerStep) ([]*series, error) {
|
||||
query, args, err := buildSamplesQuery(start, end, lookup.metricNames, matchers, lastPerStep)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return q.client.selectSamples(ctx, query, args, lookup)
|
||||
}
|
||||
|
||||
func (q *querier) selectStrings(ctx context.Context, fn, query string, args []any) ([]string, error) {
|
||||
ctx = q.client.withContext(ctx, fn)
|
||||
rows, err := q.client.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []string
|
||||
var v string
|
||||
for rows.Next() {
|
||||
if err := rows.Scan(&v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -1,184 +0,0 @@
|
||||
package clickhouseprometheusv2
|
||||
|
||||
import (
|
||||
"sort"
|
||||
|
||||
"github.com/prometheus/prometheus/model/histogram"
|
||||
"github.com/prometheus/prometheus/model/labels"
|
||||
"github.com/prometheus/prometheus/storage"
|
||||
"github.com/prometheus/prometheus/tsdb/chunkenc"
|
||||
"github.com/prometheus/prometheus/util/annotations"
|
||||
)
|
||||
|
||||
// series is one time series with samples as parallel slices ordered by
|
||||
// timestamp. Deliberately not storage.NewListSeries: that boxes every sample
|
||||
// as an interface value, a per-sample allocation this fetch path exists to
|
||||
// avoid.
|
||||
type series struct {
|
||||
lset labels.Labels
|
||||
ts []int64
|
||||
vs []float64
|
||||
}
|
||||
|
||||
var _ storage.Series = (*series)(nil)
|
||||
|
||||
func (s *series) Labels() labels.Labels {
|
||||
return s.lset
|
||||
}
|
||||
|
||||
func (s *series) Iterator(it chunkenc.Iterator) chunkenc.Iterator {
|
||||
if fit, ok := it.(*floatIterator); ok {
|
||||
fit.reset(s)
|
||||
return fit
|
||||
}
|
||||
fit := &floatIterator{}
|
||||
fit.reset(s)
|
||||
return fit
|
||||
}
|
||||
|
||||
// floatIterator implements chunkenc.Iterator over a series' sample slices.
|
||||
type floatIterator struct {
|
||||
s *series
|
||||
i int
|
||||
}
|
||||
|
||||
var _ chunkenc.Iterator = (*floatIterator)(nil)
|
||||
|
||||
func (it *floatIterator) reset(s *series) {
|
||||
it.s = s
|
||||
it.i = -1
|
||||
}
|
||||
|
||||
func (it *floatIterator) Next() chunkenc.ValueType {
|
||||
it.i++
|
||||
if it.i >= len(it.s.ts) {
|
||||
return chunkenc.ValNone
|
||||
}
|
||||
return chunkenc.ValFloat
|
||||
}
|
||||
|
||||
func (it *floatIterator) Seek(t int64) chunkenc.ValueType { //nolint:govet // stdmethods flags io.Seeker; this is chunkenc.Iterator's Seek
|
||||
if it.i < 0 {
|
||||
it.i = 0
|
||||
}
|
||||
if it.i >= len(it.s.ts) {
|
||||
return chunkenc.ValNone
|
||||
}
|
||||
// The current position, once valid, must not move backwards.
|
||||
if it.s.ts[it.i] >= t {
|
||||
return chunkenc.ValFloat
|
||||
}
|
||||
it.i += sort.Search(len(it.s.ts)-it.i, func(j int) bool {
|
||||
return it.s.ts[it.i+j] >= t
|
||||
})
|
||||
if it.i >= len(it.s.ts) {
|
||||
return chunkenc.ValNone
|
||||
}
|
||||
return chunkenc.ValFloat
|
||||
}
|
||||
|
||||
func (it *floatIterator) At() (int64, float64) {
|
||||
return it.s.ts[it.i], it.s.vs[it.i]
|
||||
}
|
||||
|
||||
func (it *floatIterator) AtHistogram(*histogram.Histogram) (int64, *histogram.Histogram) {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
func (it *floatIterator) AtFloatHistogram(*histogram.FloatHistogram) (int64, *histogram.FloatHistogram) {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
func (it *floatIterator) AtT() int64 {
|
||||
return it.s.ts[it.i]
|
||||
}
|
||||
|
||||
// AtST returns the current start timestamp; not tracked by this storage.
|
||||
func (it *floatIterator) AtST() int64 {
|
||||
return 0
|
||||
}
|
||||
|
||||
func (it *floatIterator) Err() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// seriesSet iterates a fully materialized, label-sorted list of series.
|
||||
type seriesSet struct {
|
||||
series []*series
|
||||
i int
|
||||
}
|
||||
|
||||
var _ storage.SeriesSet = (*seriesSet)(nil)
|
||||
|
||||
func newSeriesSet(list []*series) *seriesSet {
|
||||
return &seriesSet{series: list, i: -1}
|
||||
}
|
||||
|
||||
func (s *seriesSet) Next() bool {
|
||||
s.i++
|
||||
return s.i < len(s.series)
|
||||
}
|
||||
|
||||
func (s *seriesSet) At() storage.Series {
|
||||
return s.series[s.i]
|
||||
}
|
||||
|
||||
func (s *seriesSet) Err() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *seriesSet) Warnings() annotations.Annotations {
|
||||
return nil
|
||||
}
|
||||
|
||||
// sortAndMerge orders series by label set and merges identical label sets
|
||||
// by timestamp (first sample wins ties): distinct fingerprints can carry
|
||||
// identical label sets, and the engine assumes storages never emit
|
||||
// duplicates.
|
||||
func sortAndMerge(list []*series) []*series {
|
||||
if len(list) < 2 {
|
||||
return list
|
||||
}
|
||||
sort.Slice(list, func(i, j int) bool {
|
||||
return labels.Compare(list[i].lset, list[j].lset) < 0
|
||||
})
|
||||
out := list[:1]
|
||||
for _, s := range list[1:] {
|
||||
last := out[len(out)-1]
|
||||
if labels.Compare(last.lset, s.lset) != 0 {
|
||||
out = append(out, s)
|
||||
continue
|
||||
}
|
||||
merged := mergeSamples(last, s)
|
||||
out[len(out)-1] = merged
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func mergeSamples(a, b *series) *series {
|
||||
ts := make([]int64, 0, len(a.ts)+len(b.ts))
|
||||
vs := make([]float64, 0, len(a.ts)+len(b.ts))
|
||||
i, j := 0, 0
|
||||
for i < len(a.ts) && j < len(b.ts) {
|
||||
switch {
|
||||
case a.ts[i] < b.ts[j]:
|
||||
ts = append(ts, a.ts[i])
|
||||
vs = append(vs, a.vs[i])
|
||||
i++
|
||||
case a.ts[i] > b.ts[j]:
|
||||
ts = append(ts, b.ts[j])
|
||||
vs = append(vs, b.vs[j])
|
||||
j++
|
||||
default:
|
||||
ts = append(ts, a.ts[i])
|
||||
vs = append(vs, a.vs[i])
|
||||
i++
|
||||
j++
|
||||
}
|
||||
}
|
||||
ts = append(ts, a.ts[i:]...)
|
||||
vs = append(vs, a.vs[i:]...)
|
||||
ts = append(ts, b.ts[j:]...)
|
||||
vs = append(vs, b.vs[j:]...)
|
||||
return &series{lset: a.lset, ts: ts, vs: vs}
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
package clickhouseprometheusv2
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/huandu/go-sqlbuilder"
|
||||
"github.com/prometheus/prometheus/model/labels"
|
||||
)
|
||||
|
||||
// metricNameLabel is the reserved PromQL label holding the metric name.
|
||||
const metricNameLabel = "__name__"
|
||||
|
||||
// buildSeriesQuery renders the series lookup: one row per matched fingerprint
|
||||
// with its labels.
|
||||
func buildSeriesQuery(start, end int64, matchers []*labels.Matcher) (string, []any, error) {
|
||||
// The series tables hold one row per (fingerprint, bucket) at 1h/6h/1d/1w
|
||||
// granularities; the schema package picks the table whose bucket fits the
|
||||
// window and rounds the start down to the bucket boundary, so a window
|
||||
// beginning mid-bucket still matches the bucket's row.
|
||||
adjustedStart, _, table, _ := metricstelemetryschema.WhichTSTableToUse(uint64(start), uint64(end), false, nil)
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("fingerprint", "any(labels)")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, table))
|
||||
if err := applySeriesConditions(sb, int64(adjustedStart), end, matchers); err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
sb.GroupBy("fingerprint")
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
return query, args, nil
|
||||
}
|
||||
|
||||
// buildSamplesQuery renders the samples fetch for the matched series: a
|
||||
// semi-join re-runs the series predicates against the shard-local series
|
||||
// table (complete by fingerprint co-locality, see localTimeSeriesTable — a
|
||||
// GLOBAL broadcast would ship the matched set to every shard, and ClickHouse
|
||||
// materializes the subquery's set per shard before the scan, so it still
|
||||
// engages the fingerprint primary-key column). metricNames (observed on the
|
||||
// matched series when the selector had no __name__ equality) narrows the
|
||||
// primary-key scan. A non-nil lastPerStep groups to the last sample per
|
||||
// step bucket.
|
||||
func buildSamplesQuery(start, end int64, metricNames []string, matchers []*labels.Matcher, lastPerStep *lastSamplePerStep) (string, []any, error) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
if lastPerStep != nil {
|
||||
// Aliases must not shadow source columns: ClickHouse resolves aliases
|
||||
// in WHERE too, and "max(unix_milli) AS unix_milli" would put an
|
||||
// aggregate into the WHERE clause (error 184).
|
||||
sb.Select("fingerprint", "max(unix_milli) AS ts", "argMax(value, unix_milli) AS val", "argMax(flags, unix_milli) AS fl")
|
||||
} else {
|
||||
sb.Select("fingerprint", "unix_milli", "value", "flags")
|
||||
}
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName))
|
||||
|
||||
switch len(metricNames) {
|
||||
case 0:
|
||||
// No name constraint derivable; the primary-key prefix goes unused.
|
||||
case 1:
|
||||
sb.Where(sb.EQ("metric_name", metricNames[0]))
|
||||
default:
|
||||
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
|
||||
}
|
||||
// Semantically redundant (the fingerprints already come from these
|
||||
// temporalities) but engages the leading primary-key column.
|
||||
sb.Where("temporality IN ['Cumulative', 'Unspecified']")
|
||||
sub := sqlbuilder.NewSelectBuilder()
|
||||
sub.Select("fingerprint")
|
||||
adjustedStart, _, _, localTable := metricstelemetryschema.WhichTSTableToUse(uint64(start), uint64(end), false, nil)
|
||||
sub.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTable))
|
||||
if err := applySeriesConditions(sub, int64(adjustedStart), end, matchers); err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
sb.Where(sb.In("fingerprint", sub))
|
||||
sb.Where(sb.GTE("unix_milli", start), sb.LTE("unix_milli", end))
|
||||
|
||||
if lastPerStep != nil {
|
||||
sb.GroupBy("fingerprint")
|
||||
if expr := lastPerStep.bucketExpr(); expr != "" {
|
||||
sb.GroupBy(expr)
|
||||
}
|
||||
sb.OrderBy("fingerprint", "ts")
|
||||
} else {
|
||||
sb.OrderBy("fingerprint", "unix_milli")
|
||||
}
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
return query, args, nil
|
||||
}
|
||||
|
||||
// applySeriesConditions adds the WHERE conditions of a series table scan:
|
||||
// __name__ matchers (all four types) translate to the metric_name column,
|
||||
// every other matcher to a JSONExtractString condition on the labels column.
|
||||
// An equality matcher against "" matches series without the label, mirroring
|
||||
// PromQL, because JSONExtractString returns "" for missing keys. Regexes are
|
||||
// anchored: PromQL matchers match the whole value, ClickHouse match()
|
||||
// searches for a substring.
|
||||
func applySeriesConditions(sb *sqlbuilder.SelectBuilder, start, end int64, matchers []*labels.Matcher) error {
|
||||
for _, m := range matchers {
|
||||
if m.Name != metricNameLabel {
|
||||
continue
|
||||
}
|
||||
switch m.Type {
|
||||
case labels.MatchEqual:
|
||||
sb.Where(sb.EQ("metric_name", m.Value))
|
||||
case labels.MatchNotEqual:
|
||||
sb.Where(sb.NE("metric_name", m.Value))
|
||||
case labels.MatchRegexp:
|
||||
sb.Where(fmt.Sprintf("match(metric_name, %s)", sb.Var(anchorRegex(m.Value))))
|
||||
case labels.MatchNotRegexp:
|
||||
sb.Where(fmt.Sprintf("NOT match(metric_name, %s)", sb.Var(anchorRegex(m.Value))))
|
||||
default:
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported matcher type %q for __name__", m.Type)
|
||||
}
|
||||
}
|
||||
|
||||
sb.Where("temporality IN ['Cumulative', 'Unspecified']")
|
||||
// Inclusive upper bound: registration rows are hour-floored (and 6h/1d/1w
|
||||
// for the rollup tables) by the exporter, so a series first registered in
|
||||
// the bucket starting exactly at `end` would otherwise be invisible while
|
||||
// its samples (<= end) are in range.
|
||||
sb.Where(sb.GTE("unix_milli", start), sb.LTE("unix_milli", end))
|
||||
|
||||
for _, m := range matchers {
|
||||
if m.Name == metricNameLabel {
|
||||
continue
|
||||
}
|
||||
switch m.Type {
|
||||
case labels.MatchEqual:
|
||||
sb.Where(fmt.Sprintf("JSONExtractString(labels, %s) = %s", sb.Var(m.Name), sb.Var(m.Value)))
|
||||
case labels.MatchNotEqual:
|
||||
sb.Where(fmt.Sprintf("JSONExtractString(labels, %s) != %s", sb.Var(m.Name), sb.Var(m.Value)))
|
||||
case labels.MatchRegexp:
|
||||
sb.Where(fmt.Sprintf("match(JSONExtractString(labels, %s), %s)", sb.Var(m.Name), sb.Var(anchorRegex(m.Value))))
|
||||
case labels.MatchNotRegexp:
|
||||
sb.Where(fmt.Sprintf("NOT match(JSONExtractString(labels, %s), %s)", sb.Var(m.Name), sb.Var(anchorRegex(m.Value))))
|
||||
default:
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported matcher type %q", m.Type)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// anchorRegex turns a PromQL regex into its fully-anchored form (see
|
||||
// applySeriesConditions).
|
||||
func anchorRegex(v string) string {
|
||||
return "^(?:" + v + ")$"
|
||||
}
|
||||
|
||||
// lastSamplePerStep reduces an instant-selector fetch to the last sample of
|
||||
// each step bucket: bucket 0 is (start, firstEval], bucket i is
|
||||
// (firstEval+(i-1)·step, firstEval+i·step]. Anchoring buckets at the first
|
||||
// evaluation timestamp makes every bucket boundary an evaluation timestamp,
|
||||
// so a non-final sample of a bucket can never be the latest sample in any
|
||||
// (t-lookback, t] the engine resolves — the reduction is lossless. Real
|
||||
// timestamps are preserved, so the engine's own lookback and staleness
|
||||
// handling remain exact.
|
||||
type lastSamplePerStep struct {
|
||||
firstEvalMs int64
|
||||
stepMs int64
|
||||
}
|
||||
|
||||
func (t *lastSamplePerStep) bucketExpr() string {
|
||||
if t.stepMs <= 0 {
|
||||
// Instant query: a single evaluation at firstEval; one bucket.
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf(
|
||||
"if(unix_milli <= %d, 0, intDiv(unix_milli - %d - 1, %d) + 1)",
|
||||
t.firstEvalMs, t.firstEvalMs, t.stepMs,
|
||||
)
|
||||
}
|
||||
@@ -1,115 +0,0 @@
|
||||
package clickhouseprometheusv2
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/prometheus/model/labels"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func mustMatcher(t *testing.T, mt labels.MatchType, name, value string) *labels.Matcher {
|
||||
t.Helper()
|
||||
m, err := labels.NewMatcher(mt, name, value)
|
||||
require.NoError(t, err)
|
||||
return m
|
||||
}
|
||||
|
||||
func TestBuildSeriesQuery(t *testing.T) {
|
||||
start := int64(1_700_000_000_000)
|
||||
end := start + time.Hour.Milliseconds()
|
||||
// The series table window rounds down to the table's bucket boundary.
|
||||
adjustedStart := start - (start % time.Hour.Milliseconds())
|
||||
|
||||
t.Run("equality name and label matchers", func(t *testing.T) {
|
||||
query, args, err := buildSeriesQuery(start, end, []*labels.Matcher{
|
||||
mustMatcher(t, labels.MatchEqual, "__name__", "http_requests_total"),
|
||||
mustMatcher(t, labels.MatchEqual, "job", "api"),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
"SELECT fingerprint, any(labels) FROM signoz_metrics.distributed_time_series_v4 WHERE metric_name = ? AND temporality IN ['Cumulative', 'Unspecified'] AND unix_milli >= ? AND unix_milli <= ? AND JSONExtractString(labels, ?) = ? GROUP BY fingerprint",
|
||||
query,
|
||||
)
|
||||
assert.Equal(t, []any{"http_requests_total", adjustedStart, end, "job", "api"}, args)
|
||||
})
|
||||
|
||||
t.Run("regex matchers are anchored", func(t *testing.T) {
|
||||
_, args, err := buildSeriesQuery(start, end, []*labels.Matcher{
|
||||
mustMatcher(t, labels.MatchEqual, "__name__", "up"),
|
||||
mustMatcher(t, labels.MatchRegexp, "instance", "prod.*"),
|
||||
mustMatcher(t, labels.MatchNotRegexp, "env", "dev|test"),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []any{"up", adjustedStart, end, "instance", "^(?:prod.*)$", "env", "^(?:dev|test)$"}, args)
|
||||
})
|
||||
|
||||
t.Run("regex name matcher uses metric_name column", func(t *testing.T) {
|
||||
query, args, err := buildSeriesQuery(start, end, []*labels.Matcher{
|
||||
mustMatcher(t, labels.MatchRegexp, "__name__", "node_cpu.*|node_memory.*"),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, query, "match(metric_name, ?)")
|
||||
assert.NotContains(t, query, "JSONExtractString")
|
||||
assert.Equal(t, []any{"^(?:node_cpu.*|node_memory.*)$", adjustedStart, end}, args)
|
||||
})
|
||||
|
||||
t.Run("no name matcher omits metric_name condition", func(t *testing.T) {
|
||||
query, _, err := buildSeriesQuery(start, end, []*labels.Matcher{
|
||||
mustMatcher(t, labels.MatchEqual, "job", "api"),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.NotContains(t, query, "metric_name")
|
||||
})
|
||||
}
|
||||
|
||||
func TestBuildSamplesQuery(t *testing.T) {
|
||||
start := int64(1_700_000_000_000)
|
||||
end := start + time.Hour.Milliseconds()
|
||||
adjustedStart := start - (start % time.Hour.Milliseconds())
|
||||
matchers := []*labels.Matcher{
|
||||
mustMatcher(t, labels.MatchEqual, "__name__", "up"),
|
||||
mustMatcher(t, labels.MatchEqual, "job", "api"),
|
||||
}
|
||||
|
||||
t.Run("raw fetch filters by a shard-local semi-join", func(t *testing.T) {
|
||||
query, args, err := buildSamplesQuery(start, end, []string{"up"}, matchers, nil)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, query, "fingerprint IN (SELECT fingerprint FROM signoz_metrics.time_series_v4 WHERE ")
|
||||
assert.NotContains(t, query, "GLOBAL IN")
|
||||
assert.Contains(t, query, "ORDER BY fingerprint, unix_milli")
|
||||
// Args follow placeholder order: samples metric name, the semi-join's
|
||||
// series predicates, then the samples window bounds.
|
||||
assert.Equal(t, []any{"up", "up", adjustedStart, end, "job", "api", start, end}, args)
|
||||
})
|
||||
|
||||
t.Run("last-sample-per-step groups by step bucket anchored at first eval", func(t *testing.T) {
|
||||
lastPerStep := &lastSamplePerStep{firstEvalMs: start + 299_999, stepMs: 60_000}
|
||||
query, _, err := buildSamplesQuery(start, end, []string{"up"}, matchers, lastPerStep)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, query, "argMax(value, unix_milli) AS val")
|
||||
assert.Contains(t, query, "argMax(flags, unix_milli) AS fl")
|
||||
assert.Contains(t, query, "GROUP BY fingerprint, if(unix_milli <= 1700000299999, 0, intDiv(unix_milli - 1700000299999 - 1, 60000) + 1)")
|
||||
assert.Contains(t, query, "ORDER BY fingerprint, ts")
|
||||
// Aliases must not shadow the source columns referenced in WHERE.
|
||||
assert.NotContains(t, query, "AS unix_milli")
|
||||
assert.NotContains(t, query, "AS value")
|
||||
assert.NotContains(t, query, "AS flags")
|
||||
})
|
||||
|
||||
t.Run("instant query keeps one bucket", func(t *testing.T) {
|
||||
lastPerStep := &lastSamplePerStep{firstEvalMs: end, stepMs: 0}
|
||||
query, _, err := buildSamplesQuery(start, end, []string{"up"}, matchers, lastPerStep)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, query, "GROUP BY fingerprint ORDER BY fingerprint, ts")
|
||||
assert.NotContains(t, query, "intDiv")
|
||||
})
|
||||
|
||||
t.Run("multiple metric names from regex selector", func(t *testing.T) {
|
||||
query, args, err := buildSamplesQuery(start, end, []string{"node_cpu", "node_memory"}, matchers, nil)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, query, "metric_name IN (?, ?)")
|
||||
assert.Equal(t, []any{"node_cpu", "node_memory", "up", adjustedStart, end, "job", "api", start, end}, args)
|
||||
})
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
package prometheus
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/prometheus/prometheus/promql/parser"
|
||||
)
|
||||
|
||||
type queryTraitsKey struct{}
|
||||
|
||||
// QueryTraits carries per-query facts a storage implementation cannot derive
|
||||
// from SelectHints alone. Call sites that parse the PromQL expression attach
|
||||
// traits to the context before handing it to the engine; storages treat a
|
||||
// missing traits value as "unknown" and stay conservative.
|
||||
type QueryTraits struct {
|
||||
// SubqueryFree is true when the query contains no subquery expression.
|
||||
// Subquery selectors are evaluated at the subquery's own step, but
|
||||
// SelectHints.Step always carries the top-level step, so step-aligned
|
||||
// storage optimizations (e.g. keeping only the last sample per step
|
||||
// bucket) are safe only when this is true.
|
||||
SubqueryFree bool
|
||||
}
|
||||
|
||||
// DetectQueryTraits derives QueryTraits from a parsed PromQL expression.
|
||||
func DetectQueryTraits(expr parser.Expr) QueryTraits {
|
||||
subqueryFree := true
|
||||
parser.Inspect(expr, func(node parser.Node, _ []parser.Node) error {
|
||||
if _, ok := node.(*parser.SubqueryExpr); ok {
|
||||
subqueryFree = false
|
||||
}
|
||||
return nil
|
||||
})
|
||||
return QueryTraits{SubqueryFree: subqueryFree}
|
||||
}
|
||||
|
||||
// NewContextWithQueryTraits returns a context carrying the given traits.
|
||||
func NewContextWithQueryTraits(ctx context.Context, traits QueryTraits) context.Context {
|
||||
return context.WithValue(ctx, queryTraitsKey{}, traits)
|
||||
}
|
||||
|
||||
// QueryTraitsFromContext returns the traits attached to ctx, if any.
|
||||
//
|
||||
// Context is used here, unlike for backend selection, because traits must
|
||||
// cross the promql engine to reach storage.Querier.Select, and the engine's
|
||||
// interfaces offer no other channel; the alternative is a Prometheus fork.
|
||||
func QueryTraitsFromContext(ctx context.Context) (QueryTraits, bool) {
|
||||
traits, ok := ctx.Value(queryTraitsKey{}).(QueryTraits)
|
||||
return traits, ok
|
||||
}
|
||||
9
pkg/templating/markdownrenderer/blockkit/doc.go
Normal file
9
pkg/templating/markdownrenderer/blockkit/doc.go
Normal file
@@ -0,0 +1,9 @@
|
||||
// Package blockkit provides a goldmark extension that renders markdown as
|
||||
// Slack Block Kit JSON (an array of blocks suitable for the Slack API's
|
||||
// `blocks` payload field).
|
||||
//
|
||||
// The current Slack notifier uses the sibling mrkdwn package instead, because
|
||||
// Block Kit's hard limits (one table per message, 50 blocks per message,
|
||||
// 12k-character total text) would silently truncate or reject notifications
|
||||
// with rich bodies. This package is kept in tree for future use.
|
||||
package blockkit
|
||||
3
pkg/templating/markdownrenderer/doc.go
Normal file
3
pkg/templating/markdownrenderer/doc.go
Normal file
@@ -0,0 +1,3 @@
|
||||
// Package markdownrenderer renders markdown to the formats that alert
|
||||
// notifications are delivered in: HTML, Slack Block Kit JSON, and Slack mrkdwn.
|
||||
package markdownrenderer
|
||||
4
pkg/templating/markdownrenderer/mrkdwn/doc.go
Normal file
4
pkg/templating/markdownrenderer/mrkdwn/doc.go
Normal file
@@ -0,0 +1,4 @@
|
||||
// Package mrkdwn provides a goldmark extension that renders markdown as
|
||||
// Slack's "mrkdwn" text format (the legacy text field used in attachments
|
||||
// and message payloads).
|
||||
package mrkdwn
|
||||
@@ -48,8 +48,8 @@ func (CompareOperator) Enum() []any {
|
||||
ValueIsBelowLiteral,
|
||||
ValueIsEqLiteral,
|
||||
ValueIsNotEqLiteral,
|
||||
ValueAboveOrEqLiteral,
|
||||
ValueBelowOrEqLiteral,
|
||||
// ValueAboveOrEqLiteral,
|
||||
// ValueBelowOrEqLiteral,
|
||||
ValueOutsideBoundsLiteral,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
pytest_plugins = [
|
||||
@@ -34,22 +32,6 @@ pytest_plugins = [
|
||||
]
|
||||
|
||||
|
||||
def pytest_configure(config: pytest.Config):
|
||||
if config.getoption("--rebuild"):
|
||||
if not config.getoption("--reuse"):
|
||||
raise pytest.UsageError("--rebuild requires --reuse: it replaces the signoz container within an environment that is being reused.")
|
||||
if config.getoption("--teardown"):
|
||||
raise pytest.UsageError("--rebuild cannot be combined with --teardown.")
|
||||
if config.getoption("--clean"):
|
||||
raise pytest.UsageError("--rebuild cannot be combined with --clean: --clean forces a cold build, which defeats the purpose of --rebuild.")
|
||||
|
||||
|
||||
def pytest_sessionstart(session: pytest.Session):
|
||||
if session.config.getoption("--clean"):
|
||||
# The type filter removes only cache mounts, leaving images and layer cache intact.
|
||||
subprocess.run(["docker", "builder", "prune", "--force", "--filter", "type=exec.cachemount"], check=True)
|
||||
|
||||
|
||||
def pytest_addoption(parser: pytest.Parser):
|
||||
parser.addoption(
|
||||
"--reuse",
|
||||
@@ -63,18 +45,6 @@ def pytest_addoption(parser: pytest.Parser):
|
||||
default=False,
|
||||
help="Teardown environment. Run pytest --basetemp=./tmp/ -vv --teardown src/bootstrap/setup::test_teardown to teardown your local dev environment.",
|
||||
)
|
||||
parser.addoption(
|
||||
"--rebuild",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Rebuild the signoz container from the current sources while reusing the rest of the stack (databases, mocks, migrations). Only meaningful together with --reuse: pytest --basetemp=./tmp/ -vv --reuse --rebuild integration/bootstrap/setup.py::test_setup.",
|
||||
)
|
||||
parser.addoption(
|
||||
"--clean",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Prune the BuildKit cache mounts (go build and module caches) used by the signoz image build, forcing the next build to start cold. Combine with --teardown to reset everything: pytest --basetemp=./tmp/ -vv --teardown --clean integration/bootstrap/setup.py::test_teardown.",
|
||||
)
|
||||
parser.addoption(
|
||||
"--with-web",
|
||||
action="store_true",
|
||||
|
||||
11
tests/fixtures/reuse.py
vendored
11
tests/fixtures/reuse.py
vendored
@@ -41,7 +41,6 @@ def wrap( # pylint: disable=too-many-arguments,too-many-positional-arguments
|
||||
create: Callable[[], T],
|
||||
delete: Callable[[T], None],
|
||||
restore: Callable[[dict], T],
|
||||
rebuild: bool = False,
|
||||
) -> T:
|
||||
"""
|
||||
Wraps a resource creation and cleanup process with reuse and teardown options.
|
||||
@@ -52,7 +51,6 @@ def wrap( # pylint: disable=too-many-arguments,too-many-positional-arguments
|
||||
- create: function to create the resource
|
||||
- delete: function to delete the resource
|
||||
- restore: function to restore resource from cache
|
||||
- rebuild: under --reuse, delete the cached resource and recreate it instead of restoring it
|
||||
"""
|
||||
resource = empty()
|
||||
|
||||
@@ -60,13 +58,8 @@ def wrap( # pylint: disable=too-many-arguments,too-many-positional-arguments
|
||||
existing_resource = pytestconfig.cache.get(key, None)
|
||||
if existing_resource:
|
||||
assert isinstance(existing_resource, dict)
|
||||
if rebuild:
|
||||
logger.info("Rebuilding %s(%s), removing the existing one", key, existing_resource)
|
||||
delete(restore(existing_resource))
|
||||
pytestconfig.cache.set(key, None)
|
||||
else:
|
||||
logger.info("Reusing existing %s(%s)", key, existing_resource)
|
||||
return restore(existing_resource)
|
||||
logger.info("Reusing existing %s(%s)", key, existing_resource)
|
||||
return restore(existing_resource)
|
||||
|
||||
if not teardown(request):
|
||||
resource = create()
|
||||
|
||||
34
tests/fixtures/signoz.py
vendored
34
tests/fixtures/signoz.py
vendored
@@ -1,6 +1,4 @@
|
||||
import os
|
||||
import platform
|
||||
import subprocess
|
||||
import time
|
||||
from http import HTTPStatus
|
||||
from os import path
|
||||
@@ -10,6 +8,7 @@ import docker.errors
|
||||
import pytest
|
||||
import requests
|
||||
from testcontainers.core.container import DockerContainer, Network
|
||||
from testcontainers.core.image import DockerImage
|
||||
|
||||
from fixtures import reuse, types
|
||||
from fixtures.logger import setup_logger
|
||||
@@ -51,28 +50,18 @@ def create_signoz(
|
||||
|
||||
# Docker build context is the repo root — one up from pytest's
|
||||
# rootdir (tests/).
|
||||
context = pytestconfig.rootpath.parent
|
||||
|
||||
# The docker CLI is required: the Dockerfiles use BuildKit cache
|
||||
# mounts, which docker-py does not support.
|
||||
subprocess.run(
|
||||
[
|
||||
"docker",
|
||||
"build",
|
||||
"--file",
|
||||
str(context / dockerfile_path),
|
||||
"--tag",
|
||||
"signoz:integration",
|
||||
"--build-arg",
|
||||
f"TARGETARCH={arch}",
|
||||
"--build-arg",
|
||||
f"ZEUSURL={zeus.container_configs['8080'].base()}",
|
||||
str(context),
|
||||
],
|
||||
check=True,
|
||||
env=os.environ | {"DOCKER_BUILDKIT": "1"},
|
||||
self = DockerImage(
|
||||
path=str(pytestconfig.rootpath.parent),
|
||||
dockerfile_path=dockerfile_path,
|
||||
tag="signoz:integration",
|
||||
buildargs={
|
||||
"TARGETARCH": arch,
|
||||
"ZEUSURL": zeus.container_configs["8080"].base(),
|
||||
},
|
||||
)
|
||||
|
||||
self.build()
|
||||
|
||||
env = (
|
||||
{
|
||||
"SIGNOZ_WEB_ENABLED": False,
|
||||
@@ -211,7 +200,6 @@ def create_signoz(
|
||||
create=create,
|
||||
delete=delete,
|
||||
restore=restore,
|
||||
rebuild=pytestconfig.getoption("--rebuild"),
|
||||
)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user