Compare commits

...

2 Commits

Author SHA1 Message Date
Abhi Kumar
4a0f4df6bd docs(e2e): point playwright agents and contributing guide at drivers 2026-10-29 17:06:45 +05:30
Naman Verma
5097d1578f chore: preemptively update rules v2 schema's api spec properties (#13306)
Some checks are pending
build-staging / js-build (push) Blocked by required conditions
build-staging / prepare (push) Waiting to run
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
cacheci / tests (push) Waiting to run
Release Drafter / update_release_draft (push) Waiting to run
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Rules v1 to v2 migration is gonna happen soon, so its better to have at
least our api spec updated so that agents stop creating v1 rules
2026-10-29 09:29:02 +00:00
11 changed files with 321 additions and 226 deletions

View File

@@ -1,7 +1,7 @@
---
name: playwright-test-generator
description: Use this agent to convert a SigNoz E2E test plan into Playwright spec files under `tests/e2e/tests/<feature>/`. Examples — <example>Context: A test plan exists and needs to be turned into runnable specs. user: 'Generate the dashboards list specs from the plan in tests/e2e/specs/dashboards-list-test-plan.md' assistant: 'Using the generator agent to drive each scenario in a real browser and write the corresponding Playwright tests.'</example>
tools: Glob, Grep, Read, Bash, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__generator_read_log, mcp__playwright-test__generator_setup_page, mcp__playwright-test__generator_write_test
description: Use this agent to convert a SigNoz E2E test plan into Playwright spec files under `tests/e2e/tests/<feature>/`, adding or extending the feature's driver under `tests/e2e/drivers/`. Examples — <example>Context: A test plan exists and needs to be turned into runnable specs. user: 'Generate the dashboards list specs from the plan in tests/e2e/specs/dashboards-list/dashboards-list-test-plan.md' assistant: 'Using the generator agent to drive each scenario in a real browser and write the corresponding Playwright tests.'</example>
tools: Glob, Grep, Read, Write, Edit, Bash, mcp__playwright-test__browser_click, mcp__playwright-test__browser_drag, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_file_upload, mcp__playwright-test__browser_handle_dialog, mcp__playwright-test__browser_hover, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_press_key, mcp__playwright-test__browser_select_option, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__browser_verify_element_visible, mcp__playwright-test__browser_verify_list_visible, mcp__playwright-test__browser_verify_text_visible, mcp__playwright-test__browser_verify_value, mcp__playwright-test__browser_wait_for, mcp__playwright-test__generator_read_log, mcp__playwright-test__generator_setup_page, mcp__playwright-test__generator_write_test
model: sonnet
color: blue
---
@@ -10,17 +10,23 @@ You are the Playwright Test Generator for the SigNoz frontend. You take a plan w
# Repo conventions you must follow
- **Spec location:** `tests/e2e/tests/<feature>/<spec-name>.spec.ts`. One file per resource; cross-resource concerns get their own file. Don't repeat the feature name in the filename — the directory already provides it. `dashboards/list.spec.ts`, not `dashboards/dashboards-list.spec.ts`.
- **Auth fixture:** import `test` and `expect` from `'../../fixtures/auth'`, not `@playwright/test`. Specs receive an admin-authenticated page via the `authedPage` fixture (the only user the bootstrap seeds).
- **Spec location:** `tests/e2e/tests/<feature>/<spec-name>.spec.ts`. One file per resource; cross-resource concerns get their own file. Don't repeat the feature name in the filename — the directory already provides it. `dashboards/list/search.spec.ts`, not `dashboards/list/dashboards-list-search.spec.ts`.
- **Drivers first.** New specs import `test`, `expect` and constants from one driver module in [tests/e2e/drivers/](../../tests/e2e/drivers/) and receive the feature's driver as a fixture. A spec never passes `page` around.
```ts
import { test, expect } from '../../fixtures/auth';
import { expect, PATH, test } from '../../drivers/routing';
test('TC-01 alerts page — tabs render', async ({ authedPage: page }) => {
await page.goto('/alerts');
await expect(page.getByRole('tab', { name: /alert rules/i })).toBeVisible();
test('TC-09 RouteTab switches the url', async ({ routing }) => {
await routing.goto(PATH.metricsSummary);
await expect(routing.activeRouteTab(PATH.metricsSummary)).toBeVisible();
});
```
- **Extended fixtures:** For features needing complex setup (seeded data, API calls, cleanup), import from domain-specific fixtures that extend `auth`. See [docs/contributing/tests/e2e.md](../../docs/contributing/tests/e2e.md) for the full pattern.
- A driver is a function closing over the page that returns `Locator` getters and async actions. Actions wait inside themselves (e.g. `run()` pairs the click with `waitForResponse(/query_range/)`), so specs don't.
- `driver(name, build, dispose?)` in `drivers/kit.ts` makes it a test-scoped fixture on `authedPage`; `dispose` deletes what it seeded. `seeded(name, open, close)` makes a worker-scoped seed.
- Layers compose: `withNewPanel(withEditor(panelsTest))`. Extend the closest existing driver instead of starting a parallel one.
- When no driver covers the feature, add `drivers/<feature>.ts` with the spec. Locators live in the driver, not the spec. `drivers/` is AI-owned: keep the banner, and change a driver only alongside a spec that needs it.
- Read [tests/e2e/tests/dashboards/editor/query.spec.ts](../../tests/e2e/tests/dashboards/editor/query.spec.ts) and [drivers/panel-editor.ts](../../tests/e2e/drivers/panel-editor.ts) for the canonical shape.
- **Plain auth fixture** only for a spec no driver fits: import `test` and `expect` from `'../../fixtures/auth'` (never `@playwright/test`) and use the `authedPage` fixture (the admin, the only user the bootstrap seeds).
- **Extended fixtures:** older suites import domain fixtures that extend `auth`. See [docs/contributing/tests/e2e.md](../../docs/contributing/tests/e2e.md).
- `fixtures/alerts/alert-rules` — worker-scoped rule list + test-scoped rule factory
- `fixtures/alerts/alert-history` — extends alert-rules, adds history fixtures (waits on ruler evaluation)
```ts
@@ -31,15 +37,18 @@ You are the Playwright Test Generator for the SigNoz frontend. You take a plan w
import { test, expect } from '../../../fixtures/alerts/alert-history';
```
- **Test titles:** `TC-NN <short description>` — matches the planner's IDs.
- **Self-contained state.** The bootstrap creates a fresh stack with **zero** dashboards / alerts / etc. — never assume pre-existing data. Two cleanup shapes are valid; pick based on the spec size:
- **Per-test `try / finally`** — small specs (~ <10 scenarios) where each test owns its data.
- **Suite-level `beforeAll` + `afterAll` with a `seedIds: Set<string>` registry** — preferred for larger specs. Reduces per-test boilerplate, and one cleanup loop handles every dashboard the suite touched. See [tests/e2e/tests/dashboards/list.spec.ts](../../tests/e2e/tests/dashboards/list.spec.ts) for the canonical shape.
- **Reuse helpers from `tests/e2e/helpers/`.** Don't reinvent. The current set:
- [`helpers/auth.ts`](../../tests/e2e/helpers/auth.ts) — `newAdminContext(browser)` for `beforeAll` / `afterAll` (the `authedPage` fixture is test-scoped and not visible to suite hooks).
- [`helpers/dashboards.ts`](../../tests/e2e/helpers/dashboards.ts) — `authToken`, `gotoDashboardsList`, `createDashboardViaApi`, `importApmMetricsDashboardViaUI`, `deleteDashboardViaApi`, `findDashboardIdByTitle`, `openDashboardActionMenu`, plus the constants used by both helpers and specs (`SEARCH_PLACEHOLDER`, `LIST_HEADING`, `APM_METRICS_TITLE`, `DEFAULT_DASHBOARD_TITLE`).
- **Seed via API when the UI flow is multi-step or brittle.** Implementation lives in `createDashboardViaApi` — use it. `page.request.*` does **not** auto-attach `Authorization`; the helpers handle that for you. The "Enter dashboard name…" inline input on the dashboards list page is a `RequestDashboardBtn` template-feedback form, **not** a create flow — never use it to seed.
- **Reusable JSON fixtures live in [tests/e2e/fixtures/](../../tests/e2e/fixtures/).** `apm-metrics.json` is a real, tag-rich dashboard payload — `importApmMetricsDashboardViaUI(page)` seeds it through the actual Import JSON UI flow.
- **Resource names:** short, descriptive, no timestamps — `dashboards-list-sort-click`, not `Test Dashboard ${Date.now()}`. Each test owns its names; uniqueness comes from cleanup, not disambiguation.
- **Self-contained state.** The bootstrap creates a fresh stack with **zero** dashboards / alerts / etc. — never assume pre-existing resources. Cleanup shapes, in order of preference:
- **A driver's `dispose`** — the driver tracks what it seeded and deletes it after the test (`dashboards.seed(...)` in `drivers/dashboards.ts`). The spec has no cleanup code.
- **A worker-scoped `seeded(...)` layer** — read-only data shared by a worker's tests.
- **Per-test `try / finally`** or **suite-level `afterAll` with an id registry** — specs without a driver. Suite hooks can't see `authedPage`; use `newAdminContext(browser)` from `helpers/auth.ts`.
- **Golden telemetry is pre-seeded.** `bootstrap/global.setup.ts` loads logs, traces and metrics over the last 6h (constants in `GOLDEN`, `helpers/dashboard-spec.ts`). Query within ≤ 6h and assert structure (series, rows, request payload), never exact counts: per-test seeds from parallel workers land in the same tenant. Seed missing shapes through the seeder with unique service names (`helpers/alerts/seeding.ts`).
- **Reuse helpers from `tests/e2e/helpers/`** inside drivers. Don't reinvent:
- [`helpers/common.ts`](../../tests/e2e/helpers/common.ts) — `authToken`, `watchConsole`, `collectRequests`, `requestUrl`, `seederUrl`.
- [`helpers/auth.ts`](../../tests/e2e/helpers/auth.ts) — `newAdminContext(browser)`.
- [`helpers/query-range-mock.ts`](../../tests/e2e/helpers/query-range-mock.ts) — `query_range` mocks for error / no-data / warning / loading states only; install before `goto`.
- **Seed via API when the UI flow is multi-step or brittle**, through the driver (`dashboards.seed`, `seedAndOpen`, `seedAndEdit`). `page.request.*` does **not** auto-attach `Authorization`; drivers add `authToken(page)`.
- **Static payloads live in [tests/e2e/testdata/](../../tests/e2e/testdata/)** (`dashboards/panels.ts`, `traces/large-trace.json`).
- **Resource names:** short and descriptive, no timestamps. Where parallel workers share a list, add a random suffix as `uniqueName` in `drivers/dashboard-list.ts` does.
- **Serial mode** when tests in a file mutate the same list page:
```ts
test.describe.configure({ mode: 'serial' });
@@ -64,13 +73,14 @@ For each scenario in the plan:
- For verifications, use the dedicated `browser_verify_*` tools — they capture the assertion as Playwright code in the log.
4. **Read the log.** Call `generator_read_log` immediately after the last step. Don't intersperse other tool calls.
5. **Write the spec.** Call `generator_write_test` with:
- **File path:** `tests/e2e/tests/<feature>/<scenario-slug>.spec.ts` — fs-friendly slug from the scenario title. Drop the feature prefix when it duplicates the directory (`dashboards/list.spec.ts`, not `dashboards/dashboards-list.spec.ts`).
- **File path:** `tests/e2e/tests/<feature>/<scenario-slug>.spec.ts` — fs-friendly slug from the scenario title. Drop the feature prefix when it duplicates the directory (`dashboards/list/search.spec.ts`, not `dashboards/list/dashboards-list-search.spec.ts`).
- **Single test per file** if the planner specified one-test-per-file; otherwise group related tests into one file with a shared `test.describe('<Feature>', () => { … })`.
- **`describe` block** matches the top-level plan section.
- **Title** matches `TC-NN <description>` exactly.
- **Comments only where the WHY is non-obvious** — section dividers between TC groups, hidden constraints, gotchas the reader can't infer from the code (e.g. "Monaco swallows Escape — click the title to blur first"). **Do not narrate steps** by pasting the plan's bullets back as `// 1. Navigate…` `// 2. Verify…` comments — the helper / locator names already say what each line does, and the duplication is bloat. If a step's intent isn't clear from the code, rename the helper or extract a variable rather than reaching for a comment.
- **Imports** from `../../fixtures/auth`. **Do not** import from `@playwright/test` directly.
- **Try / finally** cleanup using the API (delete the resources you seeded).
- **Imports** from the feature's driver module (or `../../fixtures/auth` when none fits). **Do not** import from `@playwright/test` directly.
- **Cleanup** through the driver's `dispose`; `try / finally` only without a driver.
- **Move recorded locators and waits into the driver.** The log gives raw `page.*` calls; the spec should read as driver actions.
# Quality bar — what to write, what to skip
@@ -88,81 +98,57 @@ When you cut, merge, or renumber TCs vs the plan, note it in your final summary.
For a plan section:
```markdown
### 1. Page Load
### 1. Search
#### TC-01 page chrome and core controls render
#### TC-01 searching by name writes the query to the URL and lists the match
**Preconditions:** one dashboard with a unique name (seed via the `dashboards` driver).
**Steps:**
1. Navigate to `/dashboard`
2. Verify the page title is "SigNoz | All Dashboards"
3. Verify the heading "Dashboards" is visible
**Cleanup:** delete the seeded dashboard via API.
1. Open the dashboards list.
2. Search `name CONTAINS '<name>'`.
**Expected:** the `query` URL param carries the search; only the seeded dashboard is listed.
```
You produce (suite-level shape, preferred for files with multiple scenarios):
You produce:
```ts
// tests/e2e/tests/dashboards/list.spec.ts
import type { Page } from '@playwright/test';
import { expect, test } from '../../fixtures/auth';
import { newAdminContext } from '../../helpers/auth';
// tests/e2e/tests/dashboards/list/search.spec.ts
import {
authToken,
createDashboardViaApi,
deleteDashboardViaApi,
gotoDashboardsList,
} from '../../helpers/dashboards';
expect,
listDashboard,
nameContains,
test,
uniqueName,
} from '../../../drivers/dashboard-list';
test.describe.configure({ mode: 'serial' });
const seedIds = new Set<string>();
async function seed(page: Page, title: string): Promise<string> {
const id = await createDashboardViaApi(page, title);
seedIds.add(id);
return id;
}
test.afterAll(async ({ browser }) => {
if (seedIds.size === 0) return;
const ctx = await newAdminContext(browser);
const page = await ctx.newPage();
try {
const token = await authToken(page);
for (const id of [...seedIds]) {
await deleteDashboardViaApi(ctx.request, id, token);
seedIds.delete(id);
}
} finally {
await ctx.close();
}
});
test.describe('Dashboards List Page', () => {
test('TC-01 page chrome and core controls render', async ({
test.describe('Dashboards — list search', () => {
test('TC-01 searching by name writes the query to the URL and lists the match', async ({
authedPage: page,
dashboards,
dashboardList,
}) => {
await seed(page, 'list-chrome');
const name = uniqueName('search');
await dashboards.seed(listDashboard(name));
await dashboardList.open();
await gotoDashboardsList(page);
await dashboardList.runSearch(nameContains(name));
await expect(page).toHaveTitle('SigNoz | All Dashboards');
await expect(
page.getByRole('heading', { name: 'Dashboards', level: 1 }),
).toBeVisible();
expect(new URL(page.url()).searchParams.get('query')).toBe(
nameContains(name),
);
await expect(dashboardList.rows()).toHaveText([name]);
});
});
```
Note how the example carries no `// 1. …` `// 2. …` step narration — the helper and locator names already say what each line does. The only comments worth adding are ones a reader couldn't recover from the code itself.
Note: no cleanup code (the `dashboards` driver deletes its seed), no raw locators (they live in `drivers/dashboard-list.ts`), and no `// 1. …` step narration. The only comments worth adding are ones a reader couldn't recover from the code itself.
# Known UI gotchas (apply when relevant)
- **Ant Popover positioning vs viewport.** Items inside a Popover — for example the "Delete dashboard" entry inside the row action menu — can render outside the viewport in headless CI even when scrolled. `click({ force: true })` skips actionability checks but Playwright still requires the click coordinates to land inside the viewport. Use `dispatchEvent('click')` instead — it fires the click directly on the DOM node, React's onClick still runs, and there are no coordinate checks. Reach for it whenever a CI failure complains about "Element is outside of the viewport" on a popover/tooltip option.
- **Sticky-header rows below the fold.** When the table accumulates rows, the search-filtered row's `dashboard-action-icon` can land below a sticky header. Always `await actionIcon.scrollIntoViewIfNeeded()` before clicking. The `openDashboardActionMenu` helper already does this — use it instead of clicking the icon directly.
- **Sticky-header rows below the fold.** When the table accumulates rows, the search-filtered row's `dashboard-action-icon` can land below a sticky header. Always `await actionIcon.scrollIntoViewIfNeeded()` before clicking, inside the driver action.
- **React Query mutations vs navigation.** UI delete clicks fire an async DELETE through React Query. Navigating away before the mutation completes cancels it. Pair the click with `page.waitForResponse((r) => r.request().method() === 'DELETE' && /\/dashboards\//.test(r.url()))` and `await expect(dialog).not.toBeVisible()` before the next `page.goto(...)`.
- **Monaco editor swallows Escape.** Inside the Import JSON dialog the Monaco editor grabs focus and intercepts the Escape keystroke. Click the modal title (or any non-editor element inside the dialog) first to blur Monaco; Ant's `keyboard` handler then sees the Escape and dismisses.
- **Empty zero-state hides controls.** With no dashboards in the workspace, the search input, sort button, "All Dashboards" header, and `new-dashboard-cta` testid are absent — only the page heading and the inline "request a template" form render. Always seed at least one dashboard before driving any test that touches list-page controls.
- **Monaco editor swallows Escape.** Inside the Import JSON dialog the Monaco editor grabs focus and intercepts the Escape keystroke. Click the modal title first to blur Monaco; `dashboardList.dismissWithEscape()` does this.
- **Empty zero-state hides controls.** With no dashboards in the workspace, the list controls and `new-dashboard-cta` are absent. Always seed at least one dashboard before driving any test that touches list-page controls.
# Quality bar

View File

@@ -1,6 +1,6 @@
---
name: playwright-test-healer
description: Use this agent to debug and fix failing SigNoz E2E Playwright tests. Examples — <example>Context: A spec is red. user: 'tests/e2e/tests/dashboards/list.spec.ts is failing, fix it' assistant: 'Using the healer agent to debug each failing scenario and adjust the spec.'</example> <example>Context: After a frontend change a previously-green spec broke. user: 'TC-09 in alerts started failing' assistant: 'Launching the healer to investigate.'</example>
description: Use this agent to debug and fix failing SigNoz E2E Playwright tests. Examples — <example>Context: A spec is red. user: 'tests/e2e/tests/dashboards/list/search.spec.ts is failing, fix it' assistant: 'Using the healer agent to debug each failing scenario and adjust the spec.'</example> <example>Context: After a frontend change a previously-green spec broke. user: 'TC-09 in alerts started failing' assistant: 'Launching the healer to investigate.'</example>
tools: Glob, Grep, Read, Write, Edit, Bash, mcp__playwright-test__browser_console_messages, mcp__playwright-test__browser_evaluate, mcp__playwright-test__browser_generate_locator, mcp__playwright-test__browser_network_requests, mcp__playwright-test__browser_snapshot, mcp__playwright-test__test_debug, mcp__playwright-test__test_list, mcp__playwright-test__test_run
model: sonnet
color: red
@@ -27,11 +27,11 @@ Don't try to start the stack yourself — it can take ~4 minutes on a cold build
- `browser_network_requests` for API failures (the SigNoz API requires `Authorization: Bearer <localStorage.AUTH_TOKEN>`; 401s usually mean the test bypassed the fixture).
- `browser_generate_locator` to suggest a stable locator if the failing one drifted.
4. **Root-cause.** Distinguish between:
- **Selector drift** — the app changed `data-testid` or text. Fix the locator. Prefer `data-testid` (grep `frontend/src/<feature-dir>/` for the new one).
- **Timing** — the test races a load. Replace `waitForTimeout` with `await expect(locator).toBe…()` or `page.waitForResponse(...)` on the triggering action.
- **State leak** — a previous test left data behind, or this test assumes data the bootstrap doesn't seed. Ensure the test seeds via API and cleans up in `try / finally`. The bootstrap creates a fresh stack with **zero** dashboards / alerts.
- **Selector drift** — the app changed `data-testid` or text. Fix the locator where it lives: in the driver under `tests/e2e/drivers/` when the spec uses one, so every spec sharing it heals at once. Prefer `data-testid` (grep `frontend/src/<feature-dir>/` for the new one).
- **Timing** — the test races a load. Replace `waitForTimeout` with `await expect(locator).toBe…()` or `page.waitForResponse(...)` on the triggering action — inside the driver action when there is one.
- **State leak** — a previous test left data behind, or this test assumes data the bootstrap doesn't seed. Seed through the driver (its `dispose` cleans up) or via API with `try / finally`. The bootstrap creates a fresh stack with **zero** dashboards / alerts and only the golden telemetry; parallel per-test seeds share the tenant, so an exact-count assertion on telemetry is a test bug — assert structure instead.
- **Genuine app bug** — the app is broken, not the test. Mark the test with `test.fixme(...)` and add a one-line `// known: <description>` comment. Don't silently change the assertion to make it pass.
5. **Fix.** Edit the spec. Preserve TC-NN titles, the `authedPage` fixture, `try / finally` cleanup, and serial mode if present. If you renumber, retitle, or `test.fixme(...)` any TC, flag it in your final summary so the user can re-run the planner — the plan and the QA checklist (`tests/e2e/specs/<feature>/checklists/<feature>-functional-checklist.md`) re-derive from the current `.spec.ts` and will otherwise drift.
5. **Fix.** Edit the spec or its driver. Preserve TC-NN titles, the driver / `authedPage` fixtures, cleanup, and serial mode if present. A driver change must keep every other spec importing it green — run them too. If you renumber, retitle, or `test.fixme(...)` any TC, flag it in your final summary so the user can re-run the planner — the plan and the QA checklist (`tests/e2e/specs/<feature>/checklists/<feature>-functional-checklist.md`) re-derive from the current `.spec.ts` and will otherwise drift.
6. **Re-run only the fixed test** before moving to the next failure. Three options:
- `npx playwright test -g 'TC-09' --project=chromium` — target a single TC by title
- `npx playwright test --last-failed --project=chromium` — replay everything that failed last run
@@ -41,14 +41,13 @@ Don't try to start the stack yourself — it can take ~4 minutes on a cold build
# Repo-specific signals
- **Reuse helpers before adding new code.** [`tests/e2e/helpers/dashboards.ts`](../../tests/e2e/helpers/dashboards.ts) and [`tests/e2e/helpers/auth.ts`](../../tests/e2e/helpers/auth.ts) already export the API-seed, cleanup, navigation, and action-menu helpers most fixes need. Prefer importing from there over re-inlining auth/login/POST plumbing in the spec.
- **Reuse drivers and helpers before adding new code.** [`tests/e2e/drivers/`](../../tests/e2e/drivers/) holds the per-feature seed, cleanup, navigation and action methods (`kit.ts` explains `driver` / `seeded`); [`helpers/common.ts`](../../tests/e2e/helpers/common.ts) and [`helpers/auth.ts`](../../tests/e2e/helpers/auth.ts) the plain plumbing. Prefer them over re-inlining auth/login/POST code in the spec.
- **Ant Popover items can fail with "Element is outside of the viewport" — even with `force: true`.** `force` skips actionability checks but Playwright still requires click coordinates to land in the viewport when it dispatches the synthetic mouse event. The robust fix is `tooltip.getByText('…').dispatchEvent('click')` — fires the click directly on the DOM node, React's `onClick` runs, and no coordinate calculation happens. Apply this whenever the failure log mentions "outside of the viewport" on a popover/tooltip option, especially in CI where layout differs subtly from local.
- **Action-icon rows below the fold.** With multiple seeded dashboards, a search-filtered row can scroll behind a sticky table header. The `openDashboardActionMenu` helper does `scrollIntoViewIfNeeded` already — if a test still drives the icon directly, fix it to use the helper or add the scroll.
- **Action-icon rows below the fold.** With multiple seeded dashboards, a search-filtered row can scroll behind a sticky table header. Add `scrollIntoViewIfNeeded` before the click, in the driver action.
- **React Query mutations vs page.goto.** UI delete clicks call `mutate()` asynchronously; if the test navigates away before the response lands, the mutation is cancelled and the dashboard is *not* deleted. Wait for the DELETE response and the dialog dismissal explicitly: `page.waitForResponse((r) => r.request().method() === 'DELETE' && /\/dashboards\//.test(r.url()))` plus `await expect(dialog).not.toBeVisible()`.
- **Monaco editor swallows Escape inside the Import JSON dialog.** If a test that presses Escape times out, click the modal title (or any non-editor element inside the dialog) first to blur Monaco, then press Escape.
- **The list pages render zero-state when the workspace is empty.** Many locators (search input, sort button, `new-dashboard-cta` testid, "All Dashboards" header) are absent in zero-state. A 30s timeout on those usually means the workspace was empty — seed first via `createDashboardViaApi`.
- **The "Enter dashboard name…" inline field is a `RequestDashboardBtn` (template-request feedback form), not a create flow.** Tests that try to use it to create a named dashboard will silently no-op. The only UI create paths are the "New dashboard" dropdown → "Create dashboard" (default name "Sample Title", see `DEFAULT_DASHBOARD_TITLE`) or "Import JSON".
- **Auth.** `tests/e2e/fixtures/auth.ts` logs in once per worker and caches `storageState` (cookies + localStorage with `AUTH_TOKEN`). For API-driven seeding/cleanup, use `authToken(page)` from `helpers/dashboards.ts` and pass `Authorization: Bearer <token>`. Never re-implement login.
- **The list pages render zero-state when the workspace is empty.** List controls and the `new-dashboard-cta` testid are absent in zero-state. A timeout on those usually means the workspace was empty — seed first via `dashboards.seed(...)`.
- **Auth.** `tests/e2e/fixtures/auth.ts` logs in once per worker and caches `storageState` (cookies + localStorage with `AUTH_TOKEN`). For API-driven seeding/cleanup, use `authToken(page)` from `helpers/common.ts` and pass `Authorization: Bearer <token>`. Never re-implement login.
- **Extended fixtures.** Domain-specific fixtures extend `auth` and add seeded data. Alerts uses `fixtures/alerts/alert-rules` (worker-scoped rule list, test-scoped factory) and `fixtures/alerts/alert-history` (extends alert-rules, waits on ruler evaluation). See [docs/contributing/tests/e2e.md](../../docs/contributing/tests/e2e.md) for the pattern. When a test fails on missing data, check if it imports the wrong fixture level.
- **Ant Design popovers** (sort menu, action menu) are click-toggle. The trigger element is often an inline `<svg>` with a `data-testid` — clicking it opens the popover; clicking it again closes. After selecting an option, the popover auto-closes. If a test interacts with the popover twice, wait for the menu items to be visible explicitly between toggles.
- **Artifacts.** Every failed test writes to `tests/e2e/artifacts/results/<test-slug>/` — the `error-context.md` accessibility snapshot is the fastest way to see what the page actually looked like when it failed.

View File

@@ -20,8 +20,10 @@ You will:
- The frontend stores its JWT in `localStorage` under `AUTH_TOKEN` and the API requires `Authorization: Bearer <token>` for protected endpoints. Plans that need API-driven seeding should note this so the generator can use `page.request.*`.
2. **Check what's already wired up.**
- [tests/e2e/helpers/dashboards.ts](../../tests/e2e/helpers/dashboards.ts) and [tests/e2e/helpers/auth.ts](../../tests/e2e/helpers/auth.ts) hold reusable helpers (`createDashboardViaApi`, `gotoDashboardsList`, `openDashboardActionMenu`, `newAdminContext`, `importApmMetricsDashboardViaUI`, etc.). When the plan touches dashboards, reference these by name in the steps so the generator can reuse them rather than reinvent.
- [tests/e2e/fixtures/apm-metrics.json](../../tests/e2e/fixtures/apm-metrics.json) is a real-world dashboard payload (rich tags, panels, description) suitable as a seed fixture — note in the plan if a scenario benefits from it.
- [tests/e2e/drivers/](../../tests/e2e/drivers/) holds one driver per feature (e.g. `dashboards.ts`, `dashboard-list.ts`, `panel-editor.ts`, `routing.ts`, `channels.ts`). A driver is a fixture exposing the feature's locators and actions, including API seeding that cleans itself up (`dashboards.seed(...)`). When one covers the feature, write steps in terms of its actions by name so the generator reuses them. When none does, say so: the generator adds one.
- [tests/e2e/helpers/](../../tests/e2e/helpers/) holds plain helpers: `common.ts` (`authToken`, `watchConsole`, `collectRequests`, `seederUrl`), `auth.ts` (`newAdminContext`), `query-range-mock.ts` (mock `query_range` for error, no-data, warning and loading states only).
- **Telemetry is pre-seeded.** `bootstrap/global.setup.ts` loads a golden OTel-demo dataset (logs, traces, metrics over the last 6h; constants in `GOLDEN` in `helpers/dashboard-spec.ts`). Plans that query data use a window ≤ 6h and assert structure, not exact counts. Data the golden set lacks is seeded per test through the seeder (`helpers/alerts/seeding.ts` shows how).
- [tests/e2e/testdata/](../../tests/e2e/testdata/) holds static payloads (`dashboards/panels.ts`, `traces/large-trace.json`) — note in the plan if a scenario benefits from one.
3. **Navigate and explore**
- Invoke `planner_setup_page` once before any other browser tool.
@@ -39,7 +41,7 @@ You will:
Each scenario must include:
- **TC-NN** title — `TC-NN <short description>` (matches the naming this repo uses for test titles).
- Preconditions (what state the test expects — note that the bootstrap creates a fresh stack with **zero dashboards / alerts / etc.**, so plans must seed their own data).
- Preconditions (what state the test expects — note that the bootstrap creates a fresh stack with **zero dashboards / alerts / etc.** (only the golden telemetry), so plans must seed their own resources).
- Step-by-step user actions.
- Expected outcomes per step.
- Cleanup notes (what gets created and how to remove it — usually via API).
@@ -64,7 +66,7 @@ The dashboards list page (`/dashboard`) lists all dashboards in the workspace. F
- Open per-row actions: View, Open in New Tab, Copy Link, Export JSON, Delete dashboard
- Create a new dashboard via the "New dashboard" dropdown (Create dashboard / Import JSON / View templates)
**Bootstrap state:** the pytest harness creates a fresh stack with no pre-seeded dashboards. Every test must seed its own data. The "Enter dashboard name…" inline input is a "request a new template" feedback form — **not** a create flow. The only UI create path is the dropdown.
**Bootstrap state:** the pytest harness creates a fresh stack with no pre-seeded dashboards. Every test seeds its own through the `dashboards` driver (`dashboards.seed(...)`, deleted when the test ends).
**Known limitations:**
- Ascending sort is not yet implemented — repeated clicks on the sort button keep `order=descend`.
@@ -87,7 +89,7 @@ The dashboards list page (`/dashboard`) lists all dashboards in the workspace. F
- All Dashboards section header rendered.
- Search input, sort button, and at least one dashboard thumbnail visible.
**Cleanup:** delete the seeded dashboard via `DELETE /api/v1/dashboards/<id>`.
**Cleanup:** none in the spec — the `dashboards` driver deletes what it seeded.
#### TC-02 ...
</example-spec>
@@ -95,7 +97,7 @@ The dashboards list page (`/dashboard`) lists all dashboards in the workspace. F
**Quality bar:**
- Steps must be specific enough that any tester (or the generator agent) can follow without ambiguity.
- Include negative scenarios — empty state, no-match search, validation errors.
- Each scenario must own its preconditions and cleanup. **Do not invent cross-file global fixtures** — they break parallel-by-file execution. Suite-level `beforeAll` / `afterAll` *within* a single spec file is fine and is the preferred shape for files with > ~10 scenarios; per-test `try / finally` is fine for smaller specs.
- Each scenario must own its preconditions and cleanup. **Do not invent cross-file global fixtures** — they break parallel-by-file execution. Prefer a driver's self-cleaning seed (`dispose`) or worker-scoped `seeded(...)` layer; without a driver, suite-level `beforeAll` / `afterAll` within one spec file or per-test `try / finally` both work.
- Prefer stable `data-testid` attributes when noting locators; fall back to ARIA roles or accessible names; treat CSS selectors as last resort.
- **Don't pad coverage.** Every TC must catch a regression that would actually ship if the test were missing — a real branch in the source, a real user-visible failure mode. A TC that asserts a hard-coded string is still rendered, or that a button is still a button, adds nothing but maintenance cost: it inflates the suite, slows CI, and trains readers to skim past the directory. Before writing a scenario, ask "what code change would break this?" — if the only answer is "deleting the literal under test," cut it or fold it into a richer scenario as one assertion among many.
- **Collapse near-duplicates.** Two TCs that differ only in input value (search by title vs search by description, when the underlying code path is the same) should merge into one parameterised scenario unless each input genuinely exercises a distinct branch. Prefer one assertion-rich TC over three thin ones.

View File

@@ -11389,8 +11389,7 @@ components:
evaluation:
allOf:
- $ref: '#/components/schemas/RuletypesEvaluationEnvelope'
description: The evaluation window, as a kind and spec. Required when schemaVersion
is v2alpha1; derived from evalWindow and frequency for v1.
description: The evaluation window, as a kind and spec.
frequency:
description: How often the rule is evaluated, as a Go duration such as 1m.
Only read when schemaVersion is v1, defaulting to 1m; v2alpha1 rules set
@@ -11406,8 +11405,7 @@ components:
notificationSettings:
allOf:
- $ref: '#/components/schemas/RuletypesNotificationSettings'
description: Grouping, re-notification and routing behaviour. Required when
schemaVersion is v2alpha1; derived from the v1 fields otherwise.
description: Grouping, re-notification and routing behaviour.
preferredChannels:
description: The displayName of each notification channel to notify, for
schemaVersion v1 rules. Ignored for v2alpha1, where each threshold carries
@@ -11420,26 +11418,30 @@ components:
- $ref: '#/components/schemas/RuletypesRuleType'
description: 'The evaluation strategy: threshold_rule compares against thresholds,
promql_rule evaluates a PromQL expression, anomaly_rule scores deviation
from seasonal history. Defaults to threshold_rule for builder queries
and is forced to promql_rule for promql queries.'
from seasonal history. Defaults to threshold_rule for builder and clickhouse_sql
queries and is forced to promql_rule for promql queries.'
schemaVersion:
description: 'The rule payload schema: v1 (the default when omitted) uses
the flat condition fields, v2alpha1 uses condition.thresholds, evaluation
and notificationSettings.'
description: The rule payload schema.
enum:
- v2alpha1
type: string
source:
deprecated: true
description: The URL of the page the rule was created from. Informational;
it is not used for evaluation.
type: string
version:
description: The query payload version. Only v5 is accepted and it is filled
in when omitted.
deprecated: true
description: The query payload version. Nothing reads it.
type: string
required:
- alert
- alertType
- ruleType
- condition
- evaluation
- schemaVersion
- notificationSettings
type: object
RuletypesPostableRuleView:
properties:
@@ -11528,8 +11530,7 @@ components:
evaluation:
allOf:
- $ref: '#/components/schemas/RuletypesEvaluationEnvelope'
description: The evaluation window, as a kind and spec. Required when schemaVersion
is v2alpha1; derived from evalWindow and frequency for v1.
description: The evaluation window, as a kind and spec.
frequency:
description: How often the rule is evaluated, as a Go duration such as 1m.
Only read when schemaVersion is v1, defaulting to 1m; v2alpha1 rules set
@@ -11548,8 +11549,7 @@ components:
notificationSettings:
allOf:
- $ref: '#/components/schemas/RuletypesNotificationSettings'
description: Grouping, re-notification and routing behaviour. Required when
schemaVersion is v2alpha1; derived from the v1 fields otherwise.
description: Grouping, re-notification and routing behaviour.
preferredChannels:
description: The displayName of each notification channel to notify, for
schemaVersion v1 rules. Ignored for v2alpha1, where each threshold carries
@@ -11562,14 +11562,15 @@ components:
- $ref: '#/components/schemas/RuletypesRuleType'
description: 'The evaluation strategy: threshold_rule compares against thresholds,
promql_rule evaluates a PromQL expression, anomaly_rule scores deviation
from seasonal history. Defaults to threshold_rule for builder queries
and is forced to promql_rule for promql queries.'
from seasonal history. Defaults to threshold_rule for builder and clickhouse_sql
queries and is forced to promql_rule for promql queries.'
schemaVersion:
description: 'The rule payload schema: v1 (the default when omitted) uses
the flat condition fields, v2alpha1 uses condition.thresholds, evaluation
and notificationSettings.'
description: The rule payload schema.
enum:
- v2alpha1
type: string
source:
deprecated: true
description: The URL of the page the rule was created from. Informational;
it is not used for evaluation.
type: string
@@ -11584,8 +11585,8 @@ components:
updatedBy:
type: string
version:
description: The query payload version. Only v5 is accepted and it is filled
in when omitted.
deprecated: true
description: The query payload version. Nothing reads it.
type: string
required:
- id
@@ -11594,6 +11595,9 @@ components:
- alertType
- ruleType
- condition
- evaluation
- schemaVersion
- notificationSettings
type: object
RuletypesRuleCondition:
properties:
@@ -11643,8 +11647,7 @@ components:
against, hourly, daily or weekly.'
selectedQueryName:
description: The name of the query whose result is evaluated, such as A
or F1. Must match a query in compositeQuery. When omitted, F1 is used
if present, otherwise the enabled query with the highest name.
or F1. Must match a query in compositeQuery.
type: string
target:
description: 'schemaVersion v1 only: the value compared against, in targetUnit.
@@ -11660,10 +11663,11 @@ components:
thresholds:
allOf:
- $ref: '#/components/schemas/RuletypesRuleThresholdData'
description: The thresholds to evaluate, as a kind and spec. Required when
schemaVersion is v2alpha1; derived from op, target and matchType for v1.
description: The thresholds to evaluate, as a kind and spec.
required:
- compositeQuery
- selectedQueryName
- thresholds
type: object
RuletypesRuleThresholdData:
discriminator:
@@ -25312,13 +25316,13 @@ paths:
schemaVersion: v2alpha1
version: v5
metric_anomaly:
description: Anomaly rules are not yet supported under schemaVersion
v2alpha1, so this example uses the v1 shape. Wraps a builder query
in the `anomaly` function with daily seasonality SigNoz compares
each point against the forecast for that time of day. Fires when
the anomaly score stays below the threshold for the entire window;
`requireMinPoints` guards against noisy intervals.
summary: Metric anomaly rule (v1 only)
description: Wraps a builder query in the `anomaly` function with
daily seasonality, so SigNoz compares each point against the forecast
for that time of day. The threshold is compared against the signed
anomaly score, so `below` with a target of -2 fires when an abrupt
drop holds for the entire window; `requireMinPoints` guards against
noisy intervals.
summary: Metric anomaly rule
value:
alert: Anomalous drop in ingested spans
alertType: METRIC_BASED_ALERT
@@ -25353,22 +25357,36 @@ paths:
stepInterval: 21600
type: builder_query
queryType: builder
matchType: all_the_times
op: below
requireMinPoints: true
requiredNumPoints: 3
seasonality: daily
selectedQueryName: A
target: 2
thresholds:
kind: basic
spec:
- channels:
- slack-ingestion
matchType: all_the_times
name: warning
op: below
target: -2
description: Detect an abrupt drop in span ingestion using a z-score
anomaly function
evalWindow: 24h
frequency: 3h
evaluation:
kind: rolling
spec:
evalWindow: 24h
frequency: 3h
labels:
severity: warning
preferredChannels:
- slack-ingestion
notificationSettings:
renotify:
alertStates:
- firing
enabled: true
interval: 4h
ruleType: anomaly_rule
schemaVersion: v2alpha1
version: v5
metric_promql:
description: PromQL expression instead of the builder. Dotted OTEL
@@ -26264,13 +26282,13 @@ paths:
schemaVersion: v2alpha1
version: v5
metric_anomaly:
description: Anomaly rules are not yet supported under schemaVersion
v2alpha1, so this example uses the v1 shape. Wraps a builder query
in the `anomaly` function with daily seasonality SigNoz compares
each point against the forecast for that time of day. Fires when
the anomaly score stays below the threshold for the entire window;
`requireMinPoints` guards against noisy intervals.
summary: Metric anomaly rule (v1 only)
description: Wraps a builder query in the `anomaly` function with
daily seasonality, so SigNoz compares each point against the forecast
for that time of day. The threshold is compared against the signed
anomaly score, so `below` with a target of -2 fires when an abrupt
drop holds for the entire window; `requireMinPoints` guards against
noisy intervals.
summary: Metric anomaly rule
value:
alert: Anomalous drop in ingested spans
alertType: METRIC_BASED_ALERT
@@ -26305,22 +26323,36 @@ paths:
stepInterval: 21600
type: builder_query
queryType: builder
matchType: all_the_times
op: below
requireMinPoints: true
requiredNumPoints: 3
seasonality: daily
selectedQueryName: A
target: 2
thresholds:
kind: basic
spec:
- channels:
- slack-ingestion
matchType: all_the_times
name: warning
op: below
target: -2
description: Detect an abrupt drop in span ingestion using a z-score
anomaly function
evalWindow: 24h
frequency: 3h
evaluation:
kind: rolling
spec:
evalWindow: 24h
frequency: 3h
labels:
severity: warning
preferredChannels:
- slack-ingestion
notificationSettings:
renotify:
alertStates:
- firing
enabled: true
interval: 4h
ruleType: anomaly_rule
schemaVersion: v2alpha1
version: v5
metric_promql:
description: PromQL expression instead of the builder. Dotted OTEL
@@ -27117,13 +27149,13 @@ paths:
schemaVersion: v2alpha1
version: v5
metric_anomaly:
description: Anomaly rules are not yet supported under schemaVersion
v2alpha1, so this example uses the v1 shape. Wraps a builder query
in the `anomaly` function with daily seasonality SigNoz compares
each point against the forecast for that time of day. Fires when
the anomaly score stays below the threshold for the entire window;
`requireMinPoints` guards against noisy intervals.
summary: Metric anomaly rule (v1 only)
description: Wraps a builder query in the `anomaly` function with
daily seasonality, so SigNoz compares each point against the forecast
for that time of day. The threshold is compared against the signed
anomaly score, so `below` with a target of -2 fires when an abrupt
drop holds for the entire window; `requireMinPoints` guards against
noisy intervals.
summary: Metric anomaly rule
value:
alert: Anomalous drop in ingested spans
alertType: METRIC_BASED_ALERT
@@ -27158,22 +27190,36 @@ paths:
stepInterval: 21600
type: builder_query
queryType: builder
matchType: all_the_times
op: below
requireMinPoints: true
requiredNumPoints: 3
seasonality: daily
selectedQueryName: A
target: 2
thresholds:
kind: basic
spec:
- channels:
- slack-ingestion
matchType: all_the_times
name: warning
op: below
target: -2
description: Detect an abrupt drop in span ingestion using a z-score
anomaly function
evalWindow: 24h
frequency: 3h
evaluation:
kind: rolling
spec:
evalWindow: 24h
frequency: 3h
labels:
severity: warning
preferredChannels:
- slack-ingestion
notificationSettings:
renotify:
alertStates:
- firing
enabled: true
interval: 4h
ruleType: anomaly_rule
schemaVersion: v2alpha1
version: v5
metric_promql:
description: PromQL expression instead of the builder. Dotted OTEL
@@ -28676,13 +28722,13 @@ paths:
schemaVersion: v2alpha1
version: v5
metric_anomaly:
description: Anomaly rules are not yet supported under schemaVersion
v2alpha1, so this example uses the v1 shape. Wraps a builder query
in the `anomaly` function with daily seasonality SigNoz compares
each point against the forecast for that time of day. Fires when
the anomaly score stays below the threshold for the entire window;
`requireMinPoints` guards against noisy intervals.
summary: Metric anomaly rule (v1 only)
description: Wraps a builder query in the `anomaly` function with
daily seasonality, so SigNoz compares each point against the forecast
for that time of day. The threshold is compared against the signed
anomaly score, so `below` with a target of -2 fires when an abrupt
drop holds for the entire window; `requireMinPoints` guards against
noisy intervals.
summary: Metric anomaly rule
value:
alert: Anomalous drop in ingested spans
alertType: METRIC_BASED_ALERT
@@ -28717,22 +28763,36 @@ paths:
stepInterval: 21600
type: builder_query
queryType: builder
matchType: all_the_times
op: below
requireMinPoints: true
requiredNumPoints: 3
seasonality: daily
selectedQueryName: A
target: 2
thresholds:
kind: basic
spec:
- channels:
- slack-ingestion
matchType: all_the_times
name: warning
op: below
target: -2
description: Detect an abrupt drop in span ingestion using a z-score
anomaly function
evalWindow: 24h
frequency: 3h
evaluation:
kind: rolling
spec:
evalWindow: 24h
frequency: 3h
labels:
severity: warning
preferredChannels:
- slack-ingestion
notificationSettings:
renotify:
alertStates:
- firing
enabled: true
interval: 4h
ruleType: anomaly_rule
schemaVersion: v2alpha1
version: v5
metric_promql:
description: PromQL expression instead of the builder. Dotted OTEL

View File

@@ -173,9 +173,10 @@ Each spec follows these principles:
1. **Directory per feature**: `tests/e2e/tests/<feature>/*.spec.ts`. Cross-resource junction concerns (e.g. cascade-delete) go in their own file, not packed into one giant spec.
2. **Test titles use `TC-NN`**: `test('TC-01 alerts page — tabs render', ...)`. Preserves ordering at a glance and maps to external coverage tracking.
3. **UI-first**: drive flows through the UI. Playwright traces capture every BE request/response the UI triggers, so asserting on UI outcomes implicitly validates BE contracts. Reach for direct `page.request.*` only when the test's *purpose* is asserting a response contract (use `page.waitForResponse` on a UI click) or when a specific UI step is structurally flaky (e.g. Ant DatePicker calendar-cell indices) — and even then try UI first.
4. **Self-contained state**: each spec seeds its own data and cleans up at suite teardown. The pytest harness creates a fresh stack with **zero** dashboards / alerts / etc. — never assume pre-existing data. Two patterns work:
- **Per-test seed + cleanup in `try / finally`** — small specs where each test owns its data.
- **Suite-level seed + `afterAll` teardown** — preferred for larger specs. Each `createDashboard(...)` call adds the resulting ID to a module-level `Set<string>`, and one `test.afterAll(...)` deletes everything in the set. See `tests/e2e/tests/dashboards/list.spec.ts` for the full pattern. `test.beforeAll` / `test.afterAll` cannot use `authedPage` directly (it's test-scoped); use `newAdminContext(browser)` from `helpers/auth.ts` instead — it performs one fresh login per suite hook.
4. **Self-contained state**: each spec seeds its own data and cleans up at suite teardown. The pytest harness creates a fresh stack with **zero** dashboards / alerts / etc. — never assume pre-existing data. Patterns, in order of preference:
- **Driver `dispose`** — the driver tracks what it seeded and deletes it after the test, so the spec has no cleanup code. See `dashboards.seed(...)` in `tests/e2e/drivers/dashboards.ts`.
- **Per-test seed + cleanup in `try / finally`** — small specs without a driver, where each test owns its data.
- **Suite-level seed + `afterAll` teardown** — larger specs without a driver. One `test.afterAll(...)` deletes everything the suite seeded. See `tests/e2e/tests/llm-o11y/llm-pricing.spec.ts`. `test.beforeAll` / `test.afterAll` cannot use `authedPage` directly (it's test-scoped); use `newAdminContext(browser)` from `helpers/auth.ts` instead — it performs one fresh login per suite hook.
5. **Seed via API when the UI flow is multi-step or brittle.** The frontend stores its JWT in `localStorage` under `AUTH_TOKEN`; `page.request.*` inherits the auth fixture's storage state. A typical pattern:
```ts
const token = await page.evaluate(

View File

@@ -12584,6 +12584,12 @@ export type RuletypesPostableRuleDTOAnnotations = { [key: string]: string };
*/
export type RuletypesPostableRuleDTOLabels = { [key: string]: string };
/**
* The rule payload schema.
*/
export enum RuletypesPostableRuleDTOSchemaVersion {
v2alpha1 = 'v2alpha1',
}
export enum RuletypesSeasonalityDTO {
hourly = 'hourly',
daily = 'daily',
@@ -12660,9 +12666,9 @@ export interface RuletypesRuleConditionDTO {
seasonality?: RuletypesSeasonalityDTO;
/**
* @type string
* @description The name of the query whose result is evaluated, such as A or F1. Must match a query in compositeQuery. When omitted, F1 is used if present, otherwise the enabled query with the highest name.
* @description The name of the query whose result is evaluated, such as A or F1. Must match a query in compositeQuery.
*/
selectedQueryName?: string;
selectedQueryName: string;
/**
* @type number,null
* @format double
@@ -12675,9 +12681,9 @@ export interface RuletypesRuleConditionDTO {
*/
targetUnit?: string;
/**
* @description The thresholds to evaluate, as a kind and spec. Required when schemaVersion is v2alpha1; derived from op, target and matchType for v1.
* @description The thresholds to evaluate, as a kind and spec.
*/
thresholds?: RuletypesRuleThresholdDataDTO;
thresholds: RuletypesRuleThresholdDataDTO;
}
export interface RuletypesPostableRuleDTO {
@@ -12715,9 +12721,9 @@ export interface RuletypesPostableRuleDTO {
*/
evalWindow?: string;
/**
* @description The evaluation window, as a kind and spec. Required when schemaVersion is v2alpha1; derived from evalWindow and frequency for v1.
* @description The evaluation window, as a kind and spec.
*/
evaluation?: RuletypesEvaluationEnvelopeDTO;
evaluation: RuletypesEvaluationEnvelopeDTO;
/**
* @type string
* @description How often the rule is evaluated, as a Go duration such as 1m. Only read when schemaVersion is v1, defaulting to 1m; v2alpha1 rules set it in evaluation.spec instead.
@@ -12729,23 +12735,24 @@ export interface RuletypesPostableRuleDTO {
*/
labels?: RuletypesPostableRuleDTOLabels;
/**
* @description Grouping, re-notification and routing behaviour. Required when schemaVersion is v2alpha1; derived from the v1 fields otherwise.
* @description Grouping, re-notification and routing behaviour.
*/
notificationSettings?: RuletypesNotificationSettingsDTO;
notificationSettings: RuletypesNotificationSettingsDTO;
/**
* @type array
* @description The displayName of each notification channel to notify, for schemaVersion v1 rules. Ignored for v2alpha1, where each threshold carries its own channels.
*/
preferredChannels?: string[];
/**
* @description The evaluation strategy: threshold_rule compares against thresholds, promql_rule evaluates a PromQL expression, anomaly_rule scores deviation from seasonal history. Defaults to threshold_rule for builder queries and is forced to promql_rule for promql queries.
* @description The evaluation strategy: threshold_rule compares against thresholds, promql_rule evaluates a PromQL expression, anomaly_rule scores deviation from seasonal history. Defaults to threshold_rule for builder and clickhouse_sql queries and is forced to promql_rule for promql queries.
*/
ruleType: RuletypesRuleTypeDTO;
/**
* @enum v2alpha1
* @type string
* @description The rule payload schema: v1 (the default when omitted) uses the flat condition fields, v2alpha1 uses condition.thresholds, evaluation and notificationSettings.
* @description The rule payload schema.
*/
schemaVersion?: string;
schemaVersion: RuletypesPostableRuleDTOSchemaVersion;
/**
* @type string
* @description The URL of the page the rule was created from. Informational; it is not used for evaluation.
@@ -12753,7 +12760,7 @@ export interface RuletypesPostableRuleDTO {
source?: string;
/**
* @type string
* @description The query payload version. Only v5 is accepted and it is filled in when omitted.
* @description The query payload version. Nothing reads it.
*/
version?: string;
}
@@ -12776,6 +12783,12 @@ export type RuletypesRuleDTOAnnotations = { [key: string]: string };
*/
export type RuletypesRuleDTOLabels = { [key: string]: string };
/**
* The rule payload schema.
*/
export enum RuletypesRuleDTOSchemaVersion {
v2alpha1 = 'v2alpha1',
}
export interface RuletypesRuleDTO {
/**
* @type string
@@ -12820,9 +12833,9 @@ export interface RuletypesRuleDTO {
*/
evalWindow?: string;
/**
* @description The evaluation window, as a kind and spec. Required when schemaVersion is v2alpha1; derived from evalWindow and frequency for v1.
* @description The evaluation window, as a kind and spec.
*/
evaluation?: RuletypesEvaluationEnvelopeDTO;
evaluation: RuletypesEvaluationEnvelopeDTO;
/**
* @type string
* @description How often the rule is evaluated, as a Go duration such as 1m. Only read when schemaVersion is v1, defaulting to 1m; v2alpha1 rules set it in evaluation.spec instead.
@@ -12839,23 +12852,24 @@ export interface RuletypesRuleDTO {
*/
labels?: RuletypesRuleDTOLabels;
/**
* @description Grouping, re-notification and routing behaviour. Required when schemaVersion is v2alpha1; derived from the v1 fields otherwise.
* @description Grouping, re-notification and routing behaviour.
*/
notificationSettings?: RuletypesNotificationSettingsDTO;
notificationSettings: RuletypesNotificationSettingsDTO;
/**
* @type array
* @description The displayName of each notification channel to notify, for schemaVersion v1 rules. Ignored for v2alpha1, where each threshold carries its own channels.
*/
preferredChannels?: string[];
/**
* @description The evaluation strategy: threshold_rule compares against thresholds, promql_rule evaluates a PromQL expression, anomaly_rule scores deviation from seasonal history. Defaults to threshold_rule for builder queries and is forced to promql_rule for promql queries.
* @description The evaluation strategy: threshold_rule compares against thresholds, promql_rule evaluates a PromQL expression, anomaly_rule scores deviation from seasonal history. Defaults to threshold_rule for builder and clickhouse_sql queries and is forced to promql_rule for promql queries.
*/
ruleType: RuletypesRuleTypeDTO;
/**
* @enum v2alpha1
* @type string
* @description The rule payload schema: v1 (the default when omitted) uses the flat condition fields, v2alpha1 uses condition.thresholds, evaluation and notificationSettings.
* @description The rule payload schema.
*/
schemaVersion?: string;
schemaVersion: RuletypesRuleDTOSchemaVersion;
/**
* @type string
* @description The URL of the page the rule was created from. Informational; it is not used for evaluation.
@@ -12876,7 +12890,7 @@ export interface RuletypesRuleDTO {
updatedBy?: string;
/**
* @type string
* @description The query payload version. Only v5 is accepted and it is filled in when omitted.
* @description The query payload version. Nothing reads it.
*/
version?: string;
}

View File

@@ -1,8 +1,10 @@
import {
RuletypesAlertStateDTO,
RuletypesAlertTypeDTO,
RuletypesEvaluationRollingDTOKind,
RuletypesPanelTypeDTO,
RuletypesQueryTypeDTO,
RuletypesRuleDTOSchemaVersion,
RuletypesRuleTypeDTO,
} from 'api/generated/services/sigNoz.schemas';
import type {
@@ -38,6 +40,12 @@ const make = (
evalWindow: '5m0s',
frequency: '1m0s',
ruleType: RuletypesRuleTypeDTO.threshold_rule,
schemaVersion: RuletypesRuleDTOSchemaVersion.v2alpha1,
evaluation: {
kind: RuletypesEvaluationRollingDTOKind.rolling,
spec: { evalWindow: '5m0s', frequency: '1m0s' },
},
notificationSettings: { usePolicy: false, renotify: { enabled: false } },
...overrides,
});

View File

@@ -18,6 +18,7 @@ import {
RuletypesMatchTypeDTO,
RuletypesPanelTypeDTO,
RuletypesQueryTypeDTO,
RuletypesRuleDTOSchemaVersion,
RuletypesRuleTypeDTO,
RuletypesSeasonalityDTO,
RuletypesThresholdBasicDTOKind,
@@ -34,7 +35,6 @@ import {
type AlertmanagertypesGettableNotificationChannelDTO,
type AlertmanagertypesListedNotificationChannelDTO,
} from 'api/generated/services/sigNoz.schemas';
import { NEW_ALERT_SCHEMA_VERSION } from 'types/api/alerts/alertTypesV2';
const MINUTE = 60 * 1000;
const HOUR = 60 * MINUTE;
@@ -388,18 +388,18 @@ const buildRule = (index: number, shape: RuleShape): RuletypesRuleDTO => {
},
evalWindow: isAnomaly ? '1h0m0s' : '5m0s',
frequency: '1m0s',
...(isAnomaly && {
evaluation: {
kind: RuletypesEvaluationRollingDTOKind.rolling,
spec: { evalWindow: '1h0m0s', frequency: '1m0s' },
},
}),
evaluation: {
kind: RuletypesEvaluationRollingDTOKind.rolling,
spec: { evalWindow: isAnomaly ? '1h0m0s' : '5m0s', frequency: '1m0s' },
},
createdAt: ago((index + 3) * DAY),
updatedAt: ago((index + 1) * HOUR),
createdBy: 'ada@signoz.io',
updatedBy: 'grace@signoz.io',
schemaVersion:
shape.schema === 'classic' ? undefined : NEW_ALERT_SCHEMA_VERSION,
// classic rows predate schemaVersion, which the spec now requires
schemaVersion: (shape.schema === 'classic'
? undefined
: RuletypesRuleDTOSchemaVersion.v2alpha1) as RuletypesRuleDTOSchemaVersion,
version: 'v5',
source: 'http://localhost/alerts',
preferredChannels: ['ops-slack'],

View File

@@ -221,16 +221,15 @@ func postableRuleExamples() []handler.OpenAPIExample {
},
{
Name: "metric_anomaly",
Summary: "Metric anomaly rule (v1 only)",
Description: "Anomaly rules are not yet supported under schemaVersion v2alpha1, so this example uses the v1 shape. Wraps a builder query in the `anomaly` function with daily seasonality SigNoz compares each point against the forecast for that time of day. Fires when the anomaly score stays below the threshold for the entire window; `requireMinPoints` guards against noisy intervals.",
Summary: "Metric anomaly rule",
Description: "Wraps a builder query in the `anomaly` function with daily seasonality, so SigNoz compares each point against the forecast for that time of day. The threshold is compared against the signed anomaly score, so `below` with a target of -2 fires when an abrupt drop holds for the entire window; `requireMinPoints` guards against noisy intervals.",
Value: map[string]any{
"alert": "Anomalous drop in ingested spans",
"alertType": "METRIC_BASED_ALERT",
"description": "Detect an abrupt drop in span ingestion using a z-score anomaly function",
"ruleType": "anomaly_rule",
"version": "v5",
"evalWindow": "24h",
"frequency": "3h",
"alert": "Anomalous drop in ingested spans",
"alertType": "METRIC_BASED_ALERT",
"description": "Detect an abrupt drop in span ingestion using a z-score anomaly function",
"ruleType": "anomaly_rule",
"version": "v5",
"schemaVersion": "v2alpha1",
"condition": map[string]any{
"compositeQuery": map[string]any{
"queryType": "builder",
@@ -256,17 +255,29 @@ func postableRuleExamples() []handler.OpenAPIExample {
},
},
},
"op": "below",
"matchType": "all_the_times",
"target": 2,
"algorithm": "standard",
"seasonality": "daily",
"selectedQueryName": "A",
"requireMinPoints": true,
"requiredNumPoints": 3,
"thresholds": map[string]any{
"kind": "basic",
"spec": []any{
map[string]any{
"name": "warning",
"op": "below",
"matchType": "all_the_times",
"target": -2,
"channels": []any{"slack-ingestion"},
},
},
},
},
"labels": map[string]any{"severity": "warning"},
"preferredChannels": []any{"slack-ingestion"},
"evaluation": rolling("24h", "3h"),
"notificationSettings": map[string]any{
"renotify": renotify("4h", "firing"),
},
"labels": map[string]any{"severity": "warning"},
"annotations": map[string]any{
"description": "Ingestion rate for tenant {{$tenant_id}} is anomalously low (z-score {{$value}}).",
"summary": "Span ingestion anomaly",

View File

@@ -117,10 +117,10 @@ type RuleCondition struct {
TargetUnit string `json:"targetUnit,omitempty" description:"schemaVersion v1 only: the unit of target, such as percent or ms. Converted to compositeQuery.unit before comparison when the two differ."`
Algorithm string `json:"algorithm,omitempty" description:"anomaly_rule only: the anomaly detection algorithm. The supported value is standard."`
Seasonality Seasonality `json:"seasonality,omitzero" description:"anomaly_rule only: the seasonal period the history is compared against, hourly, daily or weekly."`
SelectedQuery string `json:"selectedQueryName,omitempty" description:"The name of the query whose result is evaluated, such as A or F1. Must match a query in compositeQuery. When omitted, F1 is used if present, otherwise the enabled query with the highest name."`
SelectedQuery string `json:"selectedQueryName,omitempty" required:"true" description:"The name of the query whose result is evaluated, such as A or F1. Must match a query in compositeQuery."`
RequireMinPoints bool `json:"requireMinPoints,omitempty" description:"Whether a series is skipped until it has at least requiredNumPoints points in the window. Defaults to false."`
RequiredNumPoints int `json:"requiredNumPoints,omitempty" description:"The minimum number of points a series needs before it is evaluated. Must be greater than 0 when requireMinPoints is true."`
Thresholds *RuleThresholdData `json:"thresholds,omitempty" description:"The thresholds to evaluate, as a kind and spec. Required when schemaVersion is v2alpha1; derived from op, target and matchType for v1."`
Thresholds *RuleThresholdData `json:"thresholds,omitempty" required:"true" description:"The thresholds to evaluate, as a kind and spec."`
}
func (rc *RuleCondition) SelectedQueryName() string {

View File

@@ -9,6 +9,7 @@ import (
"unicode/utf8"
"github.com/prometheus/alertmanager/config"
"github.com/swaggest/jsonschema-go"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
@@ -63,7 +64,7 @@ type PostableRule struct {
AlertName valuer.DisplayName `json:"alert" required:"true" description:"The display name of the alert rule."`
AlertType AlertType `json:"alertType" required:"true" description:"The signal the rule evaluates: METRIC_BASED_ALERT, TRACES_BASED_ALERT, LOGS_BASED_ALERT, EXCEPTIONS_BASED_ALERT or AI_TRACES_BASED_ALERT."`
Description string `json:"description,omitempty" description:"A human readable description of what the rule monitors."`
RuleType RuleType `json:"ruleType" required:"true" description:"The evaluation strategy: threshold_rule compares against thresholds, promql_rule evaluates a PromQL expression, anomaly_rule scores deviation from seasonal history. Defaults to threshold_rule for builder queries and is forced to promql_rule for promql queries."`
RuleType RuleType `json:"ruleType" required:"true" description:"The evaluation strategy: threshold_rule compares against thresholds, promql_rule evaluates a PromQL expression, anomaly_rule scores deviation from seasonal history. Defaults to threshold_rule for builder and clickhouse_sql queries and is forced to promql_rule for promql queries."`
EvalWindow valuer.TextDuration `json:"evalWindow,omitzero" description:"The lookback window as a Go duration such as 5m. Only read when schemaVersion is v1, defaulting to 5m; v2alpha1 rules set it in evaluation.spec instead."`
Frequency valuer.TextDuration `json:"frequency,omitzero" description:"How often the rule is evaluated, as a Go duration such as 1m. Only read when schemaVersion is v1, defaulting to 1m; v2alpha1 rules set it in evaluation.spec instead."`
@@ -77,12 +78,12 @@ type PostableRule struct {
PreferredChannels []string `json:"preferredChannels,omitempty" description:"The displayName of each notification channel to notify, for schemaVersion v1 rules. Ignored for v2alpha1, where each threshold carries its own channels."`
Version string `json:"version" description:"The query payload version. Only v5 is accepted and it is filled in when omitted."`
Version string `json:"version" description:"The query payload version. Nothing reads it."`
Evaluation *EvaluationEnvelope `json:"evaluation,omitempty" description:"The evaluation window, as a kind and spec. Required when schemaVersion is v2alpha1; derived from evalWindow and frequency for v1."`
SchemaVersion string `json:"schemaVersion,omitempty" description:"The rule payload schema: v1 (the default when omitted) uses the flat condition fields, v2alpha1 uses condition.thresholds, evaluation and notificationSettings."`
Evaluation *EvaluationEnvelope `json:"evaluation,omitempty" required:"true" description:"The evaluation window, as a kind and spec."`
SchemaVersion string `json:"schemaVersion,omitempty" required:"true" enum:"v2alpha1" description:"The rule payload schema."`
NotificationSettings *NotificationSettings `json:"notificationSettings,omitempty" description:"Grouping, re-notification and routing behaviour. Required when schemaVersion is v2alpha1; derived from the v1 fields otherwise."`
NotificationSettings *NotificationSettings `json:"notificationSettings,omitempty" required:"true" description:"Grouping, re-notification and routing behaviour."`
}
type NotificationSettings struct {
@@ -335,6 +336,19 @@ func (r *PostableRule) UnmarshalJSON(bytes []byte) error {
return r.validate()
}
var _ jsonschema.Preparer = PostableRule{}
// PrepareJSONSchema marks version and source deprecated: nothing reads them,
// they stay only so existing clients keep a stable schema.
func (PostableRule) PrepareJSONSchema(schema *jsonschema.Schema) error {
for _, name := range []string{"version", "source"} {
if property, ok := schema.Properties[name]; ok && property.TypeObject != nil {
property.TypeObject.WithDeprecated(true)
}
}
return nil
}
func isValidLabelName(ln string) bool {
if len(ln) == 0 {
return false