Compare commits

..

1 Commits

Author SHA1 Message Date
aks07
2aed4821c8 chore: update e2e code owners 2026-06-30 12:05:02 +05:30
4303 changed files with 247466 additions and 499252 deletions

View File

@@ -20,16 +20,6 @@ You are the Playwright Test Generator for the SigNoz frontend. You take a plan w
await expect(page.getByRole('tab', { name: /alert rules/i })).toBeVisible();
});
```
- **Extended fixtures:** For features needing complex setup (seeded data, API calls, cleanup), import from domain-specific fixtures that extend `auth`. See [docs/contributing/tests/e2e.md](../../docs/contributing/tests/e2e.md) for the full pattern.
- `fixtures/alerts/alert-rules` — worker-scoped rule list + test-scoped rule factory
- `fixtures/alerts/alert-history` — extends alert-rules, adds history fixtures (waits on ruler evaluation)
```ts
// Alert list tests - need rules, no history
import { test, expect } from '../../../fixtures/alerts/alert-rules';
// Alert history tests - need evaluated history rows
import { test, expect } from '../../../fixtures/alerts/alert-history';
```
- **Test titles:** `TC-NN <short description>` — matches the planner's IDs.
- **Self-contained state.** The bootstrap creates a fresh stack with **zero** dashboards / alerts / etc. — never assume pre-existing data. Two cleanup shapes are valid; pick based on the spec size:
- **Per-test `try / finally`** — small specs (~ <10 scenarios) where each test owns its data.

View File

@@ -49,7 +49,6 @@ Don't try to start the stack yourself — it can take ~4 minutes on a cold build
- **The list pages render zero-state when the workspace is empty.** Many locators (search input, sort button, `new-dashboard-cta` testid, "All Dashboards" header) are absent in zero-state. A 30s timeout on those usually means the workspace was empty — seed first via `createDashboardViaApi`.
- **The "Enter dashboard name…" inline field is a `RequestDashboardBtn` (template-request feedback form), not a create flow.** Tests that try to use it to create a named dashboard will silently no-op. The only UI create paths are the "New dashboard" dropdown → "Create dashboard" (default name "Sample Title", see `DEFAULT_DASHBOARD_TITLE`) or "Import JSON".
- **Auth.** `tests/e2e/fixtures/auth.ts` logs in once per worker and caches `storageState` (cookies + localStorage with `AUTH_TOKEN`). For API-driven seeding/cleanup, use `authToken(page)` from `helpers/dashboards.ts` and pass `Authorization: Bearer <token>`. Never re-implement login.
- **Extended fixtures.** Domain-specific fixtures extend `auth` and add seeded data. Alerts uses `fixtures/alerts/alert-rules` (worker-scoped rule list, test-scoped factory) and `fixtures/alerts/alert-history` (extends alert-rules, waits on ruler evaluation). See [docs/contributing/tests/e2e.md](../../docs/contributing/tests/e2e.md) for the pattern. When a test fails on missing data, check if it imports the wrong fixture level.
- **Ant Design popovers** (sort menu, action menu) are click-toggle. The trigger element is often an inline `<svg>` with a `data-testid` — clicking it opens the popover; clicking it again closes. After selecting an option, the popover auto-closes. If a test interacts with the popover twice, wait for the menu items to be visible explicitly between toggles.
- **Artifacts.** Every failed test writes to `tests/e2e/artifacts/results/<test-slug>/` — the `error-context.md` accessibility snapshot is the fastest way to see what the page actually looked like when it failed.
- **Type-check.** After edits, run `npx tsc --noEmit -p tests/e2e/tsconfig.json` if it succeeds, or rely on `npx playwright test --list` to validate the spec parses.

View File

@@ -1,11 +0,0 @@
# Comments
Applies to everything in the repo — code, config, workflows.
- **No unnecessary comments.** Do not comment where the code is self-explanatory; never restate what the code already says.
- **Document only** non-obvious behavior, constraints, formats, and edge cases.
- **Rationale goes in prose, not source.** Why a version is pinned, why a job exists, how a subsystem fits together — that belongs in the README or the PR.
- **Never remove pre-existing comments** when editing code. The bar above applies to comments you write, not comments already there.
- **Never talk to the reviewer.** No comments about where a change came from, what was changed, or why the change is correct — that belongs in the PR description and is noise the moment it merges.
Language rules build on this one: [`go-comments`](go-comments.md), [`py-comments`](py-comments.md).

View File

@@ -1,12 +0,0 @@
---
paths:
- "**/*.go"
---
# Go comments
The bar is the [`comments`](comments.md) rule: nothing where the code is self-explanatory.
- **Names carry the meaning.** Make function, type, and variable names self-explanatory so the comment is unnecessary in the first place. If a comment is needed to explain what a function does, fix the name, not the comment.
- **Godoc**: Skip comments that merely restate the identifier. Document only non-obvious behavior, constraints, formats, and edge cases.
- **Generated code**: If the comment is emitted by an external codegen tool, leave it as-is — do not add or trim comments in generated files.

View File

@@ -1,7 +0,0 @@
# Pull requests
- **Follow the template** (`.github/pull_request_template.md`): fill in its headings (Description / Issues closed by this PR / Screenshots / Additional Information). Don't add sections the template doesn't have.
- **Keep only the headings that apply.** Delete every heading that has nothing under it, along with its `<!--...-->` placeholder comment. The body must never contain an empty heading — if only Description applies, the body has exactly that one heading.
- **Keep the description concise and human-readable.** A few plain bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate.
- **Reference issues with `Closes #issue-number`** under "Issues closed by this PR" so they auto-close on merge. This goes in the PR description only — never in commit messages.
- **AI assistance in commits may optionally be disclosed with an `Assisted-by:` trailer** naming the model (e.g. `Assisted-by: Claude Opus 4.5`) — do NOT use a `Co-authored-by:` trailer for this.

View File

@@ -1,13 +0,0 @@
---
paths:
- "**/*.py"
---
# Python comments
The bar is the [`comments`](comments.md) rule: nothing where the code is self-explanatory.
- **Names carry the meaning.** Make function and variable names self-explanatory so the comment or docstring is unnecessary in the first place. If a docstring is needed to explain what a function does, fix the name, not the docstring.
- **No file-level docstring.** The filename says what the module is for — `tool_bin.py` gets the tool binary. A module docstring restating that is noise, and a paragraph of design prose at the top of a file goes stale where nobody is looking. A constraint belongs next to the code it constrains, not in a preamble.
- **Docstrings**: only when they say something the name and signature don't — drop them otherwise. Keep them short. A contract that genuinely needs a few lines (interacting flags, retry semantics, an edge case) is fine; a narrative is not.
- **No song and dance.** Comment the constraint or the edge case. Not the narrative, not the rationale, not what the next line does.

View File

@@ -1,19 +0,0 @@
---
paths:
- "tests/**/*.py"
---
# pytest conventions
For the Python integration suite under `tests/`. Setup, running, and suite layout live in [`docs/contributing/tests/integration.md`](../../docs/contributing/tests/integration.md).
- **No `_`-prefixed helper functions in test modules — this is the rule that matters most.** A reader must be able to see what a test does in its body alone, without chasing private helpers that scatter the meaning across the file. Inline the logic: an expression, a comprehension, a few repeated lines are all fine — repetition across tests is cheaper than indirection. When several tests genuinely share non-trivial setup or assertions, that is what fixtures are for — in `tests/fixtures/`, see the next rule. A module-level `_helper()` is never the answer.
- **Fixtures live in `tests/fixtures/` — never under `integration/tests/`.** Not in test modules, not in suite `conftest.py` files. `tests/fixtures/` is the shared library (auth, signoz, clickhouse, logs/metrics/traces seeding, …): reuse what's there before writing anything new; when a new fixture is genuinely needed, add it to the matching `tests/fixtures/` module and register new modules in `tests/conftest.py` `pytest_plugins`. **The one exception: SigNoz-level fixtures in a suite's `conftest.py`.** A suite that needs its own SigNoz spun up with different envs (`create_signoz`/`create_migrator` with `env_overrides` + `cache_key` — e.g. basepath, metricreduction, querier_json_body) keeps that in its `conftest.py`; that is always okay.
- **Fixture only when there is a lifecycle; otherwise a plain function.** A fixture earns its indirection by owning setup/teardown (`yield` + cleanup — `insert_metrics` truncating on teardown) or by provisioning a resource (containers, SigNoz instances). A stateless action or lookup (`create_saved_view`, `find_saved_view_by_name`, wiping a resource list) is a plain importable function in the matching `tests/fixtures/` module, taking `signoz`/`token` as ordinary arguments — never wrap a plain callable in a fixture-factory just to inject `signoz`.
- **Fixtures own their cleanup.** When a test needs seeded state, put the seed + cleanup pair in a fixture (`yield`, then tear down) so tests in the same suite don't interfere — the pattern `insert_metrics` sets: yield a callable, truncate on teardown.
- **Fixture-factory over indirect parametrization.** A fixture that yields a callable (e.g. `insert_metrics(metrics)`) is clearer than `@pytest.mark.parametrize(..., indirect=True)` + `request.param` — the value is an explicit argument, not resolved by magic.
- **Skip at collection, not inside the test body.** Use `pytest.param(..., marks=pytest.mark.skip(reason="…"))` so a skipped case shows as SKIPPED-with-reason **and** short-circuits before its fixtures run (no environment spin-up for a test that won't execute).
- **Test config comes from explicit `--flags`, not the environment.** Wire configuration as pytest options declared in `tests/conftest.py` (`pytest_addoption` — e.g. `--sqlstore-provider`, `--clickhouse-version`); do **not** add `os.environ` fallbacks inside tests or fixtures.
- **snake_case parametrize ids.** `ids=["fill_gaps", "fill_zero"]`, not camelCase.
- **Name suite files with the two-digit prefix (`NN_*.py`).** `pyproject.toml` restricts collection to `[0-9][0-9]_*.py` (plus the bootstrap `setup.py` / `run.py`) — a file that doesn't match is silently never collected.
- **Always run pytest from `tests/`.** `--import-mode=importlib` is what allows same-basename files across suites (`querier/01_logs.py` vs `rawexportdata/01_logs.py`), but it disables pytest's implicit `sys.path` injection — `import fixtures` only resolves via `pythonpath = ["."]` from that rootdir.

View File

@@ -1,95 +0,0 @@
---
name: signoz-page-story
description: Explore a SigNoz page, map the endpoints, states and query params it has, then write its Storybook page story with control-driven msw mocks that reach every state. Use when asked to create, extend or review a Storybook story for a page under frontend/src/pages, to add controls to an existing page story, or to write defineStoryMocks handlers and mock data for a page.
---
# SigNoz page stories
A page story renders the real page inside the real app shell against msw, and its
controls panel can reach every state the page has. The panel is the deliverable,
not the story list.
`frontend/src/storybook/README.md` is the API surface (providers, parameters,
control builders, module mocks, navigation). Read it first; this skill is the
process on top of it.
## Workflow
1. **Map the page**: [references/discovery.md](references/discovery.md). Produce
the inventory (endpoints, states, params, permissions, caps) before writing
code. No inventory, no story.
2. **Skeleton first**: story + empty `defineStoryMocks`, then run it. The console
names the endpoints step 1 missed.
3. **Inventory to controls**: [references/controls.md](references/controls.md).
4. **Mock data and handlers**: builders in `__story_mockdata__`, handlers in the page's
mocks module.
5. **Verify in the browser**: [references/verify.md](references/verify.md). Never
report the story as done without it.
## Rules
- **Default is the loaded page.** `export const Default: Story = {}` with no args,
every widget carrying data. Empty, loading and failed are variants or control
values, never the default.
- **A control is a knob on a response**, resolved through `handlers`, `config` or
`effect`. Never a component prop, never a module mock added for one story.
- **Every branch in the inventory is reachable from the panel.** A state that
needs a code edit to see is a missing control.
- **Never re-declare what every story already has**: banner, side nav, data state
(loaded/loading/error), access preset, permissions, check state.
- **A variant earns a story only when it is worth linking to**: an empty
workspace, a viewer, a page mid-load. Everything else stays a control.
- **Endpoints the page owns go through `response.json`**, so the Data control
covers loaded, loading and failed in one declaration. Endpoints the page cannot
render without (ingestion detection, preferences, feature payloads) take a
plain resolver so the shell survives the loading and error states.
- **Query-param state starts from `route`** (`/logs?tab=explorer`). In-page param
navigation works inside a story; a different pathname is blocked and reported
by the overlay. A control for a param is worth it only when the param is a page
mode someone would want to flip.
- **File layout**: every story file for a page lives under
`src/pages/<Page>/stories/`: `<Page>.stories.tsx`, `<Page>.stories.mocks.tsx`,
payload builders in `stories/__story_mockdata__/<page>.ts`. Nothing
page-specific in `src/storybook/controls/`.
- **The mocks are AI-owned and say so.** `<Page>.stories.mocks.tsx` and every file
under a `__story_mockdata__/` open with this banner, above the imports:
```ts
/**
* AI-owned. Generated and maintained by the `signoz-page-story` skill.
* Do not hand-edit: regenerate instead.
*/
```
The root `.gitattributes` marks both paths `linguist-generated=true`, so the
reviewer gets them collapsed and spends the attention on the rendered page. The
story file is the human surface and never carries the banner. A file with the
banner has to stay regenerable from the page alone: no page knowledge in
`src/storybook/`, and builders typed from `src/api/generated` where the endpoint
has types, so a contract change is a compile error instead of a mock that lies.
- **Reuse fixtures** from `src/mocks-server/` and `src/tests/fixtures/` where they
exist. An endpoint jest needs too belongs in `src/mocks-server/handlers.ts`.
- **Shared response builders live in `src/storybook/msw/__story_mockdata__/`**: typed
helpers like `queryRangeV5ScalarResponse` that multiple pages need. Before
writing a response shape inline, check if a builder exists; if not and the
shape will repeat, add it there. Page-specific builders stay in the page's
`__story_mockdata__/`.
- **No comment is the default.** Write one only for what the code cannot show:
a shape the backend dictates, an app bug the mock reproduces, an ordering or
cap the page depends on, a workaround and the reason for it. Never restate a
name, a type, or what a builder plainly builds; if the sentence reads as the
signature in prose, delete it. Nothing addressed to a reviewer. The one
comment a story always gets is its own doc comment: what it shows, in the
page's own terms.
## Done means
- [ ] `Default` shows the page with data, checked in dark and light
- [ ] the mocks module and every `__story_mockdata__` file carry the AI-owned banner
- [ ] every control flipped once, its effect seen on screen
- [ ] console clean: no `[storybook] no msw handler`, no 501, no msw unhandled
request, no React warning
- [ ] no navigation overlay on mount
- [ ] `pnpm tsgo --noEmit`, `pnpm exec oxlint <files>`,
`pnpm exec oxfmt --check <files>` all clean. The repo has no
prettier: `pnpm exec prettier` prints a pass while exiting 254

View File

@@ -1,165 +0,0 @@
# Turning the inventory into controls
Every row of the inventory becomes a control, a global control that already
exists, or a documented reason it cannot be one.
## Imports
Paths written as `src/storybook/...` in prose are repo paths, not import
specifiers. Stories import through the `@/` alias (`@/*``./src/*`); modules
inside `src/storybook/` import each other relatively.
| Import | From |
| --- | --- |
| `toggleControl`, `countControl`, `choiceControl`, `multiChoiceControl` | `../controls/controls` |
| `defineStoryMocks`, `storyMocks` | `../controls/defineStoryMocks` |
| `PageStoryArgs` | `../controls/resolveStoryMocks` |
| `MockRequest`, `MockResponse` | `../controls/types` |
| the page's mocks, from the story | `./<Page>.stories.mocks` |
| `queryRangeV5ScalarResponse`, `queryRangeV5RawResponse`, etc. | `@/storybook/msw/__story_mockdata__/queryRange` |
## Which builder
`src/storybook/controls/controls.ts`:
| The state is | Builder |
| --- | --- |
| on or off (a signal ingesting, a feature present) | `toggleControl` |
| how many rows a list has | `countControl` |
| one of several modes (tab, visibility, plan, severity filter) | `choiceControl` |
| a subset (steps skipped, columns shown, signals selected) | `multiChoiceControl` |
Rules that come with them:
- `countControl` `max` goes past what the page renders, so a story can show the
cap being hit. `0` is the empty state, which is why an empty list rarely needs
its own story. When the cap is in the *request* (`?limit=5`) rather than the
renderer, stop `max` at the limit: a longer response is a body the backend
cannot send.
- `choiceControl` options come from a `const` array typed with
`(typeof X)[number]`, not from string literals scattered in the handlers.
- Defaults describe the fully-populated page. The panel starts where `Default`
starts.
- `group` is `'<Page> · <facet>'`, such as `'Services · lists'` or
`'Alerts · rules'`. Keep a page's knobs in two or three groups, not one per
control.
- `description` only when the name does not carry the effect (what dismissing
does, what the cap is, which widget it feeds).
## Which hook
`defineStoryMocks` takes three, all optional:
- `handlers(values, response)`: the page's endpoints. Everything the page owns
goes through `response.json`, so the global Data control turns the whole page
into loading or failed without a second declaration. An endpoint the page
cannot render at all without (ingestion detection, preferences, license
payloads) takes a plain `rest.get(...)` resolver instead, so the shell stays
visible while the rest hangs or fails.
- `config(values)`: `SignozStoryConfig` for knobs no endpoint covers: `route`,
`appContext`, `reduxState`, `queryBuilder`, `theme`.
- `effect(values)`: module-level state no provider exposes.
One endpoint feeding several widgets stays one handler that reads the request.
`response.json` hands the request to the builder and awaits it, so reading a
query param, or a POST body, does not cost the Data control:
```ts
rest.get(
'http://localhost/api/v1/explorer/views',
response.json((req) =>
savedViews(values.savedViews, req.url.searchParams.get('sourcePage') ?? 'logs'),
),
),
```
```ts
rest.post(
'http://localhost/api/v5/query_range',
response.json(async (req) => {
const body = (await req.json()) as QueryRangeRequestV5;
const signal = body.compositeQuery?.queries?.[0]?.spec?.signal;
return countResponse(values[`${signal}Ingestion`] ? 4213 : 0);
}),
),
```
Reach for a plain `rest.post(url, async (req, res, ctx) => …)` only when the
endpoint has to keep answering while the Data control is on `loading` or
`error`: detection calls the page cannot render without.
## Mutations
A control drives the response, so a write the page makes against state a control
owns does not stick: the refetch answers with the control's value and the button
appears to do nothing. Two honest options: leave it declarative and say so in
the PR, or move the state into `effect` so the handler can read what the page
wrote. Never fake the write by mutating a builder's module state without saying
where the state lives.
## Wiring it up
```ts
// src/pages/Services/stories/Services.stories.mocks.tsx
/**
* AI-owned. Generated and maintained by the `signoz-page-story` skill.
* Do not hand-edit: regenerate instead.
*/
export const servicesMocks = defineStoryMocks({
controls: {
services: countControl('Services', { group: LISTS, value: 8, max: 12 }),
apdex: choiceControl<ApdexState>('Apdex', {
group: HEALTH,
options: APDEX_STATES,
value: 'mixed',
}),
},
handlers: (values, response) => [
rest.post(
'http://localhost/api/v2/services',
response.json(() => buildServices(values.services, values.apdex)),
),
],
});
```
```tsx
// src/pages/Services/stories/Services.stories.tsx
type ServicesArgs = PageStoryArgs<typeof servicesMocks>;
const meta = {
title: 'Pages/Services',
component: Services,
...storyMocks(servicesMocks, { route: ROUTES.APPLICATION, layout: 'app' }),
} satisfies Meta<ServicesArgs>;
```
`PageStoryArgs` folds in the global controls, so a story's `args` can set
`access`, `dataState` or `banner` next to the page's own knobs and stay typed.
## Not a control
- Anything the global controls already cover: banner, side nav, data state,
access preset, permissions, check state.
- A knob whose effect nobody can see on the page. Delete it or find the widget it
was supposed to drive.
- A raw payload as an object control. Controls carry intent (`5 dashboards`,
`viewer`), and the builder turns intent into the payload.
- Anything that needs a module mock or a component prop to work. If the state
cannot be produced from a response, config or module state, say so in the PR
instead of faking it.
## Control or story
Default to a control. Write a story when the state is worth a link:
- the fresh workspace, because that is what a new user sees
- the restricted user, when permissions visibly change the page
- a page-defining mode (a tab, a category) that has its own layout
Combinations of controls do not need stories, which is what the panel is for.
Each story gets one prose doc comment: what it shows, in the page's own terms.
Everywhere else the comment rule in SKILL.md applies: write one only for what
the code cannot show.

View File

@@ -1,64 +0,0 @@
# Mapping a page
Two passes: read the code, then let the running story correct you. Write the
inventory down: it is what the controls are derived from, and the only
protection against a story that renders one state and calls it a page.
## Pass 1: read the page
Start at `src/pages/<Page>/` and follow it outward: the containers it mounts
(`src/container/<Feature>/`), the hooks those use, the components with their own
fetches. Stop at leaf components that take props only.
Grep recipes, run against the page's directories:
| Looking for | Grep |
| --- | --- |
| endpoints | `useQuery\|useMutation\|useInfiniteQuery`, then the `api/` module it calls |
| endpoint URLs | the api module's `axios.get\|post` |
| endpoint URLs behind a generated hook | the hook lives in `src/api/generated/services/<name>/index.ts` and the URL only appears in the fetcher body: `rg 'url: \`' src/api/generated/services/<name>/` |
| url state | `useUrlQuery\|useUrlQueryData\|useUrlSearchState\|useQueryState\|QueryParams\.` |
| navigation | `useSafeNavigate\|history.push\|<Link` |
| permissions | `useAuthZ\|AuthZGuard\|AuthZButton\|hasEditPermission\|routePermission` |
| flags and prefs | `useFeatureFlag\|FeatureKeys\.\|USER_PREFERENCES\.\|userPreferences` |
| empty and error branches | `isLoading\|isError\|isFetching\|length === 0\|!data` |
| render caps | `slice(0,\|PAGE_SIZE\|pageSize\|limit` |
`src/constants/routes.ts` has the route, `src/constants/query.ts` the param names,
`src/lib/authz/README.md` how a permission check resolves.
## The inventory
One table, in the story's PR or scratch notes:
| Endpoint | Feeds | States it can be in |
| --- | --- | --- |
| `GET /api/v1/x` | the header count | populated, zero, error |
Plus four short lists:
- **Query params** the page reads, and what each one switches.
- **Permission checks** the page makes, and what disappears when each is denied.
- **Preferences and flags** that change layout (dismissed banners, onboarding
checklists, opt-in views).
- **Caps**: how many rows each list renders before it truncates or paginates.
A state that appears in this inventory and not in the controls panel is a bug in
the story.
## Pass 2: let it run
Write the story and an empty `defineStoryMocks({ controls: {} })`, point it at the
route with `layout: 'app'`, then open it (see verify.md). The console is the
oracle:
- `[storybook] no msw handler` or a 501 from the catch-all: an endpoint pass 1
missed. Add it to the inventory.
- an msw unhandled-request warning: a request going to an origin the handlers do
not answer on. handlers are declared against `http://localhost`.
- a spinner that never resolves with the Data control on `loaded`: a handler
whose URL does not match what the page calls.
- the navigation overlay on mount: the page redirects, usually because `route`
is wrong or a guard is failing on a permission the controls have not granted.
Repeat until the console is silent. Only then start declaring controls.

View File

@@ -1,110 +0,0 @@
# Verifying a page story
A story is not done because it compiles. It is done when each control has been
seen changing the page and the console is silent.
## Run it
```bash
cd frontend && pnpm storybook --ci --quiet # :6006, background it
```
A newly added `.stories.tsx` takes a few seconds to appear in `index.json` on an
already-running server; an empty first poll is not a broken `stories` glob.
Story ids come from the meta title: `Pages/Services``pages-services`, plus the
story export in kebab-case. Render one story on its own:
```
http://localhost:6006/iframe.html?id=pages-services--default&viewMode=story
```
## Flip controls from the URL
Args are settable in the iframe URL, so a whole sweep runs headless without
touching the panel. Booleans go as `!true` / `!false`, numbers bare, arrays
indexed, several separated by `;`, and the theme through `globals`:
```
&args=services:0;apdex:poor;access:viewer;dataState:loading
&args=signals[0]:logs;signals[1]:traces
&globals=theme:light
```
That is the cheap way to check a control does something: load with and without
it, diff the page text.
## Drive it
Playwright lives in the repo's e2e workspace, so a scratch script can use it
directly:
```js
import pw from '<repo>/tests/e2e/node_modules/playwright/index.js';
const { chromium } = pw;
const browser = await chromium.launch();
const page = await browser.newPage();
const problems = [];
page.on('console', (m) => {
if (m.type() === 'error' || m.type() === 'warning') problems.push(m.text());
});
page.on('pageerror', (e) => problems.push(e.message));
await page.goto(`${story}&args=services:0`, { waitUntil: 'networkidle' });
await page.locator('body').waitFor();
console.log((await page.locator('body').innerText()).slice(0, 1500), problems);
await browser.close();
```
Screenshots are worth taking for `Default` in both themes
(`&globals=theme:light`): text extraction does not catch an unstyled page.
## Gates
- **Console silent.** `[storybook] no msw handler`, a 501 from the catch-all, an
msw unhandled-request warning, a React key or state warning: assume the story
is wrong first. A warning that survives is sometimes the app's. Prove it by
turning off the control that renders the widget and watching the warning go
with it, and by finding the same component elsewhere doing it right. Then
report the app bug in the PR. Never invent a field the API does not return to
silence a warning.
- **No navigation overlay on mount.** "Navigation blocked in Storybook" on load
means the page is trying to leave: wrong `route`, or a guard denying on a
permission the controls did not grant.
- **Every control moves something.** Sweep them one at a time from the URL and
diff the page text. A control with no diff is either wired to nothing or aimed
at a widget that is not rendering. Some only show their effect after an
interaction, such as a tab that has to be clicked or a select that has to be
opened. Drive that interaction rather than calling the control unobservable.
- **Both themes render styled.** An unstyled page means the story is not inside
the provider decorator, or `<body data-theme>` was lost.
- **Roles agree.** `<body data-signoz-story-role>` and
`<body data-signoz-context-role>` disagreeing means the page reads a different
`AppContext` than the story config fills.
- **The page's own navigation works.** Tabs, filters and pagination that write
query params should re-render the page in place; only leaving the page belongs
in the overlay.
## Then the usual
```bash
pnpm tsgo --noEmit
pnpm exec oxlint <changed files>
pnpm exec oxfmt --check <changed files>
```
There is no prettier in this repo. `pnpm exec prettier --check` fetches something
else, prints `Prettier: All files formatted correctly` and exits 254: a pass that is
not one.
## Common failures
| Symptom | Cause |
| --- | --- |
| endless spinner with Data on `loaded` | handler URL does not match the call; handlers answer on `http://localhost` |
| page renders but empty | response shape wrong; compare against the api module's type, not a guess |
| 501 in the console | endpoint nobody mocked; the catch-all is answering |
| new control missing from the panel | project-level control added; the tab needs a reload |
| control flips but nothing changes | the widget is gated by something else: a permission, a flag, a preference |
| shell disappears in `loading` | an endpoint the shell needs went through `response.json`; give it a plain resolver |

View File

@@ -1,294 +0,0 @@
---
name: storybook-visual-diff
description: Screenshot a set of SigNoz Storybook stories, then pixel-diff two runs to see what a CSS or component change did, with the changes tinted over the new shot. Use when asked to take story screenshots, capture a visual baseline, compare before/after of a style change, or find which pages a change affects.
---
# Storybook visual diff
Two scripts under `frontend/scripts`:
- `story-shots.mjs` — screenshots stories off a running Storybook dev server.
- `story-shots-diff.mjs` — pixel-diffs two runs and paints what moved.
Output goes to `frontend/.story-shots/` (gitignored), one directory per run.
## 0. Settle what is being compared, first
A diff is only worth taking when the two runs straddle something. Run twice over
the same tree and the answer is zero, or the noise floor: true, and useless.
So before starting a server, pin down four things. Whatever the prompt already
says, take it and do not ask again; ask only for what is genuinely missing, in
**one** `AskUserQuestion` call.
| To settle | Ask | Options |
| --- | --- | --- |
| Job | "What should this run produce?" | shoot only · baseline for a change you are about to make · compare against a change already in the working tree · compare this branch against another (`main` by default, or one the user names) · compare two configurations of the same story (`--args`, clock, width) · noise floor (same tree twice) |
| Scope | "Which stories?" | offer 2-3 concrete selections read off `index.json` (a page, a `--title` prefix, everything), never open-ended |
| Themes | "Which themes?" | dark · dark + light |
| Read-out | "How should the diff read?" | `green` (changed pixels over the after shot) · `green-parallel` (before \| after \| diff, side by side) · `red` · `red-parallel` · `none` (keep both runs, do not diff) |
Skip a row when the prompt answers it, and skip the whole call when the prompt
answers all of it ("shoot the pods tooltips in both themes" needs no question).
Skip Read-out too whenever the job is *shoot only*, and take `none` for what it
says: shoot both sides, report both paths, run no comparison. When the prompt
says nothing at all, ask; a silent guess here burns ~6 min per sweep on the
wrong stories.
The job decides which loop below to run:
| Job | Loop |
| --- | --- |
| **shoot only** | §1, §2, stop. Report the paths. No diff, no second run. |
| **baseline first** | the full loop, stopping after step 2 to hand the change back. The user makes it, then continue at step 4. |
| **change already in the tree** | the tree *is* the after state. `git stash` (or check out the base commit) to shoot the before, restore, shoot the after. Confirm the working tree is clean enough to stash before touching it, and restore it even if a capture fails. |
| **branch vs branch** | shoot the current branch, then `git switch <base>` in place (stash first if the tree is dirty), restart the dev server, shoot again, switch back and unstash. Restart matters: HMR does not survive a whole-branch swap cleanly. Get the tree back to where it started even if a capture fails. |
| **noise floor** | two runs, same tree, diff. The number is the harness's floor, not a finding. |
| **config vs config** | same tree, two runs that differ only in flags: `--args`, `--clock`, `--width`, `--theme`, `--motion`. Filenames stay identical, so the pairs line up and the caption names what changed. |
## The loop
1. Capture the baseline **before touching anything**.
2. Capture it a second time and diff the two. That is the noise floor: anything
it reports is what the harness cannot hold still, and no conclusion about the
change may rest on those stories. Cheap on a handful of stories, ~6 min per
32, so on a wide sweep run it over the two or three stories the change is
aimed at instead of all of them.
3. Make the change.
4. Capture again into a third directory.
5. Diff, then read the tinted shot of the largest movers to judge the change.
## 1. One dev server, on a free port
`storybook dev` keys its Vite dep cache off the config dir, so two servers on the
same `-c` serve mismatched prebundles and every story dies with `Invalid hook
call`. Check what is already up first — port 6006 is often another repo's
Storybook, and its `index.json` then indexes the wrong stories:
```bash
for port in 6006 6007; do
curl -s -m 2 "http://localhost:$port/index.json" | head -c 60 && echo " <- $port"
done
```
Start the SigNoz one on a free port, from the repo's own binary so no package
manager shim is in the way:
```bash
cd frontend
nohup ./node_modules/.bin/storybook dev -p 6007 --no-open --quiet \
> "${TMPDIR:-/tmp}/signoz-storybook.log" 2>&1 &
```
It is ready when `curl -s localhost:6007/index.json` returns JSON whose
`entries` hold SigNoz story ids.
## 2. Capture
Playwright is not a frontend dependency. The script finds it in `tests/e2e`
(`pnpm -C tests/e2e install`, `@playwright/test` is enough) or in a global
install, and launches Playwright's own chromium, falling back to an installed
Chrome. Two escape hatches when that is not what a machine has:
```bash
export PLAYWRIGHT_MODULE=/path/to/playwright # a different install
export CHROME_PATH=/path/to/chrome # a specific browser binary
```
Then pick the stories. `--list` prints the selection without shooting anything:
```bash
# every tooltip story of every page
node scripts/story-shots.mjs .story-shots/baseline \
--port 6007 --title Pages/ --name tooltip --theme dark
# a handful of stories by id or by title/name substring, both themes
node scripts/story-shots.mjs .story-shots/baseline \
--port 6007 --stories pages-noz,dashboards/detail --theme dark,light
```
| Flag | Meaning |
| --- | --- |
| `--stories <match>` | id or `Title/Name` substring, repeatable or comma-separated. Omit for every story. |
| `--title <prefix>` | only titles starting with the prefix (`Pages/`, `Components/`) |
| `--name <match>` | only story names containing the match |
| `--theme dark,light` | one pass per theme; omit for the story's own default (dark) |
| `--args <k:v;k2:v2>` | arg overrides, Storybook's own `?args=` syntax, repeatable. A dotted value is dropped by Storybook itself, so map it to a slug inside the story's mocks |
| `--port` | dev server port, or `$SB_PORT` |
| `--width <px>` | the only fixed dimension, default 1680 |
| `--height <px>` | shortest the viewport may be, default 1200 |
| `--max-height <px>` | tallest it may grow to, default 8000 |
| `--grow <what>` | `scrollers` (default) grows the viewport until the page's own scrollers fit, `document` only follows the document height, `none` keeps `--height` |
| `--settle <ms>` | wait after the page goes quiet, default 1500 |
| `--clock <iso\|live>` | wall clock the page reads, passed to the preview as `?storyClock`; `live` unfreezes it |
| `--motion` | keep animations and transitions running (sets the `motion` global to `live`) |
| `--ignore <selector>` | hide matching elements, on top of `[data-shot-ignore]` and `[data-chromatic="ignore"]` |
| `--flat` | write `<out>/<id>.png`, no theme directory |
| `--no-caption` | leave the caption band off the shots |
| `--list` | print the matched stories and exit |
Files land at `<out>/<theme>/<story-id>.png`, next to a `shots.json` recording
what each shot is (id, title, name, theme, `ok`/`busy`, the caption's height in
rows) and how the run was configured (args, clock, width, height, grow, motion,
settle, ignore). Keep the flags identical between the two runs or the diff pairs
nothing.
Every shot carries the caption band described below, so a single screenshot says
what it is on its own. `--no-caption` leaves it off, and so does a machine
without ImageMagick (with a warning). The band never changes the shot's width
(long text wraps rather than widening the canvas) and its height is recorded, so
the diff crops it back off and never reports one caption against another. Two
runs whose captions are different heights still diff to zero. A story that never held still for two
identical frames is logged `busy` instead of `ok` — treat its diff as suspect.
Dark alone is enough while iterating on the harness; add `light` for the run you
report.
## 3. Diff
```bash
node scripts/story-shots-diff.mjs .story-shots/baseline .story-shots/capped .story-shots/diff
```
Prints `<changed pixels> <theme>/<story>.png`, largest first, and writes one
image per pair. Needs ImageMagick for PNG encode/decode (7's `magick`, or 6's
`convert`/`identify`/`montage`); the comparison itself is in the script.
| Flag | Meaning |
| --- | --- |
| `--mode green` | default. The after shot with the changed pixels painted over it, exactly the pixels that changed. What Chromatic shows. |
| `--mode green-parallel` | `previous \| current \| diff` in one image, each tile labelled above it, on a gutter inverted from the theme. The diff tile is the `green` one, so the after shot stays readable underneath. |
| `--mode red` | the after shot faded to 10%, changed pixels in red. A pixelmatch-style diff, easiest to read when the change is a thin edge. |
| `--mode red-parallel` | the same three tiles, with the `red` diff. Best when the change is a thin edge that the unfaded shot would swallow. |
| `--threshold <0..1>` | how far a pixel must move to count. Default 0.063, Chromatic's `diffThreshold`. |
| `--include-aa` | count antialiasing changes too. Off by default, as in Chromatic. |
| `--tint <#rrggbb>` | override the mode's colour. |
| `--no-caption` | drop the caption band. |
### The caption
Both scripts stamp a band on top of what they write: `story-shots.mjs` on each
shot, from the story and the run's own settings; `story-shots-diff.mjs` on each
diff, read out of the two runs' `shots.json`. It carries the story's
`Title/Name`, then its id, theme and `busy` flag, then the settings both runs
shared, each reading `key:value`. Whatever the two runs did **differently** goes
on the side it belongs to: under `previous` and `current` on the parallel tiles,
on two lines of the band otherwise. So a pair that differs only in `--args` says
so on its face, which is what makes several shots of one story tellable apart.
The shots' own bands are cropped off before comparing and before going into the
tiles, so nothing in the output is a diff of a caption. Type size follows the
image width, so it stays readable with the whole image viewed at fit-to-width;
the heading is set in an installed sans and the detail lines in a mono, falling
back to ImageMagick's default when neither is on the machine. Without a manifest
the band falls back to the file path, and a directory of captioned shots whose
`shots.json` is missing has nothing to crop by, so its captions do land in the
diff. Keep `shots.json` next to the shots.
### How the comparison works
Chromatic's own capture and diff run server-side — `chromatic-cli` uploads a
built Storybook and contains no capture or comparison code at all. What is public
is the parameter contract, and the numbers in it say what the comparison is:
`diffThreshold` defaults to `0.063` on a 0-1 scale, which is pixelmatch's
`threshold`, and `diffIncludeAntiAliasing` defaults to false, which is
pixelmatch's `includeAA: false`. So the script implements that comparison:
1. Both PNGs are read as raw RGBA through `magick … RGBA:-`.
2. Per pixel, the squared YIQ distance between the two colours (weights
`0.5053 / 0.299 / 0.1957`), compared against `35215 * threshold²` — 35215 is
the largest distance two 8-bit colours can have. Chroma is included, so a
colour swap at equal brightness still counts.
3. A pixel over the threshold is dropped when it is only antialiasing: it is the
darkest or lightest of its eight neighbours, and the other image has a pixel
around there doing the same job. This is what keeps a subpixel glyph edge from
reading as a change.
4. What survives is painted at full opacity, one output pixel per changed input
pixel. No dilation, no blobs — a one-pixel shift shows as a one-pixel line.
A pair whose shots are different sizes is compared over the overlap, and every
row and column that exists in only one of them counts as changed.
Pairing is by `<theme>/<story-id>.png`, so a story that exists on only one side
(new on the feature branch, renamed, retitled) has nothing to pair with and is
skipped silently. On a branch-vs-branch run, compare the two runs' file lists
before reading the numbers.
## What makes a shot reproducible
Most of it is in the preview, not in the script, so a Chromatic build in the
cloud shoots the same page: `.storybook/preview-head.html` freezes the clock,
and `settleForCapture` (the preview's `afterEach`, which runs after `play`)
parks the animations and snaps the bottom-pinned lists. The script drives the
rest:
- **Storybook's own render phase is the readiness signal.** It waits for
`window.__STORYBOOK_PREVIEW__.storyRenders[].phase === 'finished'`, which is
reached only after the loaders, the decorators and the story's `play` are done.
A DOM check cannot see a `play` still running. (Storybook 10 spells the final
phase `finished`, not `completed`.)
- **Network quiescence, not `networkidle`.** react-query retries and msw keep
requests going after load, and a few stories hang a request by design, so the
wait is "no request for 600ms", capped at 15s.
- **The clock is frozen** (`2026-06-15T12:00:00Z`), by the preview itself. Chart windows, `4 mins ago`
labels and trial countdowns all derive from `now`; a live clock alone moved
8000 pixels on the dashboards list and redrew every chart axis.
- **Animations are parked on their last frame** by `html.sb-still`, a
zero-length single iteration with `forwards` fill, plus `prefers-reduced-
motion`. The Motion toolbar item (`still` by default) turns it off. An infinite
spinner is otherwise caught at a random angle.
- **`document.fonts.ready`**, because text reflows when a face lands late.
- **Lists pinned to their bottom are snapped onto it**, once by the preview and
again by the script after the page goes quiet. A virtuoso list settles a
few pixels short of the end depending on the order its items were measured in.
- **Two identical frames in a row**, because what a page is still waiting on is
often not observable from outside it.
- **`[data-shot-ignore]`, `[data-chromatic="ignore"]` and `--ignore <selector>`**
hide a region that cannot be held still; Chromatic excludes the same attribute
from its comparison.
- **The width is the only fixed dimension.** Chromatic's `viewports` are widths;
the height follows the page. `src/styles.scss` pins `html, body, #root` to
`height: 100%; overflow: hidden`, so the document never outgrows the viewport
and its height says nothing: what overflows are the shell's inner scrollers.
`--grow scrollers`, the default, grows the viewport until the tallest in-flow
scroller fits, so nothing is cut off and no scrollbar is left in the shot (the
dashboards list goes to 2226px in one round). Popups are skipped — they are out
of the flow, and a tall dropdown would otherwise drag the shot to a height
nothing on the page needs. A page that sizes a panel in `vh` grows its own
content as the viewport grows, so no height ever fits it and the rounds only
chase — `.alert-chart-container` is `57vh`, which puts Create Alert's fixed
point at 4344px with an empty band on top. Those pages are shot at `--height`
with their own scrollbar, which is what they look like in a browser, and the
log says `(viewport-sized content, stopped chasing Npx)`.
With all of that, 29 of the 32 page tooltip stories are byte-identical across
runs. The three that are not, and why:
| Story | Residual | Cause |
| --- | --- | --- |
| `kubernetes-pods--tooltips-in-options-panel` | ~13k px | 24 tooltips held open in an overlapping cluster; they portal to `body` in mount order, and the drawer's own tooltips mount before or after the list's depending on when their data lands, so overlapping tooltips stack differently. Panel geometry itself is stable. |
| `settings-role-editor--tooltips-in-json-editor` | ~2.5k px | monaco re-measures and lands one pixel off. |
| `traces-trace-details--tooltips` | ~800 px | same class, one row of the waterfall. |
Each is bimodal — two stable arrangements — so the same number reappears run
after run. Diff a story against itself before believing its number, and reach
for `--ignore` when a region cannot be settled.
## Gotchas
- **Zero pixels is a real answer.** A story whose tooltips are all short is
unaffected by a tooltip rule; it is not a broken capture.
- **The selector matters more than the rule.** A global rule on
`[data-slot='…']` only reaches design-system components. antd's own tooltips
(`.ant-tooltip-inner`, e.g. the Create Alert help popups) are untouched, which
is why some stories show no diff at all.
- **Global style overrides need `!important`.** `src/styles.scss` loads before
the design system injects its CSS-module styles at runtime, so a plain rule on
a `[data-slot='…']` element loses. A component-level `!important` of the same
specificity still wins over it — `PanelStatusPopover.module.scss` keeps its own
`max-width: 520px !important`.
- **A fresh context per story** is why a full sweep takes ~6 min for 32 stories.
Reusing one page loses the msw service worker re-registration race and stories
start failing after a few navigations.
- **Stories behind a hover, drawer or modal** only render what their `play`
reaches. If a state is missing from the shot, the story needs the `play`, not
the script.

View File

@@ -1,6 +1,6 @@
services:
init-clickhouse:
image: clickhouse/clickhouse-server:25.12.5
image: clickhouse/clickhouse-server:25.5.6
container_name: init-clickhouse
command:
- bash
@@ -18,7 +18,7 @@ services:
volumes:
- ${PWD}/fs/tmp/var/lib/clickhouse/user_scripts/:/var/lib/clickhouse/user_scripts/
clickhouse:
image: clickhouse/clickhouse-server:25.12.5
image: clickhouse/clickhouse-server:25.5.6
container_name: clickhouse
volumes:
- ${PWD}/fs/etc/clickhouse-server/config.d/config.xml:/etc/clickhouse-server/config.d/config.xml
@@ -67,7 +67,7 @@ services:
timeout: 5s
retries: 3
telemetrystore-migrator:
image: signoz/signoz-otel-collector:v0.144.6
image: signoz/signoz-otel-collector:v0.142.0
container_name: telemetrystore-migrator
environment:
- SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000

4
.gitattributes vendored
View File

@@ -1,3 +1 @@
*.css linguist-detectable=false
*.stories.mocks.tsx linguist-generated=true
**/__story_mockdata__/** linguist-generated=true
*.css linguist-detectable=false

37
.github/CODEOWNERS vendored
View File

@@ -109,7 +109,10 @@ go.mod @therealpandey
/pkg/modules/role/ @therealpandey
/pkg/types/coretypes/ @therealpandey @vikrantgupta25
/frontend/src/lib/authz/ @H4ad
/frontend/src/hooks/useAuthZ/ @H4ad
/frontend/src/components/GuardAuthZ/ @H4ad
/frontend/src/components/AuthZTooltip/ @H4ad
/frontend/src/components/createGuardedRoute/ @H4ad
/frontend/src/container/RolesSettings/ @H4ad
/frontend/src/components/RolesSelect/ @H4ad
/frontend/src/pages/MembersSettings/ @H4ad
@@ -135,7 +138,7 @@ go.mod @therealpandey
/tests/integration/ @therealpandey
# e2e tests
/tests/e2e/ @AshwinBhatkal
/tests/e2e/ @SigNoz/frontend-maintainers
# Flagger Owners
@@ -152,36 +155,44 @@ go.mod @therealpandey
## Dashboard Types
/frontend/src/types/api/dashboard/ @SigNoz/pulse-frontend
/frontend/src/types/api/widgets/ @SigNoz/pulse-frontend
/frontend/src/api/types/dashboard/ @SigNoz/pulse-frontend
## Widget Card
## Dashboard List
/frontend/src/container/WidgetCard/ @SigNoz/pulse-frontend
/frontend/src/pages/DashboardsListPage/ @SigNoz/pulse-frontend
/frontend/src/container/ListOfDashboard/ @SigNoz/pulse-frontend
# Dashboard Widget Page
/frontend/src/pages/DashboardWidget/ @SigNoz/pulse-frontend
/frontend/src/container/NewWidget/ @SigNoz/pulse-frontend
## Dashboard Page
/frontend/src/pages/DashboardPage/ @SigNoz/pulse-frontend
/frontend/src/container/DashboardContainer/ @SigNoz/pulse-frontend
/frontend/src/container/GridCardLayout/ @SigNoz/pulse-frontend
## Public Dashboard Page
/frontend/src/pages/PublicDashboard/ @SigNoz/pulse-frontend
/frontend/src/container/PublicDashboardContainer/ @SigNoz/pulse-frontend
## Dashboard Libs + Components
/frontend/src/lib/uPlotV2/ @SigNoz/pulse-frontend
/frontend/src/lib/visualization/ @SigNoz/pulse-frontend
/frontend/src/lib/dashboard/ @SigNoz/pulse-frontend
/frontend/src/lib/dashboardVariables/ @SigNoz/pulse-frontend
/frontend/src/components/NewSelect/ @SigNoz/pulse-frontend
## Dashboard Pages
/frontend/src/pages/DashboardPage/ @SigNoz/pulse-frontend
/frontend/src/pages/DashboardsListPage/ @SigNoz/pulse-frontend
## Dashboard V2
/frontend/src/pages/DashboardPageV2/ @SigNoz/pulse-frontend
/frontend/src/pages/DashboardsListPageV2/ @SigNoz/pulse-frontend
## Infrastructure Monitoring
/frontend/src/pages/InfrastructureMonitoring/ @SigNoz/pulse-frontend
/frontend/src/container/InfraMonitoringHosts/ @SigNoz/pulse-frontend
/frontend/src/container/InfraMonitoringHostsV2/ @SigNoz/pulse-frontend
/frontend/src/container/InfraMonitoringK8s/ @SigNoz/pulse-frontend
/frontend/src/container/InfraMonitoringK8sV2/ @SigNoz/pulse-frontend
## Alerts
/frontend/src/pages/AlertList/ @SigNoz/pulse-frontend

View File

@@ -1,13 +1,85 @@
<!--A few plain bullets saying what changed and why, for a reviewer skimming it - not a wall of text, not a restatement of the diff, not generated boilerplate.-->
#### Description
## Pull Request
---
### 📄 Summary
> Why does this change exist?
> What problem does it solve, and why is this the right approach?
#### Screenshots / Screen Recordings (if applicable)
> Include screenshots or screen recordings that clearly show the behavior before the change and the result after the change. This helps reviewers quickly understand the impact and verify the update.
<!--Reference issues using `Closes #issue-number` to enable automatic closure on merge. -->
#### Issues closed by this PR
> Reference issues using `Closes #issue-number` to enable automatic closure on merge.
<!--If applicable, include screenshots or screen recordings that clearly show the behavior before the change and the result after the change. -->
#### Screenshots / Screen Recordings
---
<!--Anything reviewers should keep in mind while reviewing -->
#### Additional Information
### ✅ Change Type
_Select all that apply_
<!--Please delete paragraphs that you did not use before submitting.-->
- [ ] ✨ Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only
---
### 🐛 Bug Context
> Required if this PR fixes a bug
#### Root Cause
> What caused the issue?
> Regression, faulty assumption, edge case, refactor, etc.
#### Fix Strategy
> How does this PR address the root cause?
---
### 🧪 Testing Strategy
> How was this change validated?
- Tests added/updated:
- Manual verification:
- Edge cases covered:
---
### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?
- Blast radius:
- Potential regressions:
- Rollback plan:
---
### 📝 Changelog
> Fill only if this affects users, APIs, UI, or documented behavior
> Use **N/A** for internal or non-user-facing changes
| Field | Value |
|------|-------|
| Deployment Type | Cloud / OSS / Enterprise |
| Change Type | Feature / Bug Fix / Maintenance |
| Description | User-facing summary |
---
### 📋 Checklist
- [ ] Tests added or explicitly not required
- [ ] Manually tested
- [ ] Breaking changes documented
- [ ] Backward compatibility considered
---
## 👀 Notes for Reviewers
<!-- Anything reviewers should keep in mind while reviewing -->
---

View File

@@ -61,6 +61,13 @@ jobs:
echo 'VITE_SENTRY_AUTH_TOKEN="${{ secrets.SENTRY_AUTH_TOKEN }}"' >> frontend/.env
echo 'VITE_SENTRY_ORG="${{ secrets.SENTRY_ORG }}"' >> frontend/.env
echo 'VITE_SENTRY_PROJECT_ID="${{ secrets.SENTRY_PROJECT_ID }}"' >> frontend/.env
echo 'VITE_SENTRY_DSN="${{ secrets.SENTRY_DSN }}"' >> frontend/.env
echo 'VITE_TUNNEL_URL="${{ secrets.TUNNEL_URL }}"' >> frontend/.env
echo 'VITE_TUNNEL_DOMAIN="${{ secrets.TUNNEL_DOMAIN }}"' >> frontend/.env
echo 'VITE_POSTHOG_KEY="${{ secrets.POSTHOG_KEY }}"' >> frontend/.env
echo 'VITE_PYLON_APP_ID="${{ secrets.PYLON_APP_ID }}"' >> frontend/.env
echo 'VITE_APPCUES_APP_ID="${{ secrets.APPCUES_APP_ID }}"' >> frontend/.env
echo 'VITE_PYLON_IDENTITY_SECRET="${{ secrets.PYLON_IDENTITY_SECRET }}"' >> frontend/.env
echo 'VITE_DOCS_BASE_URL="https://signoz.io"' >> frontend/.env
echo 'VITE_ENVIRONMENT="production"' >> frontend/.env
echo 'VITE_VERSION="${{ steps.build-info.outputs.version }}"' >> frontend/.env
@@ -103,6 +110,7 @@ jobs:
-X github.com/SigNoz/signoz/pkg/version.branch=${{ needs.prepare.outputs.branch }}
-X github.com/SigNoz/signoz/ee/zeus.url=https://api.signoz.cloud
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=https://license.signoz.io
-X github.com/SigNoz/signoz/ee/query-service/constants.LicenseSignozIo=https://license.signoz.io/api/v1
-X github.com/SigNoz/signoz/pkg/analytics.key=9kRrJ7oPCGPEJLF6QjMPLt5bljFhRQBr'
DOCKER_BASE_IMAGES: '{"alpine": "alpine:3.20.3"}'
DOCKER_DOCKERFILE_PATH: ./cmd/enterprise/Dockerfile.multi-arch

View File

@@ -67,6 +67,12 @@ jobs:
echo 'VITE_SENTRY_AUTH_TOKEN="${{ secrets.SENTRY_AUTH_TOKEN }}"' >> frontend/.env
echo 'VITE_SENTRY_ORG="${{ secrets.SENTRY_ORG }}"' >> frontend/.env
echo 'VITE_SENTRY_PROJECT_ID="${{ secrets.SENTRY_PROJECT_ID }}"' >> frontend/.env
echo 'VITE_SENTRY_DSN="${{ secrets.SENTRY_DSN }}"' >> frontend/.env
echo 'VITE_TUNNEL_URL="${{ secrets.NP_TUNNEL_URL }}"' >> frontend/.env
echo 'VITE_TUNNEL_DOMAIN="${{ secrets.NP_TUNNEL_DOMAIN }}"' >> frontend/.env
echo 'VITE_PYLON_APP_ID="${{ secrets.NP_PYLON_APP_ID }}"' >> frontend/.env
echo 'VITE_APPCUES_APP_ID="${{ secrets.NP_APPCUES_APP_ID }}"' >> frontend/.env
echo 'VITE_PYLON_IDENTITY_SECRET="${{ secrets.NP_PYLON_IDENTITY_SECRET }}"' >> frontend/.env
echo 'VITE_DOCS_BASE_URL="https://staging.signoz.io"' >> frontend/.env
echo 'VITE_ENVIRONMENT="staging"' >> frontend/.env
echo 'VITE_VERSION="${{ steps.build-info.outputs.version }}"' >> frontend/.env
@@ -109,6 +115,7 @@ jobs:
-X github.com/SigNoz/signoz/pkg/version.branch=${{ needs.prepare.outputs.branch }}
-X github.com/SigNoz/signoz/ee/zeus.url=https://api.staging.signoz.cloud
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=https://license.staging.signoz.cloud
-X github.com/SigNoz/signoz/ee/query-service/constants.LicenseSignozIo=https://license.staging.signoz.cloud/api/v1
-X github.com/SigNoz/signoz/pkg/analytics.key=9kRrJ7oPCGPEJLF6QjMPLt5bljFhRQBr'
DOCKER_BASE_IMAGES: '{"alpine": "alpine:3.20.3"}'
DOCKER_DOCKERFILE_PATH: ./cmd/enterprise/Dockerfile.multi-arch

View File

@@ -1,92 +0,0 @@
name: cacheci
on:
push:
branches:
- main
workflow_dispatch:
permissions:
contents: read
actions: write
# Cancelling mid-rotation is safe: the sequential delete-then-save order
# leaves at most one key missing at any moment.
concurrency:
group: cacheci
cancel-in-progress: true
jobs:
tests:
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v4
- name: restore
id: restore
uses: actions/cache/restore@v4
with:
path: ${{ runner.temp }}/cacheci
key: tests-primary
restore-keys: |
tests-secondary
- name: inject
if: steps.restore.outputs.cache-matched-key != ''
run: |
cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
FROM busybox:1.37
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/pnpm/store \
--mount=type=bind,target=/restored \
tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
tar -xf /restored/pnpm-store.tar -C /pnpm/store
EOF
docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
- name: build
run: |
docker build -f cmd/enterprise/Dockerfile.integration --build-arg TARGETARCH=amd64 --build-arg ZEUSURL=http://zeus:8080 .
docker build -f cmd/enterprise/Dockerfile.with-web.integration --build-arg TARGETARCH=amd64 --build-arg ZEUSURL=http://zeus:8080 .
# docker cp instead of --output type=local (the local exporter stalls on
# multi-GB outputs); tarballs instead of raw trees so the host never hits
# the permission and symlink semantics that broke docker cp.
- name: extract
run: |
rm -rf "$RUNNER_TEMP/cacheci"
mkdir -p "$RUNNER_TEMP/cacheci" "$RUNNER_TEMP/extract-context"
cat > "$RUNNER_TEMP/extract.Dockerfile" <<'EOF'
FROM busybox:1.37
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/pnpm/store \
mkdir -p /out && \
tar -cf /out/go-build.tar -C /root/.cache/go-build . && \
tar -cf /out/go-mod.tar -C /go/pkg/mod . && \
tar -cf /out/pnpm-store.tar -C /pnpm/store .
EOF
docker build -f "$RUNNER_TEMP/extract.Dockerfile" -t cacheci-extract "$RUNNER_TEMP/extract-context"
id=$(docker create cacheci-extract)
docker cp "$id":/out/. "$RUNNER_TEMP/cacheci/"
docker rm "$id"
# Fixed cache keys are immutable, so each key must be deleted before it
# can be saved again. Rotating primary and secondary one after the other
# keeps at least one key restorable for concurrent test runs.
- name: delete-primary
env:
GH_TOKEN: ${{ github.token }}
run: gh cache delete tests-primary --repo "$GITHUB_REPOSITORY" || true
- name: save-primary
uses: actions/cache/save@v4
with:
path: ${{ runner.temp }}/cacheci
key: tests-primary
- name: delete-secondary
env:
GH_TOKEN: ${{ github.token }}
run: gh cache delete tests-secondary --repo "$GITHUB_REPOSITORY" || true
- name: save-secondary
uses: actions/cache/save@v4
with:
path: ${{ runner.temp }}/cacheci
key: tests-secondary

83
.github/workflows/docs.yml vendored Normal file
View File

@@ -0,0 +1,83 @@
name: "Update PR labels and Block PR until related docs are shipped for the feature"
on:
pull_request:
branches:
- main
types: [opened, edited, labeled, unlabeled]
permissions:
pull-requests: write
contents: read
jobs:
docs_label_check:
runs-on: ubuntu-latest
steps:
- name: Check PR Title and Manage Labels
uses: actions/github-script@v6
with:
script: |
const prTitle = context.payload.pull_request.title;
const prNumber = context.payload.pull_request.number;
const owner = context.repo.owner;
const repo = context.repo.repo;
// Fetch the current PR details to get labels
const pr = await github.rest.pulls.get({
owner,
repo,
pull_number: prNumber
});
const labels = pr.data.labels.map(label => label.name);
if (prTitle.startsWith('feat:')) {
const hasDocsRequired = labels.includes('docs required');
const hasDocsShipped = labels.includes('docs shipped');
const hasDocsNotRequired = labels.includes('docs not required');
// If "docs not required" is present, skip the checks
if (hasDocsNotRequired && !hasDocsRequired) {
console.log("Skipping checks due to 'docs not required' label.");
return; // Exit the script early
}
// If "docs shipped" is present, remove "docs required" if it exists
if (hasDocsShipped && hasDocsRequired) {
await github.rest.issues.removeLabel({
owner,
repo,
issue_number: prNumber,
name: 'docs required'
});
console.log("Removed 'docs required' label.");
}
// Add "docs required" label if neither "docs shipped" nor "docs required" are present
if (!hasDocsRequired && !hasDocsShipped) {
await github.rest.issues.addLabels({
owner,
repo,
issue_number: prNumber,
labels: ['docs required']
});
console.log("Added 'docs required' label.");
}
}
// Fetch the updated labels after any changes
const updatedPr = await github.rest.pulls.get({
owner,
repo,
pull_number: prNumber
});
const updatedLabels = updatedPr.data.labels.map(label => label.name);
const updatedHasDocsRequired = updatedLabels.includes('docs required');
const updatedHasDocsShipped = updatedLabels.includes('docs shipped');
// Block PR if "docs required" is still present and "docs shipped" is missing
if (updatedHasDocsRequired && !updatedHasDocsShipped) {
core.setFailed("This PR requires documentation. Please remove the 'docs required' label and add the 'docs shipped' label to proceed.");
}

View File

@@ -75,30 +75,6 @@ jobs:
docker rm pw
echo "PLAYWRIGHT_BROWSERS_PATH=$RUNNER_TEMP/ms-playwright" >> "$GITHUB_ENV"
cd tests/e2e && pnpm playwright install-deps ${{ matrix.project }}
# Restore-only: the cacheci workflow owns cache saves. Seeds the
# BuildKit cache mounts so the in-test image build is incremental.
- name: restore
id: restore
uses: actions/cache/restore@v4
with:
path: ${{ runner.temp }}/cacheci
key: tests-primary
restore-keys: |
tests-secondary
- name: inject
if: steps.restore.outputs.cache-matched-key != ''
run: |
cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
FROM busybox:1.37
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/pnpm/store \
--mount=type=bind,target=/restored \
tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
tar -xf /restored/pnpm-store.tar -C /pnpm/store
EOF
docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
- name: bring-up-stack
run: |
cd tests && \

View File

@@ -53,21 +53,6 @@ jobs:
with:
PRIMUS_REF: main
GO_VERSION: 1.24
semconv-generated:
if: |
github.event_name == 'merge_group' ||
(github.event_name == 'pull_request' && ! github.event.pull_request.head.repo.fork && github.event.pull_request.user.login != 'dependabot[bot]' && ! contains(github.event.pull_request.labels.*.name, 'safe-to-test')) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test'))
runs-on: ubuntu-latest
steps:
- name: self-checkout
uses: actions/checkout@v4
- name: go-install
uses: actions/setup-go@v5
with:
go-version: "1.24"
- name: check-semconv-generated-files
run: go run ./scripts/semconv -check
build:
if: |
github.event_name == 'merge_group' ||

View File

@@ -27,6 +27,13 @@ jobs:
echo 'VITE_SENTRY_AUTH_TOKEN="${{ secrets.SENTRY_AUTH_TOKEN }}"' >> .env
echo 'VITE_SENTRY_ORG="${{ secrets.SENTRY_ORG }}"' >> .env
echo 'VITE_SENTRY_PROJECT_ID="${{ secrets.SENTRY_PROJECT_ID }}"' >> .env
echo 'VITE_SENTRY_DSN="${{ secrets.SENTRY_DSN }}"' >> .env
echo 'VITE_TUNNEL_URL="${{ secrets.TUNNEL_URL }}"' >> .env
echo 'VITE_TUNNEL_DOMAIN="${{ secrets.TUNNEL_DOMAIN }}"' >> .env
echo 'VITE_POSTHOG_KEY="${{ secrets.POSTHOG_KEY }}"' >> .env
echo 'VITE_PYLON_APP_ID="${{ secrets.PYLON_APP_ID }}"' >> .env
echo 'VITE_APPCUES_APP_ID="${{ secrets.APPCUES_APP_ID }}"' >> .env
echo 'VITE_PYLON_IDENTITY_SECRET="${{ secrets.PYLON_IDENTITY_SECRET }}"' >> .env
echo 'VITE_DOCS_BASE_URL="https://signoz.io"' >> .env
echo 'VITE_ENVIRONMENT="production"' >> .env
echo 'VITE_VERSION="${{ github.ref_name }}"' >> .env

View File

@@ -39,8 +39,6 @@ jobs:
matrix:
suite:
- alerts
- alertmanager
- alertmanagerrotation
- basepath
- callbackauthn
- cloudintegrations
@@ -50,37 +48,24 @@ jobs:
- logspipelines
- passwordauthn
- preference
- quickfilter
- querierlogs
- queriertraces
- queriermetrics
- querierscalar
- queriercommon
- querierai
- querier
- rawexportdata
- promqlconformance
- promapiconformance
- querierauthz
- role
- rootuser
- savedview
- semconvfamilies
- serviceaccount
- spanmapper
- querier_json_body
- querier_skip_resource_fingerprint
- ttl
- clickhousecluster
- metricreduction
sqlstore-provider:
- postgres
- sqlite
sqlite-mode:
- wal
clickhouse-version:
- 25.5.6
- 25.12.5
schema-migrator-version:
- v0.144.6
- v0.144.3
postgres-version:
- 15
if: |
@@ -117,30 +102,6 @@ jobs:
sudo mv chromedriver-linux64/chromedriver /usr/local/bin/chromedriver
chromedriver -version
google-chrome-stable --version
# Restore-only: the cacheci workflow owns cache saves. Seeds the
# BuildKit cache mounts so the in-test image build is incremental.
- name: restore
id: restore
uses: actions/cache/restore@v4
with:
path: ${{ runner.temp }}/cacheci
key: tests-primary
restore-keys: |
tests-secondary
- name: inject
if: steps.restore.outputs.cache-matched-key != ''
run: |
cat > "$RUNNER_TEMP/inject.Dockerfile" <<'EOF'
FROM busybox:1.37
RUN --mount=type=cache,target=/root/.cache/go-build \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/pnpm/store \
--mount=type=bind,target=/restored \
tar -xf /restored/go-build.tar -C /root/.cache/go-build && \
tar -xf /restored/go-mod.tar -C /go/pkg/mod && \
tar -xf /restored/pnpm-store.tar -C /pnpm/store
EOF
docker build -f "$RUNNER_TEMP/inject.Dockerfile" "$RUNNER_TEMP/cacheci"
- name: run
run: |
cd tests && \

6
.gitignore vendored
View File

@@ -90,6 +90,8 @@ queries.active
.devenv/**/tmp/**
.qodo
.dev
### Python ###
# Byte-compiled / optimized / DLL files
__pycache__/
@@ -229,6 +231,4 @@ cython_debug/
# LSP config files
pyrightconfig.json
# dev
.dev/
.claude/worktrees/

View File

@@ -209,8 +209,8 @@ py-lint: ## Run ruff check across the shared tests project
@cd tests && uv run ruff check --fix .
.PHONY: py-test-setup
py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests, rebuilding signoz from the current sources
@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup
py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests
@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --capture=no integration/bootstrap/setup.py::test_setup
.PHONY: py-test-teardown
py-test-teardown: ## Tear down the shared SigNoz backend
@@ -233,10 +233,6 @@ py-clean: ## Clear all pycache and pytest cache from tests directory recursively
##############################################################
# generate commands
##############################################################
.PHONY: semconv-generate
semconv-generate: ## Regenerate semantic-convention families for Go and TypeScript
@go run ./scripts/semconv
.PHONY: gen-mocks
gen-mocks:
@echo ">> Generating mocks"

View File

@@ -1,190 +1,197 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/readme-assets/signoz-hero-dark.png" width="900">
<source media="(prefers-color-scheme: light)" srcset="docs/readme-assets/signoz-hero-light.png" width="900">
<img alt="SigNoz - Observability nach deinen Bedingungen, basierend auf offenen Standards." src="docs/readme-assets/signoz-hero-light.png" width="900">
</picture>
<img src="https://res.cloudinary.com/dcv3epinx/image/upload/v1618904450/signoz-images/LogoGithub_sigfbu.svg" alt="SigNoz-logo" width="240" />
<p align="center">Überwache deine Anwendungen und behebe Probleme in deinen bereitgestellten Anwendungen. SigNoz ist eine Open Source Alternative zu DataDog, New Relic, etc.</p>
</p>
<p align="center">
<a href="README.md">English</a> ·
<a href="README.zh-cn.md">中文</a> ·
<a href="README.pt-br.md">Português</a>
<img alt="Downloads" src="https://img.shields.io/docker/pulls/signoz/query-service?label=Downloads"> </a>
<img alt="GitHub issues" src="https://img.shields.io/github/issues/signoz/signoz"> </a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability">
<img alt="tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"> </a>
</p>
<p align="center">
<a href="https://github.com/SigNoz/signoz/issues"><img alt="GitHub issues" src="https://img.shields.io/github/issues/SigNoz/signoz"></a>
<a href="https://github.com/SigNoz/signoz/releases"><img alt="GitHub release" src="https://img.shields.io/github/v/release/SigNoz/signoz?label=release"></a>
<a href="https://signoz.io/slack"><img alt="Slack community" src="https://img.shields.io/badge/slack-community-4A154B?logo=slack&logoColor=white"></a>
<a href="https://www.linkedin.com/company/signozio/"><img alt="LinkedIn" src="https://img.shields.io/badge/linkedin-SigNoz-0A66C2?logo=linkedin&logoColor=white"></a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability"><img alt="Tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"></a>
</p>
<h3 align="center">
<a href="https://signoz.io/docs"><b>Dokumentation</b></a> &bull;
<a href="https://github.com/SigNoz/signoz/blob/main/README.md"><b>Readme auf Englisch </b></a> &bull;
<a href="https://github.com/SigNoz/signoz/blob/main/README.zh-cn.md"><b>ReadMe auf Chinesisch</b></a> &bull;
<a href="https://github.com/SigNoz/signoz/blob/main/README.pt-br.md"><b>ReadMe auf Portugiesisch</b></a> &bull;
<a href="https://signoz.io/slack"><b>Slack Community</b></a> &bull;
<a href="https://twitter.com/SigNozHq"><b>Twitter</b></a>
</h3>
SigNoz ist eine Open-Source-Observability-Plattform auf Basis von OpenTelemetry. Wir bauen eine Enterprise-taugliche Alternative zu fragmentierten Monitoring-Stacks, mit Logs, Metriken, Traces, Alerts und Dashboards an einem Ort.
##
### Wähle, wie du SigNoz betreibst
SigNoz hilft Entwicklern, Anwendungen zu überwachen und Probleme in ihren bereitgestellten Anwendungen zu beheben. Mit SigNoz können Sie Folgendes tun:
#### SigNoz Cloud (empfohlen)
👉 Visualisieren Sie Metriken, Traces und Logs in einer einzigen Oberfläche.
Vollständig verwaltetes SigNoz mit 30 Tagen kostenloser Testphase, ohne Kreditkarte, nutzungsbasierter Preisgestaltung ab 49 USD und regionalem Datenhosting.
👉 Sie können Metriken wie die p99-Latenz, Fehlerquoten für Ihre Dienste, externe API-Aufrufe und individuelle Endpunkte anzeigen.
[**Kostenlos starten →**](https://signoz.io/teams/)
👉 Sie können die Ursache des Problems ermitteln, indem Sie zu den genauen Traces gehen, die das Problem verursachen, und detaillierte Flammenbilder einzelner Anfragetraces anzeigen.
#### Enterprise
👉 Führen Sie Aggregationen auf Trace-Daten durch, um geschäftsrelevante Metriken zu erhalten.
Enterprise Cloud, BYOC oder Enterprise Self-Hosted mit Compliance, Support, benutzerdefinierter Aufbewahrung, RBAC, Ingestion Controls, Datenresidenz und Regionsauswahl.
👉 Filtern und Abfragen von Logs, Erstellen von Dashboards und Benachrichtigungen basierend auf Attributen in den Logs.
[**Enterprise entdecken →**](https://signoz.io/enterprise/)
👉 Automatische Aufzeichnung von Ausnahmen in Python, Java, Ruby und Javascript.
#### Community
👉 Einfache Einrichtung von Benachrichtigungen mit dem selbst erstellbaren Abfrage-Builder.
Kostenloses Open-Source-SigNoz, das in deiner eigenen Infrastruktur läuft. Deployment mit Docker, Kubernetes oder Linux, während du die volle Kontrolle über deine Datenebene behältst.
##
[**SigNoz installieren →**](https://signoz.io/docs/install/self-host/)
### Anwendung Metriken
### Was kannst du überwachen?
![application_metrics](https://user-images.githubusercontent.com/83692067/226637410-900dbc5e-6705-4b11-a10c-bd0faeb2a92f.png)
SigNoz hilft Teams, Produktionsprobleme schneller zu debuggen, indem Logs, Metriken, Traces, Alerts, Dashboards, Exceptions und agent-native Workflows an einem Ort verbunden werden.
### Verteiltes Tracing
#### APM-Überblick
<img width="2068" alt="distributed_tracing_2 2" src="https://user-images.githubusercontent.com/83692067/226536447-bae58321-6a22-4ed3-af80-e3e964cb3489.png">
Überwache Service-Latenz, Fehlerrate, Durchsatz, Apdex, wichtige Endpunkte, Datenbankaufrufe und externe Aufrufe.
<img width="2068" alt="distributed_tracing_1" src="https://user-images.githubusercontent.com/83692067/226536462-939745b6-4f9d-45a6-8016-814837e7f7b4.png">
<p align="center">
<img alt="SigNoz APM-Dashboard mit Latenz, Durchsatz, Apdex und wichtigen Operationen" src="docs/readme-assets/monitor/apm.png" width="900">
</p>
### Log Verwaltung
Mehr erfahren: [APM-Dokumentation](https://signoz.io/docs/instrumentation/overview/)
<img width="2068" alt="logs_management" src="https://user-images.githubusercontent.com/83692067/226536482-b8a5c4af-b69c-43d5-969c-338bd5eaf1a5.png">
#### Log-Management
### Infrastruktur Überwachung
Erfasse, suche, aggregiere und korreliere Logs mit Traces und Metriken über einen visuellen Query Builder.
<img width="2068" alt="infrastructure_monitoring" src="https://user-images.githubusercontent.com/83692067/226536496-f38c4dbf-e03c-4158-8be0-32d4a61158c7.png">
<p align="center">
<img alt="SigNoz Logs Explorer mit Filtern, Frequenzdiagramm und Log-Zeilen" src="docs/readme-assets/monitor/log-management.svg" width="900">
</p>
### Exceptions Monitoring
Mehr erfahren: [Log-Management-Dokumentation](https://signoz.io/docs/logs-management/overview/)
![exceptions_light](https://user-images.githubusercontent.com/83692067/226637967-4188d024-3ac9-4799-be95-f5ea9c45436f.png)
#### Metriken und Dashboards
### Alarme
Erstelle Dashboards für Anwendungs-, Infrastruktur- und benutzerdefinierte Metriken mit Query Builder, PromQL oder ClickHouse SQL.
<img width="2068" alt="alerts_management" src="https://user-images.githubusercontent.com/83692067/226536548-2c81e2e8-c12d-47e8-bad7-c6be79055def.png">
<p align="center">
<img alt="SigNoz Host-Metrics-Dashboard mit Systemlast- und Netzwerkdiagrammen" src="docs/readme-assets/monitor/metrics.png" width="900">
</p>
<br /><br />
Mehr erfahren: [Metriken-Dokumentation](https://signoz.io/docs/metrics-management/overview/)
## Werde Teil unserer Slack Community
#### Infrastruktur-Monitoring
Sag Hi zu uns auf [Slack](https://signoz.io/slack) 👋
Überwache Kubernetes-Cluster, Pods, Nodes, Workloads sowie Host-CPU, Arbeitsspeicher, Festplatten, Netzwerk, Logs und Traces.
<br /><br />
<p align="center">
<img alt="SigNoz Kubernetes-Infrastruktur-Dashboard mit Pod- und Node-Metriken" src="docs/readme-assets/monitor/infrastructure.png" width="900">
</p>
## Funktionen:
Mehr erfahren: [Infrastruktur-Monitoring-Dokumentation](https://signoz.io/docs/infrastructure-monitoring/overview/)
- Einheitliche Benutzeroberfläche für Metriken, Traces und Logs. Keine Notwendigkeit, zwischen Prometheus und Jaeger zu wechseln, um Probleme zu debuggen oder ein separates Log-Tool wie Elastic neben Ihrer Metriken- und Traces-Stack zu verwenden.
- Überblick über Anwendungsmetriken wie RPS, Latenzzeiten des 50tes/90tes/99tes Perzentils und Fehlerquoten.
- Langsamste Endpunkte in Ihrer Anwendung.
- Zeigen Sie genaue Anfragetraces an, um Probleme in nachgelagerten Diensten, langsamen Datenbankabfragen oder Aufrufen von Drittanbieterdiensten wie Zahlungsgateways zu identifizieren.
- Filtern Sie Traces nach Dienstname, Operation, Latenz, Fehler, Tags/Annotationen.
- Führen Sie Aggregationen auf Trace-Daten (Ereignisse/Spans) durch, um geschäftsrelevante Metriken zu erhalten. Beispielsweise können Sie die Fehlerquote und die 99tes Perzentillatenz für `customer_type: gold` oder `deployment_version: v2` oder `external_call: paypal` erhalten.
- Native Unterstützung für OpenTelemetry-Logs, erweiterten Log-Abfrage-Builder und automatische Log-Sammlung aus dem Kubernetes-Cluster.
- Blitzschnelle Log-Analytik ([Logs Perf. Benchmark](https://signoz.io/blog/logs-performance-benchmark/))
- End-to-End-Sichtbarkeit der Infrastrukturleistung, Aufnahme von Metriken aus allen Arten von Host-Umgebungen.
- Einfache Einrichtung von Benachrichtigungen mit dem selbst erstellbaren Abfrage-Builder.
#### LLM- und AI-Observability
<br /><br />
Verfolge LLM-Apps, RAG-Pipelines, Prompts, Tool Calls, Tokens, Latenz und Kosten zusammen mit Anwendungs- und Infrastruktur-Telemetrie.
## Wieso SigNoz?
<p align="center">
<img alt="SigNoz LLM-Observability-Dashboard für Traces, Token-Nutzung, Latenz und Kosten" src="docs/readme-assets/monitor/llm.png" width="900">
</p>
Als Entwickler fanden wir es anstrengend, uns für jede kleine Funktion, die wir haben wollten, auf Closed Source SaaS Anbieter verlassen zu müssen. Closed Source Anbieter überraschen ihre Kunden zum Monatsende oft mit hohen Rechnungen, die keine Transparenz bzgl. der Kostenaufteilung bieten.
Mehr erfahren: [LLM-Observability-Dokumentation](https://signoz.io/docs/llm-observability/)
Wir wollten eine selbst gehostete, Open Source Variante von Lösungen wie DataDog, NewRelic für Firmen anbieten, die Datenschutz und Sicherheitsbedenken haben, bei der Weitergabe von Kundendaten an Drittanbieter.
#### Agent-Native Observability und MCP
Open Source gibt dir außerdem die totale Kontrolle über deine Konfiguration, Stichprobenentnahme und Betriebszeit. Du kannst des Weiteren neue Module auf Basis von SigNoz bauen, die erweiterte, geschäftsspezifische Funktionen anbieten.
Nutze den SigNoz MCP-Server, um Telemetrie in Coding Agents zu bringen, oder nutze Noz in SigNoz, um Incidents zu untersuchen, Alerts zu verbessern und Dashboards mit Produktionskontext zu erstellen. [Noz](https://signoz.io/docs/ai/noz/) ist nur in SigNoz Cloud verfügbar.
### Languages supported:
<p align="center">
<img alt="SigNoz Noz-Oberfläche neben einem MCP-gestützten Agent-Workflow" src="docs/readme-assets/monitor/agent-native.png" width="900">
</p>
Wir unterstützen [OpenTelemetry](https://opentelemetry.io) als Bibliothek, mit der Sie Ihre Anwendungen instrumentieren können. Daher wird jedes von OpenTelemetry unterstützte Framework und jede Sprache auch von SignNoz unterstützt. Einige der wichtigsten unterstützten Sprachen sind:
Mehr erfahren: [SigNoz MCP-Server-Dokumentation](https://signoz.io/docs/ai/signoz-mcp-server/) · [Agent-Skills-Dokumentation](https://signoz.io/docs/ai/agent-skills/#install-the-plugin)
- Java
- Python
- NodeJS
- Go
- PHP
- .NET
- Ruby
- Elixir
- Rust
#### Distributed Tracing
Hier findest du die vollständige Liste von unterstützten Programmiersprachen - https://opentelemetry.io/docs/
Verfolge Requests über Services hinweg mit Flamegraphs, Waterfalls, Span Events, Filtern und Trace Analytics.
<br /><br />
<p align="center">
<img alt="SigNoz Distributed-Tracing-Ansicht mit Flamegraph und Waterfall-Spans" src="docs/readme-assets/monitor/distributed-tracing.png" width="900">
</p>
## Erste Schritte mit SigNoz
Mehr erfahren: [Distributed-Tracing-Dokumentation](https://signoz.io/docs/instrumentation/)
### Bereitstellung mit Docker
#### Trace Funnels
Bitte folge den [hier](https://signoz.io/docs/install/docker/) aufgelisteten Schritten um deine Anwendung mit Docker bereitzustellen.
Erstelle Funnels aus Traces, um Drop-offs im Request-Flow, fehlgeschlagene Übergänge und systemische Workflow-Probleme zu verstehen.
Die [Anleitungen zur Fehlerbehebung](https://signoz.io/docs/install/troubleshooting/) könnten hilfreich sein, falls du auf irgendwelche Schwierigkeiten stößt.
<p align="center">
<img alt="SigNoz Trace Funnels mit Request-Flow-Drop-offs und fehlgeschlagenen Übergängen" src="docs/readme-assets/monitor/trace-funnels.png" width="900">
</p>
<p>&nbsp </p>
Mehr erfahren: [Trace-Funnels-Dokumentation](https://signoz.io/docs/trace-funnels/overview/)
### Deploy in Kubernetes using Helm
Du kannst außerdem [**Exceptions**](https://signoz.io/docs/userguide/exceptions/), [**Alerts**](https://signoz.io/docs/alerts/), [**externe APIs**](https://signoz.io/docs/external-api-monitoring/overview/) und [**Integrationen**](https://signoz.io/docs/integrations/integrations-list/) für OpenTelemetry, Prometheus, Kubernetes, Cloud-Anbieter, Sprach-SDKs, Application Frameworks, Datenbanken und LLM-Tools überwachen.
Bitte folge den [hier](https://signoz.io/docs/deployment/helm_chart) aufgelisteten Schritten, um deine Anwendung mit Helm Charts bereitzustellen.
### Warum Teams SigNoz nutzen
<br /><br />
1. **OpenTelemetry-native**<br>
Einmal mit offenen Standards instrumentieren und die Kontrolle über deine Telemetrie behalten.
2. **Korrelierte Signale**<br>
Von Service-Charts zu Traces, Logs, Infrastrukturmetriken und Exceptions wechseln, ohne das Tool zu wechseln.
3. **Eine einzelne spaltenorientierte Datenbank**<br>
Gebaut für hochkardinale Observability-Workloads mit hohem Volumen.
4. **Vorhersehbare Preise**<br>
Keine Preise pro Host, keine Preise pro Nutzerplatz und keine Sonderpreise für Custom Metrics.
5. **Enterprise-ready**<br>
SOC 2 Type II und HIPAA Compliance, RBAC, Ingestion Controls, benutzerdefinierte Aufbewahrung, Support, BYOC und Self-Hosting.
## Vergleiche mit bekannten Tools
### Erste Schritte
### SigNoz vs Prometheus
#### Mit Cloud starten
Prometheus ist gut, falls du dich nur für Metriken interessierst. Wenn du eine nahtlose Integration von Metriken und Einzelschritt-Fehlersuchen haben möchtest, ist die Kombination aus Prometheus und Jaeger nicht das Richtige für dich.
Erstelle einen verwalteten SigNoz-Workspace und erhalte dein erstes Dashboard, ohne Observability-Infrastruktur betreiben zu müssen.
Unser Ziel ist es, eine integrierte Benutzeroberfläche aus Metriken und Einzelschritt-Fehlersuchen anzubieten, ähnlich wie es SaaS Anbieter wie Datadog tun, mit der Möglichkeit von erweitertem filtern und aggregieren von Fehlersuchen. Etwas, was in Jaeger aktuell fehlt.
[**Kostenlos mit SigNoz Cloud starten**](https://signoz.io/teams/)
<p>&nbsp </p>
#### SigNoz selbst hosten
### SigNoz vs Jaeger
Betreibe SigNoz in deiner eigenen Infrastruktur mit Foundry, Docker, Kubernetes oder Linux.
Jaeger kümmert sich nur um verteilte Einzelschritt-Fehlersuche. SigNoz erstellt sowohl Metriken als auch Einzelschritt-Fehlersuche, daneben haben wir auch Protokoll Verwaltung auf unserem Plan.
[**Foundry**](https://github.com/SigNoz/foundry) · [**Docker**](https://signoz.io/docs/install/docker/) · [**Kubernetes**](https://signoz.io/docs/install/kubernetes/) · [**Linux**](https://signoz.io/docs/install/linux/)
Außerdem hat SigNoz noch mehr spezielle Funktionen im Vergleich zu Jaeger:
#### Daten senden
- Jaeger UI zeigt keine Metriken für Einzelschritt-Fehlersuchen oder für gefilterte Einzelschritt-Fehlersuchen an.
- Jaeger erstellt keine Aggregate für gefilterte Einzelschritt-Fehlersuchen, z. B. die P99 Latenz von Abfragen mit dem Tag `customer_type=premium`, was hingegen mit SigNoz leicht umsetzbar ist.
Instrumentiere Anwendungen und Infrastruktur mit OpenTelemetry, Prometheus, Sprach-SDKs und Integrationen.
<p>&nbsp </p>
[**Instrumentation**](https://signoz.io/docs/instrumentation/) · [**Integrationen**](https://signoz.io/docs/integrations/integrations-list/)
### SigNoz vs Elastic
### Vergleich mit bekannten Tools
- Die Verwaltung von SigNoz-Protokollen basiert auf 'ClickHouse', einem spaltenbasierten OLAP-Datenspeicher, der aggregierte Protokollanalyseabfragen wesentlich effizienter macht.
- 50 % geringerer Ressourcenbedarf im Vergleich zu Elastic während der Aufnahme.
SigNoz wird häufig von Teams eingeführt, die von einzelnen Spezialtools oder kommerziellen Plattformen mit unvorhersehbarer Preisgestaltung wechseln.
Wir haben Benchmarks veröffentlicht, die Elastic mit SignNoz vergleichen. Schauen Sie es sich [hier](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)
**Prometheus**<br>
Gut, wenn du nur Metriken brauchst. SigNoz hält Metriken, Logs, Traces, Dashboards und Alerts zusammen, damit Teams mit korreliertem Kontext debuggen können.
<p>&nbsp </p>
**Jaeger**<br>
Jaeger macht ausschließlich Distributed Tracing. SigNoz ergänzt Metriken, Logs, Trace Analytics, Dashboards, Alerts, Exceptions und Trace-to-Log-Workflows.
### SigNoz vs Loki
**Elastic**<br>
SigNoz nutzt eine spaltenorientierte Datenbank für effiziente Observability-Analysen und hochkardinale Log-Workloads, mit 50 % geringerem Ressourcenbedarf gegenüber Elastic während der Ingestion. Lies die [detaillierte Studie](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark).
- SigNoz unterstützt Aggregationen von Daten mit hoher Kardinalität über ein großes Volumen, Loki hingegen nicht.
- SigNoz unterstützt Indizes über Daten mit hoher Kardinalität und hat keine Beschränkungen hinsichtlich der Anzahl der Indizes, während Loki maximale Streams erreicht, wenn ein paar Indizes hinzugefügt werden.
- Das Durchsuchen großer Datenmengen ist in Loki im Vergleich zu SigNoz schwierig und langsam.
**Loki**<br>
Im verlinkten Benchmark indexierte SigNoz alle Keys im Test-Setup, während Loki beim Hinzufügen weiterer Labels Max-Stream-Fehler erreichte. Lies die [detaillierte Studie](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark).
Wir haben Benchmarks veröffentlicht, die Loki mit SigNoz vergleichen. Schauen Sie es sich [hier](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)
## Mitwirken
<br /><br />
Wir freuen uns über große und kleine Beiträge. Lies bitte [CONTRIBUTING.md](CONTRIBUTING.md), um mit Beiträgen zu SigNoz loszulegen.
## Zum Projekt beitragen
Nicht sicher, wie du anfangen sollst? **Schreib uns einfach im Channel `#contributing` in unserer [Slack Community](https://signoz.io/slack).**
Wir ❤️ Beiträge zum Projekt, egal ob große oder kleine. Bitte lies dir zuerst die [CONTRIBUTING.md](CONTRIBUTING.md), durch, bevor du anfängst, Beiträge zu SigNoz zu machen.
Du bist dir nicht sicher, wie du anfangen sollst? Schreib uns einfach auf dem #contributing Kanal in unserer [slack community](https://signoz.io/slack)
Wie immer: Danke an unsere großartigen Contributors!
<br /><br />
## Dokumentation
Du findest unsere Dokumentation unter https://signoz.io/docs/. Falls etwas unverständlich ist oder fehlt, öffne gerne ein Github Issue mit dem Label `documentation` oder schreib uns über den Community Slack Channel.
<br /><br />
## Gemeinschaft
Werde Teil der [slack community](https://signoz.io/slack) um mehr über verteilte Einzelschritt-Fehlersuche, Messung von Systemzuständen oder SigNoz zu erfahren und sich mit anderen Nutzern und Mitwirkenden in Verbindung zu setzen.
Falls du irgendwelche Ideen, Fragen oder Feedback hast, kannst du sie gerne über unsere [Github Discussions](https://github.com/SigNoz/signoz/discussions) mit uns teilen.
Wie immer, Dank an unsere großartigen Mitwirkenden!
<a href="https://github.com/signoz/signoz/graphs/contributors">
<img alt="SigNoz Contributors" src="https://contrib.rocks/image?repo=signoz/signoz" />
<img src="https://contrib.rocks/image?repo=signoz/signoz" />
</a>

278
README.md
View File

@@ -1,190 +1,244 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/readme-assets/signoz-hero-dark.png" width="900">
<source media="(prefers-color-scheme: light)" srcset="docs/readme-assets/signoz-hero-light.png" width="900">
<img alt="SigNoz - Observability on Your Terms, Powered by Open Standards." src="docs/readme-assets/signoz-hero-light.png" width="900">
</picture>
</p>
<h1 align="center" style="border-bottom: none">
<a href="https://signoz.io" target="_blank">
<img alt="SigNoz" src="https://github.com/user-attachments/assets/ef9a33f7-12d7-4c94-8908-0a02b22f0c18" width="100" height="100">
</a>
<br>SigNoz
</h1>
<p align="center">All your logs, metrics, and traces in one place. Monitor your application, spot issues before they occur and troubleshoot downtime quickly with rich context. SigNoz is a cost-effective open-source alternative to Datadog and New Relic. Visit <a href="https://signoz.io" target="_blank">signoz.io</a> for the full documentation, tutorials, and guide.</p>
<p align="center">
<a href="README.zh-cn.md">中文</a> ·
<a href="README.de-de.md">Deutsch</a> ·
<a href="README.pt-br.md">Português</a>
<img alt="GitHub issues" src="https://img.shields.io/github/issues/signoz/signoz"> </a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability">
<img alt="tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"> </a>
</p>
<h3 align="center">
<a href="https://signoz.io/docs"><b>Documentation</b></a> &bull;
<a href="https://github.com/SigNoz/signoz/blob/main/README.zh-cn.md"><b>ReadMe in Chinese</b></a> &bull;
<a href="https://github.com/SigNoz/signoz/blob/main/README.de-de.md"><b>ReadMe in German</b></a> &bull;
<a href="https://github.com/SigNoz/signoz/blob/main/README.pt-br.md"><b>ReadMe in Portuguese</b></a> &bull;
<a href="https://signoz.io/slack"><b>Slack Community</b></a> &bull;
<a href="https://twitter.com/SigNozHq"><b>Twitter</b></a>
</h3>
<p align="center">
<a href="https://github.com/SigNoz/signoz/issues"><img alt="GitHub issues" src="https://img.shields.io/github/issues/SigNoz/signoz"></a>
<a href="https://github.com/SigNoz/signoz/releases"><img alt="GitHub release" src="https://img.shields.io/github/v/release/SigNoz/signoz?label=release"></a>
<a href="https://signoz.io/slack"><img alt="Slack community" src="https://img.shields.io/badge/slack-community-4A154B?logo=slack&logoColor=white"></a>
<a href="https://www.linkedin.com/company/signozio/"><img alt="LinkedIn" src="https://img.shields.io/badge/linkedin-SigNoz-0A66C2?logo=linkedin&logoColor=white"></a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability"><img alt="Tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"></a>
</p>
## Features
SigNoz is an open-source observability platform built on OpenTelemetry. Were building an enterprise-grade alternative to fragmented monitoring stacks, with logs, metrics, traces, alerts, and dashboards in one place.
### Choose how to run SigNoz
### Application Performance Monitoring
#### SigNoz Cloud (Recommended)
Use SigNoz APM to monitor your applications and services. It comes with out-of-box charts for key application metrics like p99 latency, error rate, Apdex and operations per second. You can also monitor the database and external calls made from your application. Read [more](https://signoz.io/application-performance-monitoring/).
Fully managed SigNoz with a 30-day free trial, no credit card required, usage-based pricing that starts at $49, and regional data hosting.
You can [instrument](https://signoz.io/docs/instrumentation/) your application with OpenTelemetry to get started.
[**Start free →**](https://signoz.io/teams/)
![apm-cover](https://github.com/user-attachments/assets/fa5c0396-0854-4c8b-b972-9b62fd2a70d2)
#### Enterprise
Enterprise Cloud, BYOC, or Enterprise Self-Hosted with compliance, support, custom retention, RBAC, ingestion controls, data residency, and region selection.
### Logs Management
[**Explore Enterprise →**](https://signoz.io/enterprise/)
SigNoz can be used as a centralized log management solution. We use ClickHouse (used by likes of Uber & Cloudflare) as a datastore, ⎯ an extremely fast and highly optimized storage for logs data. Instantly search through all your logs using quick filters and a powerful query builder.
#### Community
You can also create charts on your logs and monitor them with customized dashboards. Read [more](https://signoz.io/log-management/).
Free open-source SigNoz that runs in your own infrastructure. Deploy with Docker, Kubernetes, or Linux and keep full control of your data plane.
![logs-management-cover](https://github.com/user-attachments/assets/343588ee-98fb-4310-b3d2-c5bacf9c7384)
[**Install SigNoz →**](https://signoz.io/docs/install/self-host/)
### What can you monitor?
### Distributed Tracing
SigNoz helps teams debug production issues faster by connecting logs, metrics, traces, alerts, dashboards, exceptions, and agent-native workflows in one place.
Distributed Tracing is essential to troubleshoot issues in microservices applications. Powered by OpenTelemetry, distributed tracing in SigNoz can help you track user requests across services to help you identify performance bottlenecks.
#### APM Overview
See user requests in a detailed breakdown with the help of Flamegraphs and Gantt Charts. Click on any span to see the entire trace represented beautifully, which will help you make sense of where issues actually occurred in the flow of requests.
Monitor service latency, error rate, throughput, Apdex, top endpoints, database calls, and external calls.
Read [more](https://signoz.io/distributed-tracing/).
<p align="center">
<img alt="SigNoz APM dashboard showing latency, throughput, Apdex, and key operations" src="docs/readme-assets/monitor/apm.png" width="900">
</p>
![distributed-tracing-cover](https://github.com/user-attachments/assets/9bfe060a-0c40-4922-9b55-8a97e1a4076c)
Learn more: [APM documentation](https://signoz.io/docs/instrumentation/overview/)
#### Log Management
Ingest, search, aggregate, and correlate logs with traces and metrics using a visual query builder.
### Metrics and Dashboards
<p align="center">
<img alt="SigNoz logs explorer with filters, frequency chart, and log lines" src="docs/readme-assets/monitor/log-management.svg" width="900">
</p>
Ingest metrics from your infrastructure or applications and create customized dashboards to monitor them. Create visualization that suits your needs with a variety of panel types like pie chart, time-series, bar chart, etc.
Learn more: [Log management documentation](https://signoz.io/docs/logs-management/overview/)
Create queries on your metrics data quickly with an easy-to-use metrics query builder. Add multiple queries and combine those queries with formulae to create really complex queries quickly.
#### Metrics and Dashboards
Read [more](https://signoz.io/metrics-and-dashboards/).
Build dashboards for application, infrastructure, and custom metrics using Query Builder, PromQL, or ClickHouse SQL.
![metrics-n-dashboards-cover](https://github.com/user-attachments/assets/a536fd71-1d2c-4681-aa7e-516d754c47a5)
<p align="center">
<img alt="SigNoz host metrics dashboard with system load and network charts" src="docs/readme-assets/monitor/metrics.png" width="900">
</p>
### LLM Observability
Learn more: [Metrics documentation](https://signoz.io/docs/metrics-management/overview/)
Monitor and debug your LLM applications with comprehensive observability. Track LLM calls, analyze token usage, monitor performance, and gain insights into your AI application's behavior in production.
#### Infrastructure Monitoring
SigNoz LLM observability helps you understand how your language models are performing, identify issues with prompts and responses, track token usage and costs, and optimize your AI applications for better performance and reliability.
Monitor Kubernetes clusters, pods, nodes, workloads, and host-level CPU, memory, disk, network, logs, and traces.
[Get started with LLM Observability →](https://signoz.io/docs/llm-observability/)
<p align="center">
<img alt="SigNoz Kubernetes infrastructure dashboard with pod and node metrics" src="docs/readme-assets/monitor/infrastructure.png" width="900">
</p>
![llm-observability-cover](https://github.com/user-attachments/assets/a6cc0ca3-59df-48f9-9c16-7c843fccff96)
Learn more: [Infrastructure monitoring documentation](https://signoz.io/docs/infrastructure-monitoring/overview/)
#### LLM and AI Observability
### Alerts
Trace LLM apps, RAG pipelines, prompts, tool calls, tokens, latency, and costs alongside application and infrastructure telemetry.
Use alerts in SigNoz to get notified when anything unusual happens in your application. You can set alerts on any type of telemetry signal (logs, metrics, traces), create thresholds and set up a notification channel to get notified. Advanced features like alert history and anomaly detection can help you create smarter alerts.
<p align="center">
<img alt="SigNoz LLM observability dashboard for traces, token usage, latency, and costs" src="docs/readme-assets/monitor/llm.png" width="900">
</p>
Alerts in SigNoz help you identify issues proactively so that you can address them before they reach your customers.
Learn more: [LLM observability documentation](https://signoz.io/docs/llm-observability/)
Read [more](https://signoz.io/alerts-management/).
#### Agent-Native Observability and MCP
![alerts-cover](https://github.com/user-attachments/assets/03873bb8-1b62-4adf-8f56-28bb7b1750ea)
Use the SigNoz MCP server to bring telemetry into coding agents, or use Noz inside SigNoz to investigate incidents, tune alerts, and build dashboards with production context. [Noz](https://signoz.io/docs/ai/noz/) is available only on SigNoz Cloud.
### Exceptions Monitoring
<p align="center">
<img alt="SigNoz Noz interface alongside MCP-powered agent workflow" src="docs/readme-assets/monitor/agent-native.png" width="900">
</p>
Monitor exceptions automatically in Python, Java, Ruby, and Javascript. For other languages, just drop in a few lines of code and start monitoring exceptions.
Learn more: [SigNoz MCP server docs](https://signoz.io/docs/ai/signoz-mcp-server/) · [Agent skills docs](https://signoz.io/docs/ai/agent-skills/#install-the-plugin)
See the detailed stack trace for all exceptions caught in your application. You can also log in custom attributes to add more context to your exceptions. For example, you can add attributes to identify users for which exceptions occurred.
#### Distributed Tracing
Read [more](https://signoz.io/exceptions-monitoring/).
Follow requests across services with flamegraphs, waterfalls, span events, filters, and trace analytics.
<p align="center">
<img alt="SigNoz distributed trace view with flamegraph and waterfall spans" src="docs/readme-assets/monitor/distributed-tracing.png" width="900">
</p>
![exceptions-cover](https://github.com/user-attachments/assets/4be37864-59f2-4e8a-8d6e-e29ad04298c5)
Learn more: [Distributed tracing documentation](https://signoz.io/docs/instrumentation/)
#### Trace Funnels
<br /><br />
Create funnels from traces to understand request-flow drop-offs, failed transitions, and systemic workflow issues.
## Why SigNoz?
<p align="center">
<img alt="SigNoz trace funnels showing request-flow drop-offs and failed transitions" src="docs/readme-assets/monitor/trace-funnels.png" width="900">
</p>
SigNoz is a single tool for all your monitoring and observability needs. Here are a few reasons why you should choose SigNoz:
Learn more: [Trace funnels documentation](https://signoz.io/docs/trace-funnels/overview/)
- Single tool for observability(logs, metrics, and traces)
Also monitor: [**exceptions**](https://signoz.io/docs/userguide/exceptions/), [**alerts**](https://signoz.io/docs/alerts/), [**external APIs**](https://signoz.io/docs/external-api-monitoring/overview/), and [**integrations**](https://signoz.io/docs/integrations/integrations-list/) for OpenTelemetry, Prometheus, Kubernetes, cloud providers, language SDKs, application frameworks, databases, and LLM tools.
- Built on top of [OpenTelemetry](https://opentelemetry.io/), the open-source standard which frees you from any type of vendor lock-in
### Why teams use SigNoz
- Correlated logs, metrics and traces for much richer context while debugging
1. **OpenTelemetry-native**<br>
Instrument once with open standards and keep ownership of your telemetry.
2. **Correlated signals**<br>
Move from service charts to traces, logs, infra metrics, and exceptions without switching tools.
3. **Single columnar database**<br>
Built for high-cardinality, high-volume observability workloads.
4. **Predictable pricing**<br>
No per-host pricing, no user-seat pricing, and no special pricing for custom metrics.
5. **Enterprise ready**<br>
SOC 2 Type II and HIPAA compliance, RBAC, ingestion controls, custom retention, support, BYOC, and self-hosting.
- Uses ClickHouse (used by likes of Uber & Cloudflare) as datastore - an extremely fast and highly optimized storage for observability data
### Getting started
- DIY Query builder, PromQL, and ClickHouse queries to fulfill all your use-cases around querying observability data
#### Start on Cloud
- Open-Source - you can use open-source, our [cloud service](https://signoz.io/teams/) or a mix of both based on your use case
Create a managed SigNoz workspace and get your first dashboard without running observability infrastructure.
[**Start free on SigNoz Cloud**](https://signoz.io/teams/)
## Getting Started
#### Self-host SigNoz
### Create a SigNoz Cloud Account
Run SigNoz in your own infrastructure with Foundry, Docker, Kubernetes, or Linux.
SigNoz cloud is the easiest way to get started with SigNoz. Our cloud service is for those users who want to spend more time in getting insights for their application performance without worrying about maintenance.
[**Foundry**](https://github.com/SigNoz/foundry) · [**Docker**](https://signoz.io/docs/install/docker/) · [**Kubernetes**](https://signoz.io/docs/install/kubernetes/) · [**Linux**](https://signoz.io/docs/install/linux/)
[Get started for free](https://signoz.io/teams/)
#### Send data
### Deploy using Docker(self-hosted)
Instrument applications and infrastructure with OpenTelemetry, Prometheus, language SDKs, and integrations.
Please follow the steps listed [here](https://signoz.io/docs/install/docker/) to install using docker
[**Instrumentation**](https://signoz.io/docs/instrumentation/) · [**Integrations**](https://signoz.io/docs/integrations/integrations-list/)
The [troubleshooting instructions](https://signoz.io/docs/install/troubleshooting/) may be helpful if you face any issues.
### Comparisons to familiar tools
<p>&nbsp </p>
### Deploy in Kubernetes using Helm(self-hosted)
SigNoz is often adopted by teams moving from a stack of single-purpose tools or commercial platforms with unpredictable pricing.
Please follow the steps listed [here](https://signoz.io/docs/deployment/helm_chart) to install using helm charts
**Prometheus**<br>
Good if you just need metrics. SigNoz keeps metrics, logs, traces, dashboards, and alerts together so teams can debug with correlated context.
<br /><br />
**Jaeger**<br>
Jaeger only does distributed tracing. SigNoz adds metrics, logs, trace analytics, dashboards, alerts, exceptions, and trace-to-log workflows.
We also offer managed services in your infra. Check our [pricing plans](https://signoz.io/pricing/) for all details.
**Elastic**<br>
SigNoz uses columnar database for efficient observability analytics and high-cardinality log workloads, with 50% lower resource requirement compared to Elastic during ingestion. Check the [detailed study](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark).
**Loki**<br>
In the linked benchmark, SigNoz indexed all keys in the test setup, while Loki hit max stream errors when more labels were added. Check the [detailed study](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark).
## Join our Slack community
Come say Hi to us on [Slack](https://signoz.io/slack) 👋
<br /><br />
### Languages supported:
SigNoz supports all major programming languages for monitoring. Any framework and language supported by OpenTelemetry is supported by SigNoz. Find instructions for instrumenting different languages below:
- [Java](https://signoz.io/docs/instrumentation/java/)
- [Python](https://signoz.io/docs/instrumentation/python/)
- [Node.js or Javascript](https://signoz.io/docs/instrumentation/javascript/)
- [Go](https://signoz.io/docs/instrumentation/golang/)
- [PHP](https://signoz.io/docs/instrumentation/php/)
- [.NET](https://signoz.io/docs/instrumentation/dotnet/)
- [Ruby](https://signoz.io/docs/instrumentation/ruby-on-rails/)
- [Elixir](https://signoz.io/docs/instrumentation/elixir/)
- [Rust](https://signoz.io/docs/instrumentation/rust/)
- [Swift](https://signoz.io/docs/instrumentation/swift/)
You can find our entire documentation [here](https://signoz.io/docs/introduction/).
<br /><br />
## Comparisons to Familiar Tools
### SigNoz vs Prometheus
Prometheus is good if you want to do just metrics. But if you want to have a seamless experience between metrics, logs and traces, then current experience of stitching together Prometheus & other tools is not great.
SigNoz is a one-stop solution for metrics and other telemetry signals. And because you will use the same standard(OpenTelemetry) to collect all telemetry signals, you can also correlate these signals to troubleshoot quickly.
For example, if you see that there are issues with infrastructure metrics of your k8s cluster at a timestamp, you can jump to other signals like logs and traces to understand the issue quickly.
<p>&nbsp </p>
### SigNoz vs Jaeger
Jaeger only does distributed tracing. SigNoz supports metrics, traces and logs - all the 3 pillars of observability.
Moreover, SigNoz has few more advanced features wrt Jaeger:
- Jaegar UI doesnt show any metrics on traces or on filtered traces
- Jaeger cant get aggregates on filtered traces. For example, p99 latency of requests which have tag - customer_type='premium'. This can be done easily on SigNoz
- You can also go from traces to logs easily in SigNoz
<p>&nbsp </p>
### SigNoz vs Elastic
- SigNoz Logs management are based on ClickHouse, a columnar OLAP datastore which makes aggregate log analytics queries much more efficient
- 50% lower resource requirement compared to Elastic during ingestion
We have published benchmarks comparing Elastic with SigNoz. Check it out [here](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)
<p>&nbsp </p>
### SigNoz vs Loki
- SigNoz supports aggregations on high-cardinality data over a huge volume while loki doesnt.
- SigNoz supports indexes over high cardinality data and has no limitations on the number of indexes, while Loki reaches max streams with a few indexes added to it.
- Searching over a huge volume of data is difficult and slow in Loki compared to SigNoz
We have published benchmarks comparing Loki with SigNoz. Check it out [here](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)
<br /><br />
## Contributing
We ❤️ contributions big or small. Please read [CONTRIBUTING.md](CONTRIBUTING.md) to get started with making contributions to SigNoz.
Not sure how to get started? **Just ping us on `#contributing` in our [slack community](https://signoz.io/slack).**
Not sure how to get started? Just ping us on `#contributing` in our [slack community](https://signoz.io/slack)
<br /><br />
## Documentation
You can find docs at https://signoz.io/docs/. If you need any clarification or find something missing, feel free to raise a GitHub issue with the label `documentation` or reach out to us at the community slack channel.
<br /><br />
## Community
Join the [slack community](https://signoz.io/slack) to know more about distributed tracing, observability, or SigNoz and to connect with other users and contributors.
If you have any ideas, questions, or any feedback, please share on our [Github Discussions](https://github.com/SigNoz/signoz/discussions)
As always, thanks to our amazing contributors!
<a href="https://github.com/signoz/signoz/graphs/contributors">
<img alt="SigNoz contributors" src="https://contrib.rocks/image?repo=signoz/signoz" />
<img src="https://contrib.rocks/image?repo=signoz/signoz" />
</a>

View File

@@ -1,190 +1,158 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/readme-assets/signoz-hero-dark.png" width="900">
<source media="(prefers-color-scheme: light)" srcset="docs/readme-assets/signoz-hero-light.png" width="900">
<img alt="SigNoz - Observabilidade nos seus termos, baseada em padrões abertos." src="docs/readme-assets/signoz-hero-light.png" width="900">
</picture>
<img src="https://res.cloudinary.com/dcv3epinx/image/upload/v1618904450/signoz-images/LogoGithub_sigfbu.svg" alt="SigNoz-logo" width="240" />
<p align="center">Monitore seus aplicativos e solucione problemas em seus aplicativos implantados, uma alternativa de código aberto para soluções como DataDog, New Relic, entre outras.</p>
</p>
<p align="center">
<a href="README.md">English</a> ·
<a href="README.zh-cn.md">中文</a> ·
<a href="README.de-de.md">Deutsch</a>
<img alt="Downloads" src="https://img.shields.io/docker/pulls/signoz/frontend?label=Downloads"> </a>
<img alt="GitHub issues" src="https://img.shields.io/github/issues/signoz/signoz"> </a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability">
<img alt="tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"> </a>
</p>
<h3 align="center">
<a href="https://signoz.io/docs"><b>Documentação</b></a> &bull;
<a href="https://signoz.io/slack"><b>Comunidade no Slack</b></a> &bull;
<a href="https://twitter.com/SigNozHq"><b>Twitter</b></a>
</h3>
<p align="center">
<a href="https://github.com/SigNoz/signoz/issues"><img alt="GitHub issues" src="https://img.shields.io/github/issues/SigNoz/signoz"></a>
<a href="https://github.com/SigNoz/signoz/releases"><img alt="GitHub release" src="https://img.shields.io/github/v/release/SigNoz/signoz?label=release"></a>
<a href="https://signoz.io/slack"><img alt="Slack community" src="https://img.shields.io/badge/slack-community-4A154B?logo=slack&logoColor=white"></a>
<a href="https://www.linkedin.com/company/signozio/"><img alt="LinkedIn" src="https://img.shields.io/badge/linkedin-SigNoz-0A66C2?logo=linkedin&logoColor=white"></a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability"><img alt="Tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"></a>
</p>
##
SigNoz é uma plataforma de observabilidade open-source construída sobre OpenTelemetry. Estamos criando uma alternativa de nível empresarial a stacks de monitoramento fragmentadas, com logs, métricas, traces, alertas e dashboards em um só lugar.
SigNoz auxilia os desenvolvedores a monitorarem aplicativos e solucionar problemas em seus aplicativos implantados. SigNoz usa rastreamento distribuído para obter visibilidade em sua pilha de software.
### Escolha como executar o SigNoz
👉 Você pode verificar métricas como latência p99, taxas de erro em seus serviços, requisições às APIs externas e endpoints individuais.
#### SigNoz Cloud (recomendado)
👉 Você pode encontrar a causa raiz do problema acessando os rastreamentos exatos que estão causando o problema e verificar os quadros detalhados de cada requisição individual.
SigNoz totalmente gerenciado, com teste gratuito de 30 dias, sem cartão de crédito, preço baseado em uso a partir de US$ 49 e hospedagem de dados por região.
👉 Execute agregações em dados de rastreamento para obter métricas de negócios relevantes.
[**Comece gratuitamente →**](https://signoz.io/teams/)
#### Enterprise
![SigNoz Feature](https://signoz-public.s3.us-east-2.amazonaws.com/signoz_hero_github.png)
Enterprise Cloud, BYOC ou Enterprise Self-Hosted com compliance, suporte, retenção personalizada, RBAC, controles de ingestão, residência de dados e seleção de região.
<br /><br />
[**Conheça o Enterprise →**](https://signoz.io/enterprise/)
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/Contributing.svg" width="50px" />
#### Community
## Junte-se à nossa comunidade no Slack
SigNoz open-source gratuito, executado na sua própria infraestrutura. Faça o deploy com Docker, Kubernetes ou Linux e mantenha controle total sobre o seu plano de dados.
Venha dizer oi para nós no [Slack](https://signoz.io/slack) 👋
[**Instale o SigNoz →**](https://signoz.io/docs/install/self-host/)
<br /><br />
### O que você pode monitorar?
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/Features.svg" width="50px" />
O SigNoz ajuda equipes a depurar problemas de produção mais rapidamente ao conectar logs, métricas, traces, alertas, dashboards, exceções e fluxos agent-native em um só lugar.
## Funções:
#### Visão geral de APM
- Métricas de visão geral do aplicativo, como RPS, latências de percentual 50/90/99 e taxa de erro
- Endpoints mais lentos em seu aplicativo
- Visualize o rastreamento preciso de requisições de rede para descobrir problemas em serviços downstream, consultas lentas de banco de dados, chamadas para serviços de terceiros, como gateways de pagamento, etc.
- Filtre os rastreamentos por nome de serviço, operação, latência, erro, tags / anotações.
- Execute agregações em dados de rastreamento (eventos / extensões) para obter métricas de negócios relevantes, como por exemplo, você pode obter a taxa de erro e a latência do 99º percentil de `customer_type: gold` or `deployment_version: v2` or `external_call: paypal`
- Interface de Usuário unificada para métricas e rastreios. Não há necessidade de mudar de Prometheus para Jaeger para depurar problemas.
Monitore latência de serviço, taxa de erro, throughput, Apdex, principais endpoints, chamadas ao banco de dados e chamadas externas.
<br /><br />
<p align="center">
<img alt="Dashboard de APM do SigNoz mostrando latência, throughput, Apdex e operações principais" src="docs/readme-assets/monitor/apm.png" width="900">
</p>
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/WhatsCool.svg" width="50px" />
Saiba mais: [documentação de APM](https://signoz.io/docs/instrumentation/overview/)
## Por que escolher SigNoz?
#### Gerenciamento de logs
Sendo desenvolvedores, achamos irritante contar com fornecedores de SaaS de código fechado para cada pequeno recurso que queríamos. Fornecedores de código fechado costumam surpreendê-lo com enormes contas no final do mês de uso sem qualquer transparência .
Ingira, pesquise, agregue e correlacione logs com traces e métricas usando um construtor visual de consultas.
Queríamos fazer uma versão auto-hospedada e de código aberto de ferramentas como DataDog, NewRelic para empresas que têm preocupações com privacidade e segurança em ter dados de clientes indo para serviços de terceiros.
<p align="center">
<img alt="Explorador de logs do SigNoz com filtros, gráfico de frequência e linhas de log" src="docs/readme-assets/monitor/log-management.svg" width="900">
</p>
Ser open source também oferece controle completo de sua configuração, amostragem e tempos de atividade. Você também pode construir módulos sobre o SigNoz para estender recursos específicos do negócio.
Saiba mais: [documentação de gerenciamento de logs](https://signoz.io/docs/logs-management/overview/)
### Linguagens Suportadas:
#### Métricas e dashboards
Nós apoiamos a biblioteca [OpenTelemetry](https://opentelemetry.io) como a biblioteca que você pode usar para instrumentar seus aplicativos. Em outras palavras, SigNoz oferece suporte a qualquer framework e linguagem que suporte a biblioteca OpenTelemetry. As principais linguagens suportadas incluem:
Crie dashboards para métricas de aplicação, infraestrutura e métricas personalizadas usando Query Builder, PromQL ou ClickHouse SQL.
- Java
- Python
- NodeJS
- Go
<p align="center">
<img alt="Dashboard de métricas de host do SigNoz com gráficos de carga do sistema e rede" src="docs/readme-assets/monitor/metrics.png" width="900">
</p>
Você pode encontrar a lista completa de linguagens aqui - https://opentelemetry.io/docs/
Saiba mais: [documentação de métricas](https://signoz.io/docs/metrics-management/overview/)
<br /><br />
#### Monitoramento de infraestrutura
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/Philosophy.svg" width="50px" />
Monitore clusters Kubernetes, pods, nodes, workloads e CPU, memória, disco, rede, logs e traces em nível de host.
## Iniciando
### Implantar usando Docker
<p align="center">
<img alt="Dashboard de infraestrutura Kubernetes do SigNoz com métricas de pods e nodes" src="docs/readme-assets/monitor/infrastructure.png" width="900">
</p>
Siga as etapas listadas [aqui](https://signoz.io/docs/install/docker/) para instalar usando o Docker.
Saiba mais: [documentação de monitoramento de infraestrutura](https://signoz.io/docs/infrastructure-monitoring/overview/)
Esse [guia para solução de problemas](https://signoz.io/docs/install/troubleshooting/) pode ser útil se você enfrentar quaisquer problemas.
#### Observabilidade de LLM e AI
<p>&nbsp </p>
### Implentar no Kubernetes usando Helm
Rastreie apps LLM, pipelines RAG, prompts, chamadas de ferramentas, tokens, latência e custos junto com telemetria de aplicação e infraestrutura.
Siga as etapas listadas [aqui](https://signoz.io/docs/deployment/helm_chart) para instalar usando helm charts.
<p align="center">
<img alt="Dashboard de observabilidade de LLM do SigNoz para traces, uso de tokens, latência e custos" src="docs/readme-assets/monitor/llm.png" width="900">
</p>
<br /><br />
Saiba mais: [documentação de observabilidade de LLM](https://signoz.io/docs/llm-observability/)
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/UseSigNoz.svg" width="50px" />
#### Observabilidade agent-native e MCP
## Comparações com ferramentas similares
Use o servidor MCP do SigNoz para levar telemetria aos agentes de programação, ou use o Noz dentro do SigNoz para investigar incidentes, ajustar alertas e criar dashboards com contexto de produção. O [Noz](https://signoz.io/docs/ai/noz/) está disponível apenas no SigNoz Cloud.
### SigNoz ou Prometheus
<p align="center">
<img alt="Interface Noz do SigNoz ao lado de um fluxo agent via MCP" src="docs/readme-assets/monitor/agent-native.png" width="900">
</p>
Prometheus é bom se você quiser apenas fazer métricas. Mas se você quiser ter uma experiência perfeita entre métricas e rastreamentos, a experiência atual de unir Prometheus e Jaeger não é ótima.
Saiba mais: [documentação do servidor MCP do SigNoz](https://signoz.io/docs/ai/signoz-mcp-server/) · [documentação de agent skills](https://signoz.io/docs/ai/agent-skills/#install-the-plugin)
Nosso objetivo é fornecer uma interface do usuário integrada entre métricas e rastreamentos - semelhante ao que fornecedores de SaaS como o Datadog fornecem - e fornecer filtragem e agregação avançada sobre rastreamentos, algo que a Jaeger atualmente carece.
#### Tracing distribuído
<p>&nbsp </p>
Acompanhe requisições entre serviços com flamegraphs, waterfalls, eventos de span, filtros e análise de traces.
### SigNoz ou Jaeger
<p align="center">
<img alt="Visualização de tracing distribuído do SigNoz com flamegraph e spans em waterfall" src="docs/readme-assets/monitor/distributed-tracing.png" width="900">
</p>
Jaeger só faz rastreamento distribuído. SigNoz faz métricas e rastreia, e também temos gerenciamento de log em nossos planos.
Saiba mais: [documentação de tracing distribuído](https://signoz.io/docs/instrumentation/)
Além disso, SigNoz tem alguns recursos mais avançados do que Jaeger:
#### Trace Funnels
- A interface de usuário do Jaegar não mostra nenhuma métrica em traces ou em traces filtrados
- Jaeger não pode obter agregados em rastros filtrados. Por exemplo, latência p99 de solicitações que possuem tag - customer_type='premium'. Isso pode ser feito facilmente com SigNoz.
Crie funis a partir de traces para entender quedas no fluxo de requisições, transições com falha e problemas sistêmicos de workflow.
<br /><br />
<p align="center">
<img alt="Trace Funnels do SigNoz mostrando quedas no fluxo de requisições e transições com falha" src="docs/readme-assets/monitor/trace-funnels.png" width="900">
</p>
Saiba mais: [documentação de Trace Funnels](https://signoz.io/docs/trace-funnels/overview/)
Também monitore: [**exceções**](https://signoz.io/docs/userguide/exceptions/), [**alertas**](https://signoz.io/docs/alerts/), [**APIs externas**](https://signoz.io/docs/external-api-monitoring/overview/) e [**integrações**](https://signoz.io/docs/integrations/integrations-list/) para OpenTelemetry, Prometheus, Kubernetes, provedores de nuvem, SDKs de linguagem, frameworks de aplicação, bancos de dados e ferramentas de LLM.
### Por que equipes usam o SigNoz
1. **Nativo em OpenTelemetry**<br>
Instrumente uma vez com padrões abertos e mantenha a posse da sua telemetria.
2. **Sinais correlacionados**<br>
Vá de gráficos de serviço para traces, logs, métricas de infraestrutura e exceções sem trocar de ferramenta.
3. **Um único banco de dados colunar**<br>
Construído para workloads de observabilidade de alto volume e alta cardinalidade.
4. **Preço previsível**<br>
Sem cobrança por host, sem cobrança por usuário e sem preço especial para métricas personalizadas.
5. **Pronto para enterprise**<br>
Compliance SOC 2 Type II e HIPAA, RBAC, controles de ingestão, retenção personalizada, suporte, BYOC e self-hosting.
### Primeiros passos
#### Comece na Cloud
Crie um workspace gerenciado do SigNoz e obtenha seu primeiro dashboard sem operar infraestrutura de observabilidade.
[**Comece gratuitamente no SigNoz Cloud**](https://signoz.io/teams/)
#### Self-host SigNoz
Execute o SigNoz na sua própria infraestrutura com Foundry, Docker, Kubernetes ou Linux.
[**Foundry**](https://github.com/SigNoz/foundry) · [**Docker**](https://signoz.io/docs/install/docker/) · [**Kubernetes**](https://signoz.io/docs/install/kubernetes/) · [**Linux**](https://signoz.io/docs/install/linux/)
#### Envie dados
Instrumente aplicações e infraestrutura com OpenTelemetry, Prometheus, SDKs de linguagem e integrações.
[**Instrumentação**](https://signoz.io/docs/instrumentation/) · [**Integrações**](https://signoz.io/docs/integrations/integrations-list/)
### Comparações com ferramentas conhecidas
O SigNoz é frequentemente adotado por equipes que estão migrando de ferramentas de propósito único ou plataformas comerciais com preços imprevisíveis.
**Prometheus**<br>
Bom se você precisa apenas de métricas. O SigNoz mantém métricas, logs, traces, dashboards e alertas juntos para que equipes possam depurar com contexto correlacionado.
**Jaeger**<br>
Jaeger faz apenas tracing distribuído. O SigNoz adiciona métricas, logs, análise de traces, dashboards, alertas, exceções e fluxos de trace para log.
**Elastic**<br>
O SigNoz usa banco de dados colunar para análises de observabilidade eficientes e workloads de logs de alta cardinalidade, com 50% menos necessidade de recursos em comparação ao Elastic durante a ingestão. Confira o [estudo detalhado](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark).
**Loki**<br>
No benchmark vinculado, o SigNoz indexou todas as chaves na configuração de teste, enquanto o Loki atingiu erros de max streams ao adicionar mais labels. Confira o [estudo detalhado](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark).
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/Contributors.svg" width="50px" />
## Contribuindo
Adoramos contribuições grandes ou pequenas. Leia [CONTRIBUTING.md](CONTRIBUTING.md) para começar a contribuir com o SigNoz.
Não sabe como começar? **Fale conosco no `#contributing` na nossa [comunidade Slack](https://signoz.io/slack).**
Nós ❤️ contribuições grandes ou pequenas. Leia [CONTRIBUTING.md](CONTRIBUTING.md) para começar a fazer contribuições para o SigNoz.
Como sempre, obrigado aos nossos incríveis contribuidores!
Não sabe como começar? Basta enviar um sinal para nós no canal `#contributing` em nossa [comunidade no Slack.](https://signoz.io/slack)
<br /><br />
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/DevelopingLocally.svg" width="50px" />
## Documentação
Você pode encontrar a documentação em https://signoz.io/docs/. Se você tiver alguma dúvida ou sentir falta de algo, sinta-se à vontade para criar uma issue com a tag `documentation` no GitHub ou entre em contato conosco no canal da comunidade no Slack.
<br /><br />
<img align="left" src="https://signoz-public.s3.us-east-2.amazonaws.com/Contributing.svg" width="50px" />
## Comunidade
Junte-se a [comunidade no Slack](https://signoz.io/slack) para saber mais sobre rastreamento distribuído, observabilidade ou SigNoz e para se conectar com outros usuários e colaboradores.
Se você tiver alguma ideia, pergunta ou feedback, compartilhe em nosso [Github Discussões](https://github.com/SigNoz/signoz/discussions)
Como sempre, obrigado aos nossos incríveis colaboradores!
<a href="https://github.com/signoz/signoz/graphs/contributors">
<img alt="Contribuidores do SigNoz" src="https://contrib.rocks/image?repo=signoz/signoz" />
<img src="https://contrib.rocks/image?repo=signoz/signoz" />
</a>

View File

@@ -1,190 +1,208 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/readme-assets/signoz-hero-dark.png" width="900">
<source media="(prefers-color-scheme: light)" srcset="docs/readme-assets/signoz-hero-light.png" width="900">
<img alt="SigNoz - 按你的方式运行的可观测性,由开放标准驱动。" src="docs/readme-assets/signoz-hero-light.png" width="900">
</picture>
<img src="https://res.cloudinary.com/dcv3epinx/image/upload/v1618904450/signoz-images/LogoGithub_sigfbu.svg" alt="SigNoz-logo" width="240" />
<p align="center">监控你的应用,并且可排查已部署应用的问题,这是一个可替代 DataDog、NewRelic 的开源方案</p>
</p>
<p align="center">
<a href="README.md">English</a> ·
<a href="README.de-de.md">Deutsch</a> ·
<a href="README.pt-br.md">Português</a>
<img alt="Downloads" src="https://img.shields.io/docker/pulls/signoz/query-service?label=Docker Downloads"> </a>
<img alt="GitHub issues" src="https://img.shields.io/github/issues/signoz/signoz"> </a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability">
<img alt="tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"> </a>
</p>
<p align="center">
<a href="https://github.com/SigNoz/signoz/issues"><img alt="GitHub issues" src="https://img.shields.io/github/issues/SigNoz/signoz"></a>
<a href="https://github.com/SigNoz/signoz/releases"><img alt="GitHub release" src="https://img.shields.io/github/v/release/SigNoz/signoz?label=release"></a>
<a href="https://signoz.io/slack"><img alt="Slack community" src="https://img.shields.io/badge/slack-community-4A154B?logo=slack&logoColor=white"></a>
<a href="https://www.linkedin.com/company/signozio/"><img alt="LinkedIn" src="https://img.shields.io/badge/linkedin-SigNoz-0A66C2?logo=linkedin&logoColor=white"></a>
<a href="https://twitter.com/intent/tweet?text=Monitor%20your%20applications%20and%20troubleshoot%20problems%20with%20SigNoz,%20an%20open-source%20alternative%20to%20DataDog,%20NewRelic.&url=https://signoz.io/&via=SigNozHQ&hashtags=opensource,signoz,observability"><img alt="Tweet" src="https://img.shields.io/twitter/url/http/shields.io.svg?style=social"></a>
</p>
<h3 align="center">
<a href="https://signoz.io/docs"><b>文档</b></a>
<a href="https://github.com/SigNoz/signoz/blob/main/README.zh-cn.md"><b>中文ReadMe</b></a>
<a href="https://github.com/SigNoz/signoz/blob/main/README.de-de.md"><b>德文ReadMe</b></a>
<a href="https://github.com/SigNoz/signoz/blob/main/README.pt-br.md"><b>葡萄牙语ReadMe</b></a>
<a href="https://signoz.io/slack"><b>Slack 社区</b></a>
<a href="https://twitter.com/SigNozHq"><b>Twitter</b></a>
</h3>
SigNoz 是一个基于 OpenTelemetry 构建的开源可观测性平台。我们正在构建一个企业级替代方案,用来替代分散的监控工具栈,把日志、指标、链路追踪、告警和仪表盘放在同一个地方。
##
### 选择 SigNoz 的运行方式
SigNoz 帮助开发人员监控应用并排查已部署应用的问题。你可以使用 SigNoz 实现如下能力:
#### SigNoz Cloud推荐
👉 在同一块面板上,可视化 Metrics, Traces 和 Logs 内容。
完全托管的 SigNoz提供 30 天免费试用,无需信用卡,按用量计费,起价为 49 美元,并支持区域化数据托管
👉 你可以关注服务的 p99 延迟和错误率, 包括外部 API 调用和个别的端点
[**免费开始 →**](https://signoz.io/teams/)
👉 你可以找到问题的根因,通过提取相关问题的 traces 日志、单独查看请求 traces 的火焰图详情。
#### 企业版
👉 执行 trace 数据聚合,以获取业务相关的 metrics
Enterprise Cloud、BYOC 或 Enterprise Self-Hosted提供合规、支持、自定义保留期、RBAC、摄取控制、数据驻留和区域选择。
👉 对日志过滤和查询,通过日志的属性建立看板和告警
[**了解企业版 →**](https://signoz.io/enterprise/)
👉 通过 PythonjavaRuby 和 Javascript 自动记录异常
#### 社区版
👉 轻松的自定义查询和设置告警
免费的开源 SigNoz可运行在你自己的基础设施中。使用 Docker、Kubernetes 或 Linux 部署,并完全掌控你的数据平面。
### 应用 Metrics 展示
[**安装 SigNoz →**](https://signoz.io/docs/install/self-host/)
![application_metrics](https://user-images.githubusercontent.com/83692067/226637410-900dbc5e-6705-4b11-a10c-bd0faeb2a92f.png)
### 你可以监控什么?
### 分布式追踪
SigNoz 将日志、指标、链路追踪、告警、仪表盘、异常和面向 Agent 的工作流连接在一起,帮助团队更快地调试生产问题。
<img width="2068" alt="distributed_tracing_2 2" src="https://user-images.githubusercontent.com/83692067/226536447-bae58321-6a22-4ed3-af80-e3e964cb3489.png">
#### APM 概览
<img width="2068" alt="distributed_tracing_1" src="https://user-images.githubusercontent.com/83692067/226536462-939745b6-4f9d-45a6-8016-814837e7f7b4.png">
监控服务延迟、错误率、吞吐量、Apdex、核心端点、数据库调用和外部调用。
### 日志管理
<p align="center">
<img alt="SigNoz APM 仪表盘展示延迟、吞吐量、Apdex 和关键操作" src="docs/readme-assets/monitor/apm.png" width="900">
</p>
<img width="2068" alt="logs_management" src="https://user-images.githubusercontent.com/83692067/226536482-b8a5c4af-b69c-43d5-969c-338bd5eaf1a5.png">
了解更多:[APM 文档](https://signoz.io/docs/instrumentation/overview/)
### 基础设施监控
#### 日志管理
<img width="2068" alt="infrastructure_monitoring" src="https://user-images.githubusercontent.com/83692067/226536496-f38c4dbf-e03c-4158-8be0-32d4a61158c7.png">
摄取、搜索、聚合日志,并通过可视化查询构建器将日志与链路追踪和指标关联起来。
### 异常监控
<p align="center">
<img alt="SigNoz 日志浏览器,包含过滤器、频率图和日志行" src="docs/readme-assets/monitor/log-management.svg" width="900">
</p>
![exceptions_light](https://user-images.githubusercontent.com/83692067/226637967-4188d024-3ac9-4799-be95-f5ea9c45436f.png)
了解更多:[日志管理文档](https://signoz.io/docs/logs-management/overview/)
### 告警
#### 指标和仪表盘
<img width="2068" alt="alerts_management" src="https://user-images.githubusercontent.com/83692067/226536548-2c81e2e8-c12d-47e8-bad7-c6be79055def.png">
使用 Query Builder、PromQL 或 ClickHouse SQL 为应用、基础设施和自定义指标构建仪表盘。
<br /><br />
<p align="center">
<img alt="SigNoz 主机指标仪表盘,展示系统负载和网络图表" src="docs/readme-assets/monitor/metrics.png" width="900">
</p>
## 加入我们 Slack 社区
了解更多:[指标文档](https://signoz.io/docs/metrics-management/overview/)
来 [Slack](https://signoz.io/slack) 和我们打招呼吧 👋
#### 基础设施监控
<br /><br />
监控 Kubernetes 集群、Pod、节点、工作负载以及主机级 CPU、内存、磁盘、网络、日志和链路追踪。
## 特性:
<p align="center">
<img alt="SigNoz Kubernetes 基础设施仪表盘,展示 Pod 和节点指标" src="docs/readme-assets/monitor/infrastructure.png" width="900">
</p>
- 为 metrics, traces and logs 制定统一的 UI。 无需切换 Prometheus 到 Jaeger 去查找问题,也无需使用想 Elastic 这样的日志工具分开你的 metrics 和 traces
了解更多:[基础设施监控文档](https://signoz.io/docs/infrastructure-monitoring/overview/)
- 默认统计应用的 metrics 数据,像 RPS (每秒请求数) 50th/90th/99th 的分位数延迟数据,还有相关的错误率
#### LLM 和 AI 可观测性
- 找到应用中最慢的端点
追踪 LLM 应用、RAG 流水线、Prompt、工具调用、Token、延迟和成本并与应用和基础设施遥测数据放在一起分析。
- 查看准确的请求跟踪数据,找到下游服务的问题了,比如 DB 慢查询,或者调用第三方的支付网关等
<p align="center">
<img alt="SigNoz LLM 可观测性仪表盘展示链路追踪、Token 使用、延迟和成本" src="docs/readme-assets/monitor/llm.png" width="900">
</p>
- 通过 服务名、操作方式、延迟、错误、标签/注释 过滤 traces 数据
了解更多:[LLM 可观测性文档](https://signoz.io/docs/llm-observability/)
- 通过聚合 trace 数据而获得业务相关的 metrics。 比如你可以通过 `customer_type: gold` 或者 `deployment_version: v2` 或者 `external_call: paypal` 获取错误率和 P99 延迟数据
#### Agent 原生可观测性和 MCP
- 原生支持 OpenTelemetry 日志,高级日志查询,自动收集 k8s 相关日志
使用 SigNoz MCP server 将遥测数据带入编程 Agent或在 SigNoz 中使用 Noz基于生产上下文调查事故、优化告警并构建仪表盘。[Noz](https://signoz.io/docs/ai/noz/) 仅适用于 SigNoz Cloud。
- 快如闪电的日志分析 ([Logs Perf. Benchmark](https://signoz.io/blog/logs-performance-benchmark/))
<p align="center">
<img alt="SigNoz Noz 界面与基于 MCP 的 Agent 工作流" src="docs/readme-assets/monitor/agent-native.png" width="900">
</p>
- 可视化点到点的基础设施性能,提取有所有类型机器的 metrics 数据
了解更多:[SigNoz MCP server 文档](https://signoz.io/docs/ai/signoz-mcp-server/) · [Agent skills 文档](https://signoz.io/docs/ai/agent-skills/#install-the-plugin)
- 轻易自定义告警查询
#### 分布式链路追踪
<br /><br />
通过火焰图、瀑布图、Span 事件、过滤器和 Trace 分析,跟踪请求在各个服务之间的流转。
## 为什么使用 SigNoz?
<p align="center">
<img alt="SigNoz 分布式链路追踪视图,包含火焰图和瀑布图 Span" src="docs/readme-assets/monitor/distributed-tracing.png" width="900">
</p>
作为开发者, 我们发现 SaaS 厂商对一些大家想要的小功能都是闭源的,这种行为真的让人有点恼火。 闭源厂商还会在月底给你一张没有明细的巨额账单。
了解更多:[分布式链路追踪文档](https://signoz.io/docs/instrumentation/)
我们想做一个自托管并且可开源的工具,像 DataDog 和 NewRelic 那样, 为那些担心数据隐私和安全的公司提供第三方服务。
#### Trace Funnels
作为开源的项目,你完全可以自己掌控你的配置、样本和更新。你同样可以基于 SigNoz 拓展特定的业务模块。
基于链路追踪创建漏斗,用于理解请求流中的掉点、失败转换和系统性工作流问题。
### 支持的编程语言:
<p align="center">
<img alt="SigNoz Trace Funnels展示请求流掉点和失败转换" src="docs/readme-assets/monitor/trace-funnels.png" width="900">
</p>
我们支持 [OpenTelemetry](https://opentelemetry.io)。作为一个观测你应用的库文件。所以任何 OpenTelemetry 支持的框架和语言,对于 SigNoz 也同样支持。 一些主要支持的语言如下:
了解更多:[Trace Funnels 文档](https://signoz.io/docs/trace-funnels/overview/)
- Java
- Python
- NodeJS
- Go
- PHP
- .NET
- Ruby
- Elixir
- Rust
可以监控:[**异常**](https://signoz.io/docs/userguide/exceptions/)、[**告警**](https://signoz.io/docs/alerts/)、[**外部 API**](https://signoz.io/docs/external-api-monitoring/overview/),以及面向 OpenTelemetry、Prometheus、Kubernetes、云服务商、语言 SDK、应用框架、数据库和 LLM 工具的[**集成**](https://signoz.io/docs/integrations/integrations-list/)。
你可以在这里找到全部支持的语言列表 - https://opentelemetry.io/docs/
### 为什么团队选择 SigNoz
<br /><br />
1. **OpenTelemetry 原生**<br>
用开放标准完成一次接入,并保持对遥测数据的所有权。
2. **信号关联**<br>
在服务图表、链路追踪、日志、基础设施指标和异常之间切换时,不需要更换工具。
3. **单一列式数据库**<br>
为高基数、高吞吐量的可观测性工作负载而构建。
4. **可预测的定价**<br>
不按主机收费,不按用户席位收费,也不对自定义指标设置特殊价格。
5. **企业就绪**<br>
SOC 2 Type II 和 HIPAA 合规、RBAC、摄取控制、自定义保留期、支持、BYOC 和自托管。
## 让我们开始吧
### 快速开始
### 使用 Docker 部署
#### 从 Cloud 开始
请一步步跟随 [这里](https://signoz.io/docs/install/docker/) 通过 docker 来安装。
创建一个托管的 SigNoz 工作区,无需运行可观测性基础设施,即可获得第一个仪表盘
这个 [排障说明书](https://signoz.io/docs/install/troubleshooting/) 可以帮助你解决碰到的问题
[**免费开始使用 SigNoz Cloud**](https://signoz.io/teams/)
<p>&nbsp </p>
#### 自托管 SigNoz
### 使用 Helm 在 Kubernetes 部署
在你自己的基础设施中通过 Foundry、Docker、Kubernetes 或 Linux 运行 SigNoz。
请一步步跟随 [这里](https://signoz.io/docs/deployment/helm_chart) 通过 helm 来安装
[**Foundry**](https://github.com/SigNoz/foundry) · [**Docker**](https://signoz.io/docs/install/docker/) · [**Kubernetes**](https://signoz.io/docs/install/kubernetes/) · [**Linux**](https://signoz.io/docs/install/linux/)
<br /><br />
#### 发送数据
## 比较相似的工具
使用 OpenTelemetry、Prometheus、语言 SDK 和集成来接入应用与基础设施。
### SigNoz vs Prometheus
[**接入文档**](https://signoz.io/docs/instrumentation/) · [**集成列表**](https://signoz.io/docs/integrations/integrations-list/)
Prometheus 是一个针对 metrics 监控的强大工具。但是如果你想无缝的切换 metrics 和 traces 查询,你当前大概率需要在 Prometheus 和 Jaeger 之间切换。
### 与常见工具的对比
我们的目标是提供一个客户观测 metrics 和 traces 整合的 UI。就像 SaaS 供应商 DataDog它提供很多 jaeger 缺失的功能,比如针对 traces 过滤功能和聚合功能。
许多团队在从单一用途工具或价格不可预测的商业平台迁移时,会选择 SigNoz。
<p>&nbsp </p>
**Prometheus**<br>
如果你只需要指标Prometheus 很合适。SigNoz 将指标、日志、链路追踪、仪表盘和告警放在一起,让团队可以通过关联上下文进行调试。
### SigNoz vs Jaeger
**Jaeger**<br>
Jaeger 只做分布式链路追踪。SigNoz 增加了指标、日志、Trace 分析、仪表盘、告警、异常和 Trace 到日志的工作流。
Jaeger 仅仅是一个分布式追踪系统。 但是 SigNoz 可以提供 metrics, traces 和 logs 所有的观测。
**Elastic**<br>
SigNoz 使用列式数据库来高效处理可观测性分析和高基数日志工作负载。在摄取阶段,相比 Elastic 可降低 50% 的资源需求。查看[详细评测](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)。
而且, SigNoz 相较于 Jaeger 拥有更对的高级功能:
**Loki**<br>
在链接的评测中SigNoz 在测试设置中索引了所有键,而 Loki 在增加更多标签时遇到了 max stream 错误。查看[详细评测](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)。
- Jaegar UI 不能提供任何基于 traces 的 metrics 查询和过滤。
- Jaeger 不能针对过滤的 traces 做聚合。 比如, p99 延迟的请求有个标签是 customer_type='premium'。 而这些在 SigNoz 可以轻松做到。
<p>&nbsp </p>
### SigNoz vs Elastic
- SigNoz 的日志管理是基于 ClickHouse 实现的,可以使日志的聚合更加高效,因为它是基于 OLAP 的数据仓储。
- 与 Elastic 相比,可以节省 50% 的资源成本
我们已经公布了 Elastic 和 SigNoz 的性能对比。 请点击 [这里](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)
<p>&nbsp </p>
### SigNoz vs Loki
- SigNoz 支持大容量高基数的聚合,但是 loki 是不支持的。
- SigNoz 支持索引的高基数查询,并且对索引没有数量限制,而 Loki 会在添加部分索引后到达最大上限。
- 相较于 SigNozLoki 在搜索大量数据下既困难又缓慢。
我们已经发布了基准测试对比 Loki 和 SigNoz 性能。请点击 [这里](https://signoz.io/blog/logs-performance-benchmark/?utm_source=github-readme&utm_medium=logs-benchmark)
<br /><br />
## 贡献
无论贡献大小,我们都非常欢迎。请阅读 [CONTRIBUTING.md](CONTRIBUTING.md)开始 SigNoz 做贡献。
我们 ❤️ 你的贡献,无论大小。 请先阅读 [CONTRIBUTING.md](CONTRIBUTING.md)开始 SigNoz 做贡献。
不确定如何开始?**可以在我们的 [Slack 社区](https://signoz.io/slack)通过 `#contributing` 联系我们。**
如果你不知道如何开始? 只需要在 [slack 社区](https://signoz.io/slack) 通过 `#contributing` 频道联系我们。
一如既往,感谢所有出色的贡献者!
<br /><br />
## 文档
你可以通过 https://signoz.io/docs/ 找到相关文档。如果你需要阐述问题或者发现一些确实的事件, 通过标签为 `documentation` 提交 Github 问题。或者通过 slack 社区频道。
<br /><br />
## 社区
加入 [slack 社区](https://signoz.io/slack) 去了解更多关于分布式追踪、可观测性系统 。或者与 SigNoz 其他用户和贡献者交流。
如果你有任何想法、问题、或者任何反馈, 请通过 [Github Discussions](https://github.com/SigNoz/signoz/discussions) 分享。
不管怎么样,感谢这个项目的所有贡献者!
<a href="https://github.com/signoz/signoz/graphs/contributors">
<img alt="SigNoz 贡献者" src="https://contrib.rocks/image?repo=signoz/signoz" />
<img src="https://contrib.rocks/image?repo=signoz/signoz" />
</a>

View File

@@ -12,7 +12,7 @@ import (
"github.com/spf13/cobra"
)
const permissionsTypePath = "frontend/src/lib/authz/hooks/useAuthZ/permissions.type.ts"
const permissionsTypePath = "frontend/src/hooks/useAuthZ/permissions.type.ts"
var permissionsTypeTemplate = template.Must(template.New("permissions").Parse(
`// AUTO GENERATED FILE - DO NOT EDIT - GENERATED BY cmd/enterprise/*.go generate authz
@@ -93,15 +93,9 @@ func runGenerateAuthz(_ context.Context) error {
registry := coretypes.NewRegistry()
allowedResources := map[string]bool{
coretypes.NewResourceRef(coretypes.ResourceServiceAccount).String(): true,
coretypes.NewResourceRef(coretypes.ResourceRole).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourceFactorAPIKey).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourceLicense).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourceSubscription).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceLogs).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceTraces).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceMetrics).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceMeterMetrics).String(): true,
coretypes.NewResourceRef(coretypes.ResourceServiceAccount).String(): true,
coretypes.NewResourceRef(coretypes.ResourceRole).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourceFactorAPIKey).String(): true,
}
allowedTypes := map[string]bool{}

View File

@@ -44,11 +44,8 @@ import (
"github.com/SigNoz/signoz/pkg/ruler/signozruler"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/subscription"
"github.com/SigNoz/signoz/pkg/subscription/noopsubscription"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/zeus"
@@ -89,9 +86,6 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(_ sqlstore.SQLStore, _ zeus.Zeus, _ organization.Getter, _ analytics.Analytics) factory.ProviderFactory[licensing.Licensing, licensing.Config] {
return nooplicensing.NewFactory()
},
func(_ zeus.Zeus, _ licensing.Licensing) subscription.Subscription {
return noopsubscription.New()
},
signoz.NewEmailingProviderFactories(),
signoz.NewCacheProviderFactories(),
signoz.NewWebProviderFactories(config.Global),
@@ -109,8 +103,8 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), openfgaDataStore, authtypes.NewRegistry()), nil
},
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, _ querier.Querier, _ licensing.Licensing, tagModule tag.Module, systemDashboardRegistry dashboardtypes.SystemDashboardRegistry) dashboard.Module {
return impldashboard.NewModule(impldashboard.NewStore(store), settings, analytics, orgGetter, queryParser, tagModule, systemDashboardRegistry)
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, _ querier.Querier, _ licensing.Licensing, tagModule tag.Module) dashboard.Module {
return impldashboard.NewModule(impldashboard.NewStore(store), settings, analytics, orgGetter, queryParser, tagModule)
},
func(_ licensing.Licensing) factory.ProviderFactory[gateway.Gateway, gateway.Config] {
return noopgateway.NewProviderFactory()

View File

@@ -31,6 +31,7 @@ builds:
- -X github.com/SigNoz/signoz/pkg/version.branch={{ .Branch }}
- -X github.com/SigNoz/signoz/ee/zeus.url=https://api.signoz.cloud
- -X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=https://license.signoz.io
- -X github.com/SigNoz/signoz/ee/query-service/constants.LicenseSignozIo=https://license.signoz.io/api/v1
- -X github.com/SigNoz/signoz/pkg/analytics.key=9kRrJ7oPCGPEJLF6QjMPLt5bljFhRQBr
mod_timestamp: "{{ .CommitTimestamp }}"
tags:

View File

@@ -4,13 +4,9 @@ ARG OS="linux"
ARG TARGETARCH
ARG ZEUSURL
# HOME comes from the build user, not the image config; declare it so the
# /root paths below trace to it.
ENV HOME=/root
# This path is important for stacktraces
WORKDIR $GOPATH/src/github.com/signoz/signoz
WORKDIR $HOME
WORKDIR /root
RUN set -eux; \
apt-get update; \
@@ -18,36 +14,23 @@ RUN set -eux; \
g++ \
gcc \
libc6-dev \
make \
pkg-config \
; \
rm -rf /var/lib/apt/lists/*
# Keep the literal cache-mount targets below in sync with these. The caches
# are shared with Dockerfile.with-web.integration (same target paths).
ENV GOCACHE=$HOME/.cache/go-build
ENV GOMODCACHE=$GOPATH/pkg/mod
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
go mod download
RUN go mod download
COPY ./cmd/ ./cmd/
COPY ./ee/ ./ee/
COPY ./pkg/ ./pkg/
COPY ./templates /root/templates
# Invoked directly instead of via make so Makefile changes don't invalidate
# this layer; the Makefile's git-derived ldflags resolve to empty in here
# anyway (.git is dockerignored).
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
-ldflags "-s -w \
-X github.com/SigNoz/signoz/pkg/version.version=integration \
-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"
COPY Makefile Makefile
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
RUN chmod 755 /root /root/signoz

View File

@@ -1,23 +1,10 @@
FROM node:22-bookworm AS build
WORKDIR /opt/
# HOME comes from the build user, not the image config.
ENV HOME=/root
# pnpm's store lives at $PNPM_HOME/store — a dedicated directory pnpm
# manages. Keep the literal cache-mount targets below in sync.
ENV PNPM_HOME=/pnpm
ENV NODE_OPTIONS=--max-old-space-size=8192
RUN CI=1 npm i -g pnpm@10
# pnpm fetch resolves from the lockfile alone and runs no lifecycle scripts;
# the repo's postinstall needs source files that are not copied yet.
COPY ./frontend/package.json ./frontend/pnpm-lock.yaml ./frontend/pnpm-workspace.yaml ./
RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm fetch
COPY ./frontend/ ./
RUN --mount=type=cache,target=/pnpm/store CI=1 pnpm install --offline
ENV NODE_OPTIONS=--max-old-space-size=8192
RUN CI=1 npm i -g pnpm@10
RUN CI=1 pnpm install
RUN CI=1 pnpm build
FROM golang:1.25-bookworm
@@ -26,13 +13,9 @@ ARG OS="linux"
ARG TARGETARCH
ARG ZEUSURL
# HOME comes from the build user, not the image config; declare it so the
# /root paths below trace to it.
ENV HOME=/root
# This path is important for stacktraces
WORKDIR $GOPATH/src/github.com/signoz/signoz
WORKDIR $HOME
WORKDIR /root
RUN set -eux; \
apt-get update; \
@@ -40,36 +23,23 @@ RUN set -eux; \
g++ \
gcc \
libc6-dev \
make \
pkg-config \
; \
rm -rf /var/lib/apt/lists/*
# Keep the literal cache-mount targets below in sync with these. The caches
# are shared with Dockerfile.integration (same target paths).
ENV GOCACHE=$HOME/.cache/go-build
ENV GOMODCACHE=$GOPATH/pkg/mod
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
go mod download
RUN go mod download
COPY ./cmd/ ./cmd/
COPY ./ee/ ./ee/
COPY ./pkg/ ./pkg/
COPY ./templates /root/templates
# Invoked directly instead of via make so Makefile changes don't invalidate
# this layer; the Makefile's git-derived ldflags resolve to empty in here
# anyway (.git is dockerignored).
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
GOARCH=${TARGETARCH} GOOS=${OS} go build -C ./cmd/enterprise -race -tags timetzdata -o /root/signoz \
-ldflags "-s -w \
-X github.com/SigNoz/signoz/pkg/version.version=integration \
-X github.com/SigNoz/signoz/pkg/version.variant=enterprise \
-X github.com/SigNoz/signoz/ee/zeus.url=${ZEUSURL} \
-X github.com/SigNoz/signoz/ee/zeus.deprecatedURL=${ZEUSURL}/api/v1"
COPY Makefile Makefile
RUN TARGET_DIR=/root ARCHS=${TARGETARCH} ZEUS_URL=${ZEUSURL} LICENSE_URL=${ZEUSURL}/api/v1 make go-build-enterprise-race
RUN mv /root/linux-${TARGETARCH}/signoz /root/signoz
COPY --from=build /opt/build ./web/

View File

@@ -2,9 +2,9 @@ package main
import (
"github.com/SigNoz/signoz/pkg/metercollector"
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
"github.com/SigNoz/signoz/pkg/types/zeustypes"
)
@@ -25,8 +25,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterLogSize,
Unit: zeustypes.MeterUnitBytes,
Aggregation: zeustypes.MeterAggregationSum,
DBName: logstelemetryschema.DBName,
TableName: logstelemetryschema.LogsV2LocalTableName,
DBName: telemetrylogs.DBName,
TableName: telemetrylogs.LogsV2LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultLogsRetentionDays,
},
},
@@ -36,8 +36,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterSpanSize,
Unit: zeustypes.MeterUnitBytes,
Aggregation: zeustypes.MeterAggregationSum,
DBName: tracestelemetryschema.DBName,
TableName: tracestelemetryschema.SpanIndexV3LocalTableName,
DBName: telemetrytraces.DBName,
TableName: telemetrytraces.SpanIndexV3LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultTracesRetentionDays,
},
},
@@ -47,8 +47,8 @@ var meterConfigs = []metercollector.Config{
Name: zeustypes.MeterDatapointCount,
Unit: zeustypes.MeterUnitCount,
Aggregation: zeustypes.MeterAggregationSum,
DBName: metricstelemetryschema.DBName,
TableName: metricstelemetryschema.SamplesV4LocalTableName,
DBName: telemetrymetrics.DBName,
TableName: telemetrymetrics.SamplesV4LocalTableName,
DefaultRetentionDays: retentiontypes.DefaultMetricsRetentionDays,
},
},

View File

@@ -28,7 +28,6 @@ import (
eequerier "github.com/SigNoz/signoz/ee/querier"
enterpriseapp "github.com/SigNoz/signoz/ee/query-service/app"
eerules "github.com/SigNoz/signoz/ee/query-service/rules"
"github.com/SigNoz/signoz/ee/subscription/httpsubscription"
enterprisezeus "github.com/SigNoz/signoz/ee/zeus"
"github.com/SigNoz/signoz/ee/zeus/httpzeus"
"github.com/SigNoz/signoz/pkg/alertmanager"
@@ -61,11 +60,9 @@ import (
"github.com/SigNoz/signoz/pkg/ruler/signozruler"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/subscription"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/zeus"
@@ -105,9 +102,6 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(sqlstore sqlstore.SQLStore, zeus zeus.Zeus, orgGetter organization.Getter, analytics analytics.Analytics) factory.ProviderFactory[licensing.Licensing, licensing.Config] {
return httplicensing.NewProviderFactory(sqlstore, zeus, orgGetter, analytics)
},
func(zeus zeus.Zeus, licensing licensing.Licensing) subscription.Subscription {
return httpsubscription.New(zeus, licensing)
},
signoz.NewEmailingProviderFactories(),
signoz.NewCacheProviderFactories(),
signoz.NewWebProviderFactories(config.Global),
@@ -142,8 +136,8 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
}
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), openfgaDataStore, licensing, onBeforeRoleDelete, authtypes.NewRegistry()), nil
},
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, querier querier.Querier, licensing licensing.Licensing, tagModule tag.Module, systemDashboardRegistry dashboardtypes.SystemDashboardRegistry) dashboard.Module {
return impldashboard.NewModule(pkgimpldashboard.NewStore(store), settings, analytics, orgGetter, queryParser, querier, licensing, tagModule, systemDashboardRegistry)
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, querier querier.Querier, licensing licensing.Licensing, tagModule tag.Module) dashboard.Module {
return impldashboard.NewModule(pkgimpldashboard.NewStore(store), settings, analytics, orgGetter, queryParser, querier, licensing, tagModule)
},
func(licensing licensing.Licensing) factory.ProviderFactory[gateway.Gateway, gateway.Config] {
return httpgateway.NewProviderFactory(licensing)
@@ -183,11 +177,9 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
return nil, err
}
azureCloudProviderModule := implcloudprovider.NewAzureCloudProvider(defStore)
gcpCloudProviderModule := implcloudprovider.NewGCPCloudProvider(defStore)
cloudProvidersMap := map[cloudintegrationtypes.CloudProviderType]cloudintegration.CloudProviderModule{
cloudintegrationtypes.CloudProviderTypeAWS: awsCloudProviderModule,
cloudintegrationtypes.CloudProviderTypeAzure: azureCloudProviderModule,
cloudintegrationtypes.CloudProviderTypeGCP: gcpCloudProviderModule,
}
return implcloudintegration.NewModule(pkgcloudintegration.NewStore(sqlStore), dashboardModule, global, zeus, gateway, licensing, serviceAccount, cloudProvidersMap, config)

View File

@@ -8,8 +8,6 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/instrumentation"
"github.com/SigNoz/signoz/pkg/modules/dashboard/impldashboard"
"github.com/SigNoz/signoz/pkg/modules/tag/impltag"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/sqlmigration"
"github.com/SigNoz/signoz/pkg/sqlmigrator"
@@ -147,7 +145,7 @@ func newSyncMigrator(ctx context.Context, config signoz.Config, sqlstoreProvider
return nil, err
}
sqlmigrations, err := sqlmigration.New(ctx, providerSettings, config.SQLMigration, signoz.NewSQLMigrationProviderFactories(sqlstore, sqlschema, telemetrystore, providerSettings, impldashboard.NewStore(sqlstore), impltag.NewModule(impltag.NewStore(sqlstore))))
sqlmigrations, err := sqlmigration.New(ctx, providerSettings, config.SQLMigration, signoz.NewSQLMigrationProviderFactories(sqlstore, sqlschema, telemetrystore, providerSettings))
if err != nil {
return nil, err
}

View File

@@ -9,11 +9,6 @@ global:
# the path component (e.g. /signoz in https://example.com/signoz) is used
# as the base path for all HTTP routes (both API and web frontend).
external_url: <unset>
# origins (scheme://host[:port], no path) allowed as login redirect targets. include
# the origin the signoz ui is served on. when not configured, redirect targets are
# not validated.
# allowed_origins:
# - https://signoz.example.com
# the url where the SigNoz backend receives telemetry data (traces, metrics, logs) from instrumented applications.
ingestion_url: <unset>
# the url of the SigNoz MCP server. when unset, the MCP settings page is hidden in the frontend.
@@ -70,31 +65,15 @@ web:
posthog:
# Whether to enable PostHog in web.
enabled: false
# The PostHog project API key.
key: ""
# The PostHog API host. Defaults to https://us.i.posthog.com when empty.
api_host: ""
# The PostHog UI host. Used when api_host points at a reverse proxy.
ui_host: ""
appcues:
# Whether to enable Appcues in web.
enabled: false
# The Appcues account/app ID.
app_id: ""
sentry:
# Whether to enable Sentry in web.
enabled: false
# The Sentry DSN.
dsn: ""
# The Sentry tunnel URL.
tunnel: ""
pylon:
# Whether to enable Pylon in web.
enabled: false
# The Pylon app ID.
app_id: ""
# The Pylon identity verification secret.
identity_secret: ""
##################### Cache #####################
cache:
@@ -134,7 +113,7 @@ sqlstore:
# The timeout for the sqlite database to wait for a lock.
busy_timeout: 10s
# The default transaction locking behavior. Supported values: deferred, immediate, exclusive.
transaction_mode: immediate
transaction_mode: deferred
##################### APIServer #####################
apiserver:
@@ -160,8 +139,8 @@ querier:
cache_ttl: 168h
# The interval for recent data that should not be cached.
flux_interval: 5m
# The maximum number of queries a single query range request runs at once.
max_concurrent_queries: 8
# The maximum number of concurrent queries for missing ranges.
max_concurrent_queries: 4
# When filtering logs by trace_id, clamp the query window to the trace time
# range with padding to include slightly delayed log exports. Logs only; set
# to 0 to disable.
@@ -397,6 +376,7 @@ identn:
# headers to use for tokenizer identN resolver
headers:
- Authorization
- Sec-WebSocket-Protocol
apikey:
# toggle apikey identN
enabled: true

File diff suppressed because it is too large Load Diff

View File

@@ -1,149 +0,0 @@
# Authz
SigNoz uses [OpenFGA](https://openfga.dev/), a relationship-based access control (ReBAC) system, to authorize every request. Transactions are never attached to users directly — they are attached to **roles** (as relationship tuples in OpenFGA), and users or service accounts (principals) are made **assignees** of those roles. The central interface is `AuthZ` in [pkg/authz/authz.go](/pkg/authz/authz.go), backed by an embedded OpenFGA server in [pkg/authz/openfgaserver](/pkg/authz/openfgaserver/server.go).
As a feature author, you will rarely touch OpenFGA directly. You interact with two layers:
1. **The registries** in [pkg/types/coretypes](/pkg/types/coretypes) — where every resource, verb, and managed-role permission is declared in code.
2. **The route wiring** in [pkg/apiserver/signozapiserver](/pkg/apiserver/signozapiserver) — where each route declares what resource it touches and which verb it needs, and the middleware turns that into a check.
## What are the building blocks?
### Type
A `Type` is the coarse FGA type of a resource. The set is finite and defined in [pkg/types/coretypes/registry_type.go](/pkg/types/coretypes/registry_type.go): `user`, `serviceaccount`, `anonymous`, `role`, `organization`, `metaresource`, and `telemetryresource`. Each type carries a selector regex (what a valid ID looks like) and the verbs allowed on it.
Almost every feature resource is a `metaresource` (dashboards, rules, pipelines, ...) or a `telemetryresource` (logs, traces, metrics). You should almost never need a new type.
### Verb
A `Verb` is the action being authorized. All verbs are defined in [pkg/types/coretypes/registry_verb.go](/pkg/types/coretypes/registry_verb.go): `create`, `read`, `update`, `delete`, `list`, `assignee`, `attach`, and `detach`.
- `assignee` is special: it is the membership relation between a subject and a role ("user X is an assignee of role Y"), not an action a route checks for.
- `attach`/`detach` authorize linking two resources together (e.g. assigning a role to a service account).
### Kind
A `Kind` is the fine-grained name of your resource — `dashboard`, `rule`, `quick-filter`. Kinds are registered in [pkg/types/coretypes/registry_kind.go](/pkg/types/coretypes/registry_kind.go). A kind rides on top of an existing type, so adding one does **not** require any OpenFGA schema change.
### Resource
A `Resource` ties a type and a kind together and knows how to render itself as an FGA object string. The interface lives in [pkg/types/coretypes/resource.go](/pkg/types/coretypes/resource.go):
```go
type Resource interface {
Type() Type
Kind() Kind
Prefix(orgId valuer.UUID) string // metaresource:organization/<orgID>/dashboard
Object(orgId valuer.UUID, selector string) string
Scope(verb Verb) string // dashboard:read
AllowedVerbs() []Verb
}
```
All resources are registered in [pkg/types/coretypes/registry_resource.go](/pkg/types/coretypes/registry_resource.go) using constructors like `NewResourceMetaResource(KindDashboard)` or `NewResourceTelemetryResource(KindLogs)`.
### Selector
A `Selector` identifies *which* instance(s) of a resource a check is about — a UUID, a role name, or the wildcard `*`. A `SelectorFunc` maps the extracted resource ID to selectors at request time. Two prebuilt ones cover most routes ([pkg/types/coretypes/selector.go](/pkg/types/coretypes/selector.go)):
- `WildcardSelector` — the check is against all instances of the resource (`create`, `list`).
- `IDSelector` — the check is against the specific instance *or* the wildcard (`read`, `update`, `delete` of one object). A subject authorized on `*` is authorized on every instance.
When the ID in the request is not what FGA needs (e.g. routes receive a role UUID but FGA objects use role names), write a custom `SelectorFunc` — see `roleSelector` in [pkg/apiserver/signozapiserver/serviceaccount.go](/pkg/apiserver/signozapiserver/serviceaccount.go).
### Roles, transactions, and tuples
SigNoz ships four managed roles, declared in [pkg/types/coretypes/registry_managed_role.go](/pkg/types/coretypes/registry_managed_role.go): `signoz-admin`, `signoz-editor`, `signoz-viewer`, and `signoz-anonymous`. Their permissions are declared in code as `Transaction`s (a verb on an object) in `ManagedRoleToTransactions` — this map is the single source of truth for what each managed role can do.
At organization bootstrap, `CreateManagedRoles` and `CreateManagedUserRoleTransactions` (see [pkg/authz/authz.go](/pkg/authz/authz.go)) persist the role rows and write one OpenFGA tuple per transaction, linking `role:organization/<orgID>/role/<name>#assignee` to each permitted object. Users and service accounts are then granted roles via `Grant`/`Revoke`, which writes `assignee` tuples. Custom roles (enterprise) are managed through the roles API in [pkg/authz/signozauthzapi/handler.go](/pkg/authz/signozauthzapi/handler.go).
### Schema
The OpenFGA authorization model is a hand-written DSL, embedded at build time: [pkg/authz/openfgaschema/base.fga](/pkg/authz/openfgaschema/base.fga) for community and [ee/authz/openfgaschema/base.fga](/ee/authz/openfgaschema/base.fga) for enterprise. The community model only supports role assignment; the enterprise model defines per-verb relations on every type, enabling genuine per-resource checks. This split is why `CheckWithTupleCreation` behaves differently per edition (see [How does a check work at runtime?](#how-does-a-check-work-at-runtime)). You only touch these files when introducing a brand-new **type** — never for a new kind.
## How do I add authz to my feature?
### 1. Register the kind
Add your kind in [pkg/types/coretypes/registry_kind.go](/pkg/types/coretypes/registry_kind.go) and append it to `Kinds`:
```go
KindThing = MustNewKind("thing")
```
### 2. Register the resource
Add the resource in [pkg/types/coretypes/registry_resource.go](/pkg/types/coretypes/registry_resource.go) and append it to `Resources`:
```go
ResourceMetaResourceThing = NewResourceMetaResource(KindThing)
```
Pass an explicit verb list to `NewResourceMetaResource` only if your resource supports fewer verbs than the type default.
### 3. Grant permissions to managed roles
Decide what each managed role can do with your resource and add the transactions in [pkg/types/coretypes/registry_managed_role.go](/pkg/types/coretypes/registry_managed_role.go):
```go
// thing — editors manage, viewers read
{Verb: VerbCreate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindThing}, WildCardSelectorString)},
```
The convention so far: admin gets everything, editor gets CRUD on day-to-day observability resources, viewer gets `read`/`list`, anonymous gets nothing (except public dashboards).
### 4. Wire the route
Register the route in [pkg/apiserver/signozapiserver](/pkg/apiserver/signozapiserver), wrapping your handler with `CheckResources` and declaring what the route touches via a `ResourceDef` ([pkg/http/handler/resourcedef.go](/pkg/http/handler/resourcedef.go)). A complete example from [pkg/apiserver/signozapiserver/serviceaccount.go](/pkg/apiserver/signozapiserver/serviceaccount.go):
```go
router.Handle("/api/v1/service_accounts", handler.New(
provider.authzMiddleware.CheckResources(provider.serviceAccountHandler.Create, authtypes.SigNozAdminRoleName),
handler.OpenAPIDef{
ID: "CreateServiceAccount",
// ...
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceServiceAccount.Scope(coretypes.VerbCreate)}),
},
handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceServiceAccount,
Verb: coretypes.VerbCreate,
Category: coretypes.ActionCategoryAccessControl,
ID: coretypes.ResponseJSONPath("data.id"),
Selector: coretypes.WildcardSelector,
}),
)).Methods(http.MethodPost)
```
The pieces:
- **`CheckResources(handlerFn, roles...)`** — the resource-aware authorization wrapper from [pkg/http/middleware/authz.go](/pkg/http/middleware/authz.go). The role list is the community-edition fallback: which managed roles may call this route when per-resource checks are unavailable.
- **`ResourceDef`** — declares the resource, verb, audit category, how to extract the instance ID, and how to turn that ID into selectors. ID extractors live in [pkg/types/coretypes/extractor.go](/pkg/types/coretypes/extractor.go): `PathParam("id")`, `BodyJSONPath("data.id")`, `BodyJSONArray("ids")`, and `ResponseJSONPath("data.id")` for IDs only known after the handler runs (e.g. `create`).
- **`SecuritySchemes`** — advertises the required scope (`resource.Scope(verb)`, e.g. `serviceaccount:create`) in the OpenAPI spec.
For routes that link two resources, use `AttachDetachSiblingResourceDef` (both sides are authz-checked, e.g. attaching a role to a service account requires `attach` on **both** the service account and the role). For parent-child routes (e.g. creating an API key under a service account), both sides are checked too, but with different verbs: declare a `BasicResourceDef` checking the child with `create`/`delete`, alongside an `AttachDetachParentChildResourceDef` checking the parent with `attach`/`detach` (within that def the child is only recorded for audit) — see the `/api/v1/service_accounts/{id}/keys` route in [pkg/apiserver/signozapiserver/serviceaccount.go](/pkg/apiserver/signozapiserver/serviceaccount.go).
Prefer `CheckResources` with a `ResourceDef` for anything resource-shaped. The older coarse gates `ViewAccess`/`EditAccess`/`AdminAccess` only check "does the caller hold one of these roles" and give up per-resource granularity; `OpenAccess` performs no authorization (authentication still applies); `CheckWithoutClaims` serves anonymous routes such as public dashboards.
### 5. Backfill existing organizations (only if needed)
Managed-role tuples are written from the registry at organization creation, so **new resources need no migration for new organizations**. If existing organizations must get the new permissions, add a migration in [pkg/sqlmigration](/pkg/sqlmigration) that inserts the tuples — see [083_add_role_crud_tuples.go](/pkg/sqlmigration/083_add_role_crud_tuples.go) for the pattern.
## How does a check work at runtime?
1. The resource middleware ([pkg/http/middleware/resource.go](/pkg/http/middleware/resource.go)) runs on every request. It reads the matched handler's `ResourceDef`s, extracts the resource IDs from path/body, and stores the resolved resources in the request context.
2. `CheckResources` reads them back, runs each `SelectorFunc`, and calls `AuthZ.CheckWithTupleCreation(ctx, claims, orgID, relation, resource, selectors, roleSelectors)`.
3. What happens next depends on the edition:
- **Community** ([pkg/authz/openfgaserver/server.go](/pkg/authz/openfgaserver/server.go)) ignores the resource and selectors and only checks whether the subject is an `assignee` of one of the allowed roles — a plain role gate.
- **Enterprise** ([ee/authz/openfgaserver/server.go](/ee/authz/openfgaserver/server.go)) builds tuples via `authtypes.NewTuples` — subject `user:organization/<orgID>/user/<userID>`, relation `create`, object `serviceaccount:organization/<orgID>/serviceaccount/*` — and batch-checks them against OpenFGA: genuine per-resource authorization, including custom roles.
Because both paths go through the same middleware and the same `ResourceDef` declarations, a route wired once works correctly in both editions.
## What should I remember?
- Declare authz in the registries ([pkg/types/coretypes](/pkg/types/coretypes)), not in migrations — tuples for new organizations are derived from code at bootstrap.
- A new kind never needs an OpenFGA schema change; only a new type does, and then **both** [pkg/authz/openfgaschema/base.fga](/pkg/authz/openfgaschema/base.fga) and [ee/authz/openfgaschema/base.fga](/ee/authz/openfgaschema/base.fga) must be updated together.
- Prefer `CheckResources` + `ResourceDef` over the coarse `ViewAccess`/`EditAccess`/`AdminAccess` gates for new routes.
- Use `WildcardSelector` for `create`/`list`, `IDSelector` for instance operations, and a custom `SelectorFunc` when the request ID is not the FGA selector.
- Linking routes check **both** sides: peers via `AttachDetachSiblingResourceDef` (same verb on both), parent-child via a `BasicResourceDef` on the child (`create`/`delete`) paired with an `AttachDetachParentChildResourceDef` on the parent (`attach`/`detach`).
- Changing `ManagedRoleToTransactions` only affects organizations created afterwards — add a [pkg/sqlmigration](/pkg/sqlmigration) migration to backfill existing ones.

View File

@@ -11,7 +11,6 @@ We adhere to three primary style guides as our foundation:
We **recommend** (almost enforce) reviewing these guides before contributing to the codebase. They provide valuable insights into writing idiomatic Go code and will help you understand our approach to backend development. In addition, we have a few additional rules that make certain areas stricter than the above which can be found in area-specific files in this package:
- [Abstractions](abstractions.md) - When to introduce new types and intermediate representations
- [Authz](authz.md) - Authorization, roles, and access control
- [Errors](errors.md) - Structured error handling
- [Endpoint](endpoint.md) - HTTP endpoint patterns
- [Flagger](flagger.md) - Feature flag patterns

View File

@@ -61,37 +61,31 @@ type Channel struct {
```go
type AuthDomain struct {
storableAuthDomain *StorableAuthDomain
storableAuthDomainConfig *StorableAuthDomainConfig
storableAuthDomain *StorableAuthDomain
authDomainConfig *AuthDomainConfig
}
type StorableAuthDomain struct {
bun.BaseModel `bun:"table:auth_domain"`
types.Identifiable
Name string `bun:"name"`
Data string `bun:"data"` // StorableAuthDomainConfig serialized as JSON
Data string `bun:"data"` // AuthDomainConfig serialized as JSON
OrgID valuer.UUID `bun:"org_id"`
types.TimeAuditable
}
type PostableAuthDomain struct {
Name string `json:"name" required:"true"`
Enabled bool `json:"enabled"`
Config AuthDomainConfig `json:"config" required:"true"`
RoleMapping *RoleMapping `json:"roleMapping"`
Config AuthDomainConfig `json:"config"`
Name string `json:"name"`
}
type UpdatableAuthDomain struct {
Enabled bool `json:"enabled"` // Name intentionally absent
Config AuthDomainConfig `json:"config" required:"true"`
RoleMapping *RoleMapping `json:"roleMapping"`
type UpdateableAuthDomain struct {
Config AuthDomainConfig `json:"config"` // Name intentionally absent
}
type GettableAuthDomain struct {
StorableAuthDomain
Enabled bool `json:"enabled"`
Config AuthDomainConfig `json:"config"`
RoleMapping *RoleMapping `json:"roleMapping"`
*StorableAuthDomain
*AuthDomainConfig
AuthNProviderInfo *AuthNProviderInfo `json:"authNProviderInfo"`
}
```
@@ -99,74 +93,11 @@ type GettableAuthDomain struct {
Each flavor exists for a concrete reason:
- `StorableAuthDomain` stores the typed config as an opaque `Data string` column, so the schema does not need to migrate every time a config field is added.
- `PostableAuthDomain` carries the config as a structured object (not a string) for the request; `AuthDomainConfig` is a kind/spec envelope.
- `UpdatableAuthDomain` excludes `Name` because a domain's name cannot change after creation.
- `PostableAuthDomain` carries the config as a structured object (not a string) for the request.
- `UpdateableAuthDomain` excludes `Name` because a domain's name cannot change after creation.
- `GettableAuthDomain` adds `AuthNProviderInfo`, which is derived at read time and never persisted.
The core `AuthDomain` holds the two live halves — `storableAuthDomain` and `storableAuthDomainConfig` — and owns business methods such as `Update(updatable)` and `Patch(patchable)`. Conversions use the `New<Output>From<Input>` form: `NewAuthDomainFromPostableAuthDomain`, `NewAuthDomainFromStorableAuthDomain`, `NewGettableAuthDomainFromAuthDomain`.
## Sum types: the kind/spec envelope
When a domain type is a *sum type* — exactly one of several variants, selected by a discriminator — model it as an envelope with a `kind` and a `spec`:
```go
type FooConfig struct {
Kind FooKind `json:"kind" required:"true"`
Spec any `json:"spec" required:"true"`
}
```
```json
{ "kind": "bar", "spec": { "url": "...", "timeout": "30s" } }
```
`Kind` is a `valuer.String` enum implementing `Enum()`; `Spec` holds exactly one concrete variant type (`BarSpec`, `BazSpec`, …). `RuleThresholdData` and `EvaluationEnvelope` in `pkg/types/ruletypes/` are the canonical in-tree examples; the dashboard panel/query/variable plugins in `pkg/types/dashboardtypes/` are the same pattern behind generics. (`QueryEnvelope` in querybuildertypes uses `type` as the discriminator key for historical reasons; new envelopes use `kind`.)
### The envelope goes at the point of variance, not the resource root
Put the envelope on the field that actually varies. The resource root is almost never a sum type — a `Foo` has a `name` and an `enabled` flag regardless of which kind it is configured with; only its configuration varies, so the envelope is the `config` field:
```json
{ "name": "my-foo", "enabled": true, "config": { "kind": "bar", "spec": { "...": "..." } } }
```
Hoisting `kind`/`spec` to the root would turn the whole resource into a `oneOf`: every flavor (`PostableFoo`, `UpdatableFoo`, `GettableFoo`) then needs one variant schema per kind, each repeating the common fields; every new common field has to be added to all of them; and generated clients get unions of large objects instead of one small union that narrows on `config.kind`. A root-level `kind` also collides with the resource-model meaning of the word — root `kind` conventionally answers "what resource is this" (`Dashboard`), never "which flavor of config does it hold".
The existing domains already follow this placement:
- **Rules** — plain root; envelopes on the varying fields: `thresholds: {kind, spec}` and `evaluation: {kind, spec}`.
- **Dashboards** — metadata at the root plus one typed `spec`; the unions sit deep inside, at each panel/query/variable plugin (`{kind, spec}` in `perses_plugin_wrappers.go`).
- **Saved views** — root `{schemaVersion, spec}`, where `spec` is a *versioning* envelope holding one fixed type, not a union; the unions are inside it (`spec.queries: [{type, spec}]`). Same word, different job — a versioned body is not a discriminated union.
### Why this tagging style
Of the union encodings in common use, the envelope is the *adjacently tagged* one — tag and payload side by side. Variant payloads stay collision-free, and each kind maps to a named wrapper schema that carries the discriminator, which is exactly what OpenAPI generators need. The alternatives lose on those points: *internally tagged* (`{"kind": "bar", ...fields flattened}`) mixes common and variant fields, admits cross-variant key collisions, and forces every variant schema to redeclare the discriminator; *sibling optional fields* (`{"kind": "bar", "barConfig": {}, "bazConfig": {}}`) is the anti-pattern the first rule below exists to prevent.
The rules that make the envelope work:
- **Never model variants as sibling fields.** A struct with `Bar *BarSpec`, `Baz *BazSpec` next to a discriminator cannot be expressed as an OpenAPI discriminated union, forces nilability checks on every consumer, and silently admits contradictory payloads (kind=bar with a baz spec). The chosen variant *is* the payload.
- **The envelope owns `UnmarshalJSON`.** Decode `kind` first, then switch on it to decode and validate the matching concrete type into `Spec`. Unknown kinds and missing specs are rejected at the boundary:
```go
func (typ *FooConfig) UnmarshalJSON(data []byte) error {
var raw map[string]json.RawMessage
// ... unmarshal raw, decode raw["kind"] ...
switch kind {
case FooKindBar:
spec := BarSpec{}
if err := json.Unmarshal(raw["spec"], &spec); err != nil {
return err
}
typ.Spec = spec
// ... one case per kind, default rejects ...
}
typ.Kind = kind
return nil
}
```
- **Consumers type-assert on `Spec`** (`config.Spec.(BarSpec)`) after switching on `Kind`. If assertion sites multiply, add typed accessors on the envelope (see `EvaluationEnvelope.GetEvaluation()`).
- **OpenAPI needs one unexported variant struct per kind** (`fooConfigBar{Kind; Spec BarSpec}`), exposed via `JSONSchemaOneOf()` and mapped via `PrepareJSONSchema` with the `x-signoz-discriminator` extension. The schema mechanics are covered in [handler.md](handler.md#oneof-with-a-discriminator).
- **A legacy persisted shape gets a data migration or a `StorableX`.** When rows were written before the envelope existed, prefer an idempotent `sqlmigration` that rewrites them into the new shape, so the storable type simply nests the envelope. Only when the old shape must keep being written (external writers, rollback windows) keep it in a storable twin and convert at the type boundary.
The core `AuthDomain` holds the two live halves — `storableAuthDomain` and `authDomainConfig` — and owns business methods such as `Update(config)`. Conversions use the `New<Output>From<Input>` form: `NewAuthDomainFromConfig`, `NewAuthDomainFromStorableAuthDomain`, `NewGettableAuthDomainFromAuthDomain`.
## Conventions that tie the flavors together
@@ -208,8 +139,6 @@ Both are optional. Do not introduce them if `PostableX` already covers the case.
- Every domain package defines the core type `X`. Only `X` is mandatory.
- Add `PostableX` / `GettableX` / `UpdatableX` / `StorableX` one at a time, only when the shape actually diverges from `X`.
- Model sum types as a `{kind, spec}` envelope with a validating `UnmarshalJSON` — never as sibling variant fields next to a discriminator.
- The envelope goes on the field that varies, never at the resource root — common fields stay on the resource, outside the union.
- Domain logic lives on `X`, not on the flavor types.
- Conversions can be a `New<Output>From<Input>` constructor or a receiver-style `ToY()` method — pick whichever reads best at the call site.
- Use a type alias when two shapes are truly identical.

View File

@@ -1,381 +0,0 @@
# PromQL Serving — clickhouseprometheusv2
This document gives the context for `pkg/prometheus/clickhouseprometheusv2`.
This package is the second-generation ClickHouse-backed Prometheus provider.
The document tells you why the package exists. It tells you the correctness
rules that shaped it. It shows how we prove that each construct does not
change results. Keep these invariants when you change the provider. If your
change breaks an invariant, flag it and discuss it first.
---
## Why a second provider
The v1 provider (`pkg/prometheus/clickhouseprometheus`) serves the promql
engine through the remote-read protobuf adapter. It fetches every raw sample
of a query's union window. It serializes all of them and gives them to the
engine. The cost follows the ingested data, not the question. This is how a
dashboard of PromQL panels can take an instance down.
In v2, each query runs in one of two ways. The classifier decides per query:
- **Transpiled**: ClickHouse evaluates the query. Only final (or near-final)
per-group grid arrays come back. The statements use the
`timeSeries*ToGrid` aggregate functions. The supported ClickHouse floor is
25.6 or later, so these functions are assumed available.
- **Engine**: the stock promql engine evaluates over this package's native
`storage.Querier`. Every shape that does not transpile takes this path.
**The core rule: a PromQL result that differs from upstream Prometheus is a
lost user. A construct that cannot reproduce engine semantics exactly falls
back. It does not approximate.** The conformance suite
(`tests/integration/tests/promqlconformance/`) replays Prometheus' own test
corpus against both providers. It is the arbiter. The classification golden
(`testdata/classification_golden.json`) freezes the route of each corpus
expression. The rest of this document is the PromQL-to-SQL story. That
mapping is where correctness is won or lost.
---
## The evaluation model the SQL must reproduce
A PromQL range query is an instant query evaluated at each grid point
`t_i = start + i*step`, for `i = 0..(end-start)/step`. At each `t_i`:
- An instant selector resolves to the latest sample in the left-open
lookback window `(t_i - lookback, t_i]`. If that latest sample is a stale
marker, the selector resolves to nothing. Older real samples in the window
do not change this.
- A range selector `[r]` collects every sample in `(t_i - r, t_i]`. Stale
markers are excluded.
- `offset d` shifts both windows to `(t_i - d - w, t_i - d]`.
The transpilation invariant follows from this model. Each transpiled
construct produces one array per output series. The array has exactly one
slot per grid point. Slot `i` holds the value at `t_i`. NULL means absent.
This makes composition correct, not only convenient. The engine evaluates
these operators independently per `t_i`. A representation that gets every
slot right gets the whole query right. Spatial aggregation over arrays is
sound because it combines values that belong to the same `t_i` by
construction. Scan time maps slot `i` back to `t_i = start + i*step`
(`toMatrix`). The sections below fill those slots with exactly the numbers
the engine computes. We validated each equivalence against the vendored
engine on live data before its shape entered the allowlist. An unproven
shape stays on the engine path.
## Classification: finding what a statement can answer
`classify` walks the parsed AST and looks for "core units". A core unit is a
maximal subtree of this shape:
[agg by/without (...)] [fn(] selector[range] [offset d] [)] [op scalar]...
`classifyCore` peels that chain from the outside in. It takes an optional
sum/min/max/avg/count aggregation. It then takes one allowlisted function or
a bare instant selector. It then takes the selector with its offset. On the
way out, it collects number-literal arithmetic, comparisons (including
`bool`), and unary minus into a scalar-op pipeline. A node qualifies only if
its type, arguments, and children are in the proven set. This is an
allowlist. An overlooked construct becomes a fallback, not a wrong number.
Three unit kinds come out. Each kind has its own SQL form:
- `unitRange`: rate, irate, increase, delta, idelta over a range selector.
- `unitInstant`: instant vector selection, bare or comparison-filtered.
- `unitOverTime`: avg/min/max/sum/count/last `_over_time`.
If the whole tree is one unit, the plan is "full". The statement's rows are
the query result. Otherwise, `rewrite` cuts out each maximal unit and puts a
synthetic selector `__signoz_transpiled_N__` in its place. The engine then
runs the rewritten expression over the units' materialized results. This is
a "hybrid" plan. `histogram_quantile`, `topk`, `or`/`and`/`unless`, and
vector matching keep exact engine semantics. Their expensive inputs were
aggregated server-side.
Classification refuses a shape when it cannot guarantee exact semantics
server-side:
- The `@` modifier, anywhere.
- Default-resolution subqueries. Their resolution is a server runtime
setting that the transpiler cannot see.
- Duration expressions (`offset step()`, `[range()]`, ...), anywhere. The
engine resolves them into the selector's static fields only at evaluation
time. At classification time those fields hold zero values. A transpile
would silently use the wrong offset or range.
- Steps or ranges that are not whole seconds. The grid functions take
whole-second parameters.
- Grouping by `__name__`, or matching on it, in hybrid plans. The synthetic
name would leak into results.
- Name-keeping units in hybrid plans. Bare and comparison-filtered instant
selectors and `last_over_time` keep their real `__name__` (`keepsName`).
Substitution would replace that name. These units transpile only as full
plans.
- Every function outside the allowlist: changes, resets,
quantile_over_time, absent, native-histogram functions, and more.
Units inside a fixed-resolution subquery evaluate on the subquery's own
grid, not the query grid. That grid is the set of epoch-aligned multiples of
the resolution strictly after `outerStart - offset - range`, ending at
`outer end - offset`. This is the exact derivation the engine uses. A grid
shifted by one step changes which samples every window sees.
## From one unit to one statement
`buildUnitSQL` renders each unit as one statement. For
`sum by (pod) (rate(m{job="api"}[5m]))` the skeleton is:
SELECT g0, sumForEach(grid) AS grid FROM (
SELECT any(series.g0) AS g0,
timeSeriesRateToGrid(<start>, <end>, <step>, <range>)(fromUnixTimestamp64Milli(unix_milli), value) AS grid
FROM signoz_metrics.distributed_samples_v4 AS points
INNER JOIN (
SELECT fingerprint, JSONExtractString(labels, 'pod') AS g0
FROM signoz_metrics.time_series_v4
WHERE <series predicates>
GROUP BY fingerprint, g0
) AS series ON points.fingerprint = series.fingerprint
WHERE metric_name = ? AND temporality IN ['Cumulative', 'Unspecified']
AND unix_milli > <start - range> AND unix_milli <= <end>
AND bitAnd(flags, 1) = 0
GROUP BY points.fingerprint
) GROUP BY g0
SETTINGS allow_experimental_ts_to_grid_aggregate_function = 1
Read it from the inside out.
**The time window** is the selector's semantics, verbatim. Strict `>` on the
lower bound and `<=` on the upper bound is the left-open `(t - w, t]` rule.
The offset shifts the whole window. `bitAnd(flags, 1) = 0` drops stale
markers. PromQL excludes them from range vectors.
**The inner GROUP BY** computes one grid array per series.
`timeSeriesRateToGrid(start, end, step, range)` is a parametric aggregate.
It takes (timestamp, value) pairs and produces `Array(Nullable(Float64))`
with one slot per grid point. It is correct because it implements the
engine's `extrapolatedRate`, decision for decision: counter resets, the
zero-point clamp, the extrapolation thresholds, the two-samples rule, and
the left-open window. We verified this: we fed identical samples to both and
compared slot for slot. The only observed difference is the last bit.
ClickHouse's C++ and Go round the same formula differently. That is the
floating-point floor, not a semantic gap. irate/delta/idelta map to their
own `timeSeries*ToGrid` functions, with the same verification. `increase`
has no function of its own. We emit
`arrayMap(x -> x * <range seconds>, <rate expr>)`. This is exact by
definition: `extrapolatedRate` computes the same extrapolated delta for both
and divides by the range only when `isRate`. The multiplication reverses it
exactly. The grid parameters render as literals, not bound args. They are
aggregate-function parameters. The experimental gate rides as a SETTINGS
clause on the statement itself, so telemetrystore hooks cannot remove it.
The group key is functionally dependent on the fingerprint: one fingerprint
is the hash of one labelset. So the inner query groups by the fingerprint
alone and reads the key columns with `any()`. This is exact, and it makes
the per-row hash key smaller.
**The join** gives each series its group key, in one of two forms.
`by (...)` extracts each listed label as a plain column
(`JSONExtractString(labels, 'pod') AS g0`) and groups on the columns. The
projection is a known short list, and the label names live in Go. To build,
sort, and stringify every label pair per row would be waste. This is correct
because column-tuple equality is label-set equality on the projection. An
extracted `''` means the label is absent. That is Prometheus semantics for
`by()` over missing labels. The empties are skipped when the columns turn
back into labels. `without` and no-aggregation project a label set that
varies per series. They get the canonical key: `toJSONString` of the sorted
[label, value] pairs that the unit projects. `without` excludes the listed
labels plus `__name__`. No-aggregation keeps everything; the name comes off
in Go, per the engine's name-dropping rules. Here the sort is load-bearing.
Stored JSON key order is not canonical across fingerprints. Two orderings of
the same labels must land in one group. Empty values are filtered for the
same absent-label reason. The same string parses back into the output label
set (`labelsFromGroupKey`).
**The outer GROUP BY** is the spatial aggregation. sum/min/max/avg/count
by/without become the `-ForEach` combinators. Element-wise aggregation over
grid arrays is the engine's per-`t_i` aggregation: slot `i` of every input
array refers to the same `t_i`. The combinators skip NULLs. That is the
engine aggregating only the series present at `t_i`. An index where every
series is absent stays NULL. Two edges need explicit handling. First,
`countForEach` wraps in a map of 0 back to NULL. A count over an all-absent
index is an absent point, not 0. Second, a unit without aggregation still
passes through `maxForEach`. That is the identity for the common
one-fingerprint group. It is a deterministic NULL-skipping merge when a
regex `__name__` selector collapses distinct metrics onto one projected
label set. One caveat is inherent: the summation order over series differs
from the engine's. Spatial aggregates can differ in the last ULP. Float
addition is not associative. No ordering reproduces the engine's result
bit-exactly from inside a GROUP BY.
## Instant selectors: staleness needs two aggregates
`unitInstant` uses window = lookback. It must reproduce the shadowing rule:
the point is absent when the latest in-window sample is a stale marker.
`timeSeriesLastToGrid` alone cannot express that. To skip stale rows in
WHERE would resurrect the older real sample that the marker buried. So stale
rows stay in the scan for this kind only. The grid expression compares three
aggregates per slot:
arrayMap((tall, tok, vok) -> if(tall IS NULL OR tok IS NULL OR tall != tok, NULL, vok),
timeSeriesLastToGrid(...)(ts, toFloat64(unix_milli)), -- last sample overall
timeSeriesLastToGridIf(...)(ts, toFloat64(unix_milli), bitAnd(flags, 1) = 0), -- last non-stale, its timestamp
timeSeriesLastToGridIf(...)(ts, value, bitAnd(flags, 1) = 0)) -- last non-stale, its value
This is correct by cases on a slot's window. No samples at all: both
timestamp aggregates are NULL, so the slot is NULL. That is absent, as the
engine says. Latest sample non-stale: it is the latest overall and the
latest non-stale. The timestamps agree. The slot takes its value. That is
the engine's pick. Latest sample stale: the last-overall timestamp is the
marker's. The last-non-stale timestamp is older, or NULL when the window
holds only markers. They disagree. The slot is NULL. The marker shadows,
exactly as the engine's rule says. Timestamps are unique per series (ingest
dedups). So timestamp equality identifies "the same sample" without
ambiguity. We probed the `-If` combinator against these experimental
aggregates before we trusted it.
## Windowed *_over_time: whole buckets instead of a grid function
avg/min/max/sum/count `_over_time` aggregate every raw sample in the window.
No `timeSeries*ToGrid` function computes them. (`last_over_time` is the
exception. The last sample of a range vector is exactly
`timeSeriesLastToGrid`. PromQL excludes stale markers from range vectors; we
exclude them in WHERE.) These shapes transpile only when the range is a
whole multiple of the step. Then the window needs no per-sample fan-out.
With `W = range/step`, the window `(t_k - range, t_k]` is exactly the union
of W step buckets. Both are left-open on the same boundaries. So bucket
membership fully determines window membership. Each sample lands in exactly
one bucket:
intDiv(unix_milli - <start> + <range> - 1, <step>)
This is `ceil((ts - start)/step)` shifted by W-1, so the earliest in-window
sample sits at 0. Slot k's window is buckets in `[k, k+W-1]`. The
alternative fans each sample into all W windows that cover it. That
multiplies rows by W. For a long range over a short step, that is a row
explosion measured in billions. The bucketed form's row count is
series × buckets: the size of the output, for any W.
Each series aggregates in one group. The `-Resample` combinator
(`sumResample`, `countResample`) holds the dense per-bucket partials inside
one group state: a bucket count, plus the function's value aggregate (sum
for sum/avg, min, max). An earlier form grouped by (series, bucket) and
assembled with `groupArrayInsertAt`. At scale that made 37M hash groups, and
per-thread partials scaled memory with the thread count. The slide then
combines each slot's at-most-W bucket partials by direct aggregation
(`arraySum(arraySlice(...))`). Window sums are added the way the engine adds
them. There is no prefix-sum differencing: its large-minus-large
cancellation would drift past the shadow tolerance on counter-sized values.
This is correct per slot because the bucket union is the exact window
multiset, and avg/min/max/sum/count are order-insensitive on a multiset
(sum/avg up to summation order; see the float caveat above). A slot with
zero window count is absent. min/max filter their slices on the bucket
counts. An empty bucket's default can never look like a value: a real sample
can legitimately be +Inf.
Two shapes fall back to the engine path, which is exact: a range that does
not divide the step, and a window wider than `maxWindowBuckets` buckets (the
slide costs W combines per slot). A range narrower than the step needs
neither gate: the windows are pairwise disjoint, one bucket per slot, no
slide. That form is exact only together with the window-sliver predicate
below.
## Scalar ops, full plans, hybrid plans
The scalar-op pipeline runs in Go on the returned arrays
(`applyScalarOps`), slot by slot. Arithmetic operators compute. Comparisons
filter: the slot keeps the vector-side value or becomes NULL. Under `bool`
they return 0/1. This is trivially correct. It is the same float64 operation
the engine applies, to the same slot value, in the same operator order the
AST dictates. Go instead of another SQL layer changes where, not what.
A full plan's arrays map straight to the result matrix. A hybrid plan
materializes each unit's arrays as synthetic series under its
`__signoz_transpiled_N__` name. The engine evaluates the rewritten
expression over a storage that serves synthetic names from memory and
everything else live. Substitution is sound because a unit's output is a
plain instant vector to the engine: same values at same timestamps, under a
different name. The name cannot matter. Plans that group by or match on
`__name__` were refused at classification. Name-keeping units are never
substituted. One subtlety makes it exact: we write stale markers at absent
grid points. Without them, the engine's lookback would resurrect a point
from up to `lookback` earlier. The marker encodes "absent here" the way the
engine itself encodes it. Units evaluate concurrently. Each unit is one
grid statement: the group-key join resolves the matchers, and the samples
primary key takes the metric name straight from the selector. Only a
selector without a static `__name__` runs the series lookup first, to learn
the concrete metric names. A step of 0 is an instant query: a single
evaluation at `end`.
A note on the window sliver: when the window is narrower than the step, the
grid windows cover only `window/step` of the timeline. A sample in a gap
belongs to no window. It cannot move any grid point, but the grid aggregate
would buffer it. A WHERE predicate keeps only the in-window rows:
`positiveModulo(selStart - unix_milli, step) < window`, with the scan capped
at the last grid point. The lattice anchors at the selector start, because
the end can sit off-lattice on unaligned grids. This cut a 36k-series
one-week rate from 74s/28GiB to 16s/4.3GiB on fleet data. Over slivered
rows, `timeSeriesLastToGrid`'s window widening is harmless, so instant
selectors and `last_over_time` transpile at window < step too.
## Series lookup
The engine path resolves matchers once per selector (`selectSeries`); the
transpiled path builds the same conditions into its group-key join. Both
read the same tables. The series tables hold one row per (fingerprint, bucket)
at 1h/6h/1d/1w granularities. The shared schema package
(`pkg/telemetryschema/metricstelemetryschema`) picks the table whose bucket
fits the window. It rounds the window start down to the bucket boundary, so
a window that begins mid-bucket still matches the bucket's row. How matchers
become SQL, and why regexes are anchored, is documented at
`applySeriesConditions`. Empty-valued labels come off at this boundary. An
empty value means "label absent" in Prometheus, but stored attribute JSON
can carry them.
## The engine path
Queries that do not transpile run in the stock engine over this package's
`storage.Querier`. This is still not the v1 path. Samples are fetched per
selector with the engine's per-selector hints, not the query-wide union
window. So `foo / foo offset 1d` reads two narrow windows, not the widest
one twice. Instant selectors of subquery-free queries fetch only the last
sample per step bucket (`lastSamplePerStep`). Buckets anchor at the
selector's first evaluation timestamp. The code recovers it from the hints
as `hints.Start + lookback - 1ms`, the inverse of how the engine derives
`hints.Start`. Bucket boundaries then coincide with evaluation timestamps.
A non-final sample of a bucket can never be the latest sample in
`(t - lookback, t]` for any grid `t`. Real timestamps are preserved, so the
engine's own lookback and staleness handling stay exact. Range selectors
always fetch raw: every sample feeds the range function. The subquery-free
proof travels in the context as `prometheus.QueryTraits`. Subquery selectors
evaluate at the subquery's step, while the hints carry the top-level step.
Row assembly maps stale flags to the engine's StaleNaN. It merges series
with identical label sets (`sortAndMerge`): the engine assumes storages
never emit duplicates.
## Sharding
`samples_v4` and `time_series_v4` (and all their rollups) shard on the same
key: `cityHash64(env, temporality, metric_name, fingerprint)`. So a series'
samples and catalog rows live on the same shard. The transpiled statement
exploits that. The distributed samples table at the top-level FROM makes
ClickHouse rewrite the whole inner query per shard. The join against the
shard-local series table and the per-series grid aggregation run next to
the data. The initiator only merges aggregate states and applies the
spatial `-ForEach` step. This is the same layout as the telemetrymetrics
statement builder. The group-key join alone restricts the transpiled scan
to the matched series. The engine path's samples fetch restricts by the
same predicates as a shard-local semi-join, not a GLOBAL broadcast of the
matched set. The temporality filter on every samples statement is a
semantic no-op: the matched fingerprints already come from those
temporalities. It engages the leading samples primary-key column.
Delta-temporality series stay invisible to PromQL here, exactly as in v1.
The rollout gate is parity with v1. To make Delta visible is its own change
with its own semantics to design. A Delta stream fed to `rate()`
as-if-cumulative would be wrong, not just new.
## Observability
Every statement carries a `log_comment` with
`code.namespace=clickhouse-prometheus-v2` and `code.function.name` naming
the call site (`selectSeries`, `selectSamples`, `transpiledUnit`,
`LabelValues`, `LabelNames`). This provider's work is attributable in
`system.query_log` without guessing from query text.

View File

@@ -30,11 +30,11 @@ yarn install:browsers # one-time Playwright browser install
### Starting the Test Environment
To spin up the backend stack (SigNoz, ClickHouse, Postgres, ClickHouse Keeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
To spin up the backend stack (SigNoz, ClickHouse, Postgres, Zookeeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
```bash
cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --with-web \
uv run pytest --basetemp=./tmp/ -vv --reuse --with-web \
e2e/bootstrap/setup.py::test_setup
```
@@ -45,13 +45,8 @@ This command will:
- Start the HTTP seeder container (`tests/seeder/` — exposing `/telemetry/{traces,logs,metrics}` POST + DELETE)
- Write backend coordinates to `tests/e2e/.env.local` (loaded by `playwright.config.ts` via dotenv)
- Keep containers running via the `--reuse` flag
- Rebuild the SigNoz container from the current sources via the `--rebuild` flag
The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start; later builds are incremental.
### Rebuilding After Source Changes
The `--with-web` image bakes the built frontend in, so neither backend nor frontend changes are picked up while `--reuse` keeps the container running. `--rebuild` fixes that for both: it kills the SigNoz container, rebuilds the image incrementally (go build cache + pnpm store — a frontend-only change rebuilds in about a minute), and starts a fresh one while databases, mocks, migrations, and the seeder stay reused. The setup command above passes it, so the iteration loop is: change code → re-run the setup command → re-run your specs. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.
The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start.
### Stopping the Test Environment
@@ -112,41 +107,6 @@ These two folders look similar but mean different things:
Rule of thumb: if it's a `test.extend` fixture, put it in `fixtures/`. If it's a function you call explicitly (or a constant the function uses), put it in `helpers/`. If it's a static file the helpers read, put it in `testdata/`.
### Extended fixtures
For features needing complex setup (API-seeded data, ruler evaluation waits, cleanup), create domain-specific fixtures that extend `auth`. Group them in `fixtures/<domain>/`.
**Fixture scopes:**
- **test scope** — fresh data per test. Use for mutations (edit, delete, rename).
- **worker scope** — shared across tests in one worker. Use for read-only data. Worker scope pays the setup cost once per worker instead of once per test.
**The alerts pattern** (`fixtures/alerts/`) demonstrates extending fixtures:
```
fixtures/alerts/
├── alert-rules.ts # extends auth — worker-scoped rule list + test-scoped factory
└── alert-history.ts # extends alert-rules — adds history fixtures (waits on ruler)
```
Specs import from the fixture they need:
```ts
// List tests — just need rules, no history
import { test, expect } from '../../../fixtures/alerts/alert-rules';
// History tests — need history rows from ruler evaluation
import { test, expect } from '../../../fixtures/alerts/alert-history';
```
**When creating new fixtures:**
1. **Identify scope** — Will tests mutate the data? If yes, test-scoped. If read-only, worker-scoped.
2. **Group by domain** — Put fixtures in `fixtures/<domain>/`. Helpers in `helpers/<domain>/`.
3. **Extend existing fixtures** — Chain from `auth` or another fixture to inherit its setup.
4. **Handle timeouts** — Worker-scoped fixtures that wait on backend processing need explicit timeouts.
5. **Clean up** — Always delete seeded data in the fixture teardown (after `use()`).
6. **Extract logic into functions** — Keep the `test.extend()` block lean; move setup/teardown logic to named functions so the extend block reads as a manifest of "what fixtures exist."
Each spec follows these principles:
1. **Directory per feature**: `tests/e2e/tests/<feature>/*.spec.ts`. Cross-resource junction concerns (e.g. cascade-delete) go in their own file, not packed into one giant spec.
@@ -267,14 +227,11 @@ cd tests/e2e
# Single feature dir
npx playwright test tests/alerts/ --project=chromium
# Single sub-area
npx playwright test tests/alerts/history/ --project=chromium
# Single file
npx playwright test tests/alerts/page.spec.ts --project=chromium
npx playwright test tests/alerts/alerts.spec.ts --project=chromium
# Single test by title grep
npx playwright test --project=chromium -g "AL-01"
npx playwright test --project=chromium -g "TC-01"
```
### Iterative modes
@@ -308,14 +265,7 @@ yarn test:staging
| `SIGNOZ_E2E_PASSWORD` | Admin password. Bootstrap writes the integration-test default. |
| `SIGNOZ_E2E_SEEDER_URL` | Seeder HTTP base URL — hit by specs that need per-test telemetry. |
Precedence in `playwright.config.ts`, lowest to highest: `.env` (user-provided, staging) `.env.local` (bootstrap-generated, local mode) → whatever is already in `process.env`. The config parses both files itself and only fills in keys the environment does not already define, so exporting a variable always wins:
```bash
# runs against a locally served frontend, not whatever .env.local points at
SIGNOZ_E2E_BASE_URL=http://127.0.0.1:3301 pnpm test tests/alerts
```
This is deliberately not `dotenv.config({ override: true })`. That flag makes the *file* beat `process.env`, which silently discarded exported values — including the `SIGNOZ_E2E_BASE_URL` in `pnpm test:staging`, whenever a `.env.local` happened to exist.
Loading order in `playwright.config.ts`: `.env` first (user-provided, staging), then `.env.local` with `override: true` (bootstrap-generated, local mode). Anything already set in `process.env` at yarn-test time wins because dotenv doesn't touch vars that are already present.
### Playwright options
@@ -331,16 +281,13 @@ The full `playwright.config.ts` is the source of truth. Common things to tweak:
The same pytest flags integration tests expose work here, since E2E reuses the shared fixture graph:
- `--reuse` — keep containers warm between runs (required for all iteration).
- `--rebuild` — recreate the SigNoz container from the current sources (backend and, with `--with-web`, frontend) while the rest of the stack stays up. Requires `--reuse`.
- `--teardown` — tear everything down.
- `--clean` — prune the docker build caches, forcing the next image build to start cold.
- `--with-web` — build the frontend into the SigNoz container. **Required for E2E**; integration tests don't need it.
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/tests/integration.md`.
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/integration.md`.
## What should I remember?
- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build on a cold start; later builds are incremental.
- **Changed backend or frontend code? Re-run the setup command** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build; you only want to pay that once.
- **Don't teardown before setup.** `--reuse` correctly handles partially-set-up state, so chaining teardown → setup wastes time.
- **Prefer UI-driven flows.** Playwright captures BE requests in the trace; a parallel `fetch` probe is almost always redundant. Drop to `page.request.*` only when the UI can't reach what you need.
- **Use `page.waitForResponse` on UI clicks** to assert BE contracts — it still exercises the UI trigger path.

View File

@@ -37,34 +37,13 @@ make py-test-setup
Under the hood this runs, from `tests/`:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild --capture=no integration/bootstrap/setup.py::test_setup
uv run pytest --basetemp=./tmp/ -vv --reuse integration/bootstrap/setup.py::test_setup
```
This command will:
- Start all required services (ClickHouse, PostgreSQL, ClickHouse Keeper, SigNoz, Zeus mock, gateway mock)
- Start all required services (ClickHouse, PostgreSQL, Zookeeper, SigNoz, Zeus mock, gateway mock)
- Register an admin user
- Keep containers running via the `--reuse` flag
- Rebuild the SigNoz container from the current sources via the `--rebuild` flag
### Rebuilding After Source Changes
`--reuse` keeps the running SigNoz container, which means backend source changes are not picked up. `--rebuild` fixes exactly that: it kills the existing SigNoz container, rebuilds the image (incremental — only changed packages recompile thanks to the build cache), and starts a fresh one, while everything else (databases, mocks, migrations) stays reused. `make py-test-setup` passes it by default, so the iteration loop is simply:
```bash
make py-test-setup # (re)build signoz from your current sources
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
# ... edit backend code or tests ...
make py-test-setup # pick up the backend changes
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
```
The same applies to the e2e stack. `--rebuild` requires `--reuse` and cannot be combined with `--teardown` or `--clean`.
Some suites define their own SigNoz variant in a suite-local `conftest.py` (`create_signoz(..., cache_key=...)` — e.g. `basepath`, `metricreduction`, `querier_json_body`). Those containers are not touched by `make py-test-setup`, which only rebuilds the default instance. For such suites, pass `--rebuild` on the suite run itself — it rebuilds every SigNoz variant the run instantiates:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse --rebuild integration/tests/<suite>/
```
### Stopping the Test Environment
@@ -77,21 +56,11 @@ make py-test-teardown
Which runs:
```bash
uv run pytest --basetemp=./tmp/ -vv --teardown --capture=no integration/bootstrap/setup.py::test_teardown
uv run pytest --basetemp=./tmp/ -vv --teardown integration/bootstrap/setup.py::test_teardown
```
This destroys the running integration test setup and cleans up resources.
### Cleaning the Image Build Cache
The `signoz:integration` image build keeps its Go build and module caches in BuildKit cache mounts, so rebuilds only recompile what changed. These caches survive `--teardown` (they belong to the Docker builder, not to any container). If a cache ever needs to be nuked — suspected corruption, disk pressure, or to force a genuinely cold build — pass the `--clean` flag:
```bash
uv run pytest --basetemp=./tmp/ -vv --teardown --clean integration/bootstrap/setup.py::test_teardown
```
`--clean` prunes the docker build artifacts backing the incremental image build at session start, so the next build starts from a clean slate. Images and regular layer cache stay intact, but note the pruning is host-wide — it clears build caches for other projects too, not just SigNoz's. The flag composes with any invocation — passing it on a normal `--reuse` run simply makes the next image build start cold (~34 minutes instead of seconds).
## Understanding the Integration Test Framework
Python and pytest form the foundation of the integration testing framework. Testcontainers are used to spin up disposable integration environments. WireMock is used to spin up **test doubles** of external services (Zeus cloud API, gateway, etc.).
@@ -130,7 +99,7 @@ tests/
│ ├── passwordauthn/
│ ├── querier/
│ └── ...
└── e2e/ # Playwright suite (see docs/contributing/tests/e2e.md)
└── e2e/ # Playwright suite (see docs/contributing/e2e.md)
```
Each test suite follows these principles:
@@ -255,9 +224,9 @@ Tests can be configured using pytest options:
- `--sqlstore-provider` — Choose the SQL store provider (default: `postgres`)
- `--sqlite-mode` — SQLite journal mode: `delete` or `wal` (default: `delete`). Only relevant when `--sqlstore-provider=sqlite`.
- `--postgres-version` — PostgreSQL version (default: `15`)
- `--clickhouse-version` — ClickHouse version, also used for ClickHouse Keeper (default: `25.12.5`)
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.6`)
- `--with-web` — Build the frontend into the SigNoz image (required for e2e)
- `--clickhouse-version` — ClickHouse version (default: `25.5.6`)
- `--zookeeper-version` — Zookeeper version (default: `3.7.1`)
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.2`)
Example:
@@ -270,7 +239,6 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
## What should I remember?
- **Always use the `--reuse` flag** when setting up the environment or running tests to keep containers warm. Without it every run rebuilds the stack (~4 mins).
- **Changed backend code? Re-run `make py-test-setup`** — it passes `--rebuild`, swapping the SigNoz container for one built from your current sources while the rest of the stack stays up.
- **Use the `--teardown` flag** only when cleaning up — mixing `--teardown` with `--reuse` is a contradiction.
- **Do not pre-emptively teardown before setup.** If the stack is partially up, `--reuse` picks up from wherever it is. `make py-test-teardown` then `make py-test-setup` wastes minutes.
- **Follow the naming convention** with two-digit numeric prefixes (`01_`, `02_`) for ordered test execution within a suite.
@@ -279,5 +247,5 @@ uv run pytest --basetemp=./tmp/ -vv --reuse \
- **Use descriptive test names** that clearly indicate what is being tested.
- **Leverage fixtures** for common setup. The shared fixture package is at `tests/fixtures/` — reuse before adding new ones.
- **Test both success and failure scenarios** (4xx / 5xx paths) to ensure robust functionality.
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — ruff format + ruff check.
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — black + isort + autoflake + pylint.
- **`--sqlite-mode=wal` does not work on macOS.** The integration test environment runs SigNoz inside a Linux container with the SQLite database file mounted from the macOS host. WAL mode requires shared memory between connections, and connections crossing the VM boundary (macOS host ↔ Linux container) cannot share the WAL index, resulting in `SQLITE_IOERR_SHORT_READ`. WAL mode is tested in CI on Linux only.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 629 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 274 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 250 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 563 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 434 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 783 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 482 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 254 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 171 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 142 KiB

View File

@@ -53,6 +53,10 @@ func New(store authtypes.AuthNStore, licensing licensing.Licensing, providerSett
}
func (a *AuthN) LoginURL(ctx context.Context, siteURL *url.URL, authDomain *authtypes.AuthDomain) (string, error) {
if authDomain.AuthDomainConfig().AuthNProvider != authtypes.AuthNProviderOIDC {
return "", errors.Newf(errors.TypeInternal, authtypes.ErrCodeAuthDomainMismatch, "domain type is not oidc")
}
_, oauth2Config, err := a.oidcProviderAndoauth2Config(ctx, siteURL, authDomain)
if err != nil {
return "", err
@@ -81,11 +85,6 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
oidcConfig, err := authDomain.Config().OIDCConfig()
if err != nil {
return nil, err
}
oidcProvider, oauth2Config, err := a.oidcProviderAndoauth2Config(ctx, state.URL, authDomain)
if err != nil {
return nil, err
@@ -107,14 +106,14 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
return nil, err
}
if claims == nil && oidcConfig.GetUserInfo {
if claims == nil && authDomain.AuthDomainConfig().OIDC.GetUserInfo {
claims, err = a.claimsFromUserInfo(ctx, oidcProvider, token)
if err != nil {
return nil, err
}
}
emailClaim, ok := claims[oidcConfig.ClaimMapping.Email].(string)
emailClaim, ok := claims[authDomain.AuthDomainConfig().OIDC.ClaimMapping.Email].(string)
if !ok {
return nil, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "oidc: missing email in claims")
}
@@ -124,7 +123,7 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "oidc: failed to parse email").WithAdditional(err.Error())
}
if !oidcConfig.InsecureSkipEmailVerified {
if !authDomain.AuthDomainConfig().OIDC.InsecureSkipEmailVerified {
emailVerifiedClaim, ok := claims["email_verified"].(bool)
if !ok {
return nil, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "oidc: missing email_verified in claims")
@@ -136,14 +135,14 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
}
name := ""
if nameClaim := oidcConfig.ClaimMapping.Name; nameClaim != "" {
if nameClaim := authDomain.AuthDomainConfig().OIDC.ClaimMapping.Name; nameClaim != "" {
if n, ok := claims[nameClaim].(string); ok {
name = n
}
}
var groups []string
if groupsClaim := oidcConfig.ClaimMapping.Groups; groupsClaim != "" {
if groupsClaim := authDomain.AuthDomainConfig().OIDC.ClaimMapping.Groups; groupsClaim != "" {
if claimValue, exists := claims[groupsClaim]; exists {
switch g := claimValue.(type) {
case []any:
@@ -162,7 +161,7 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
}
role := ""
if roleClaim := oidcConfig.ClaimMapping.Role; roleClaim != "" {
if roleClaim := authDomain.AuthDomainConfig().OIDC.ClaimMapping.Role; roleClaim != "" {
if r, ok := claims[roleClaim].(string); ok {
role = r
}
@@ -178,16 +177,11 @@ func (a *AuthN) ProviderInfo(ctx context.Context, authDomain *authtypes.AuthDoma
}
func (a *AuthN) oidcProviderAndoauth2Config(ctx context.Context, siteURL *url.URL, authDomain *authtypes.AuthDomain) (*oidc.Provider, *oauth2.Config, error) {
oidcConfig, err := authDomain.Config().OIDCConfig()
if err != nil {
return nil, nil, err
if authDomain.AuthDomainConfig().OIDC.IssuerAlias != "" {
ctx = oidc.InsecureIssuerURLContext(ctx, authDomain.AuthDomainConfig().OIDC.IssuerAlias)
}
if oidcConfig.IssuerAlias != "" {
ctx = oidc.InsecureIssuerURLContext(ctx, oidcConfig.IssuerAlias)
}
oidcProvider, err := oidc.NewProvider(ctx, oidcConfig.Issuer)
oidcProvider, err := oidc.NewProvider(ctx, authDomain.AuthDomainConfig().OIDC.Issuer)
if err != nil {
return nil, nil, err
}
@@ -195,13 +189,13 @@ func (a *AuthN) oidcProviderAndoauth2Config(ctx context.Context, siteURL *url.UR
scopes := make([]string, len(defaultScopes))
copy(scopes, defaultScopes)
if authDomain.RoleMapping() != nil && len(authDomain.RoleMapping().GroupMappings) > 0 {
if authDomain.AuthDomainConfig().RoleMapping != nil && len(authDomain.AuthDomainConfig().RoleMapping.GroupMappings) > 0 {
scopes = append(scopes, "groups")
}
return oidcProvider, &oauth2.Config{
ClientID: oidcConfig.ClientID,
ClientSecret: oidcConfig.ClientSecret,
ClientID: authDomain.AuthDomainConfig().OIDC.ClientID,
ClientSecret: authDomain.AuthDomainConfig().OIDC.ClientSecret,
Endpoint: oidcProvider.Endpoint(),
Scopes: scopes,
RedirectURL: (&url.URL{
@@ -218,12 +212,7 @@ func (a *AuthN) claimsFromIDToken(ctx context.Context, authDomain *authtypes.Aut
return nil, errors.New(errors.TypeNotFound, errors.CodeNotFound, "oidc: no id_token in token response")
}
oidcConfig, err := authDomain.Config().OIDCConfig()
if err != nil {
return nil, err
}
verifier := provider.Verifier(&oidc.Config{ClientID: oidcConfig.ClientID})
verifier := provider.Verifier(&oidc.Config{ClientID: authDomain.AuthDomainConfig().OIDC.ClientID})
idToken, err := verifier.Verify(ctx, rawIDToken)
if err != nil {
return nil, errors.Newf(errors.TypeForbidden, errors.CodeForbidden, "oidc: failed to verify token").WithAdditional(err.Error())

View File

@@ -40,6 +40,10 @@ func New(ctx context.Context, store authtypes.AuthNStore, licensing licensing.Li
}
func (a *AuthN) LoginURL(ctx context.Context, siteURL *url.URL, authDomain *authtypes.AuthDomain) (string, error) {
if authDomain.AuthDomainConfig().AuthNProvider != authtypes.AuthNProviderSAML {
return "", errors.Newf(errors.TypeInternal, authtypes.ErrCodeAuthDomainMismatch, "saml: domain type is not saml")
}
sp, err := a.serviceProvider(siteURL, authDomain)
if err != nil {
return "", err
@@ -69,11 +73,6 @@ func (a *AuthN) HandleCallback(ctx context.Context, formValues url.Values) (*aut
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
samlConfig, err := authDomain.Config().SamlConfig()
if err != nil {
return nil, err
}
sp, err := a.serviceProvider(state.URL, authDomain)
if err != nil {
return nil, err
@@ -102,19 +101,19 @@ func (a *AuthN) HandleCallback(ctx context.Context, formValues url.Values) (*aut
}
name := ""
if nameAttribute := samlConfig.AttributeMapping.Name; nameAttribute != "" {
if nameAttribute := authDomain.AuthDomainConfig().SAML.AttributeMapping.Name; nameAttribute != "" {
if val := assertionInfo.Values.Get(nameAttribute); val != "" {
name = val
}
}
var groups []string
if groupAttribute := samlConfig.AttributeMapping.Groups; groupAttribute != "" {
if groupAttribute := authDomain.AuthDomainConfig().SAML.AttributeMapping.Groups; groupAttribute != "" {
groups = assertionInfo.Values.GetAll(groupAttribute)
}
role := ""
if roleAttribute := samlConfig.AttributeMapping.Role; roleAttribute != "" {
if roleAttribute := authDomain.AuthDomainConfig().SAML.AttributeMapping.Role; roleAttribute != "" {
if val := assertionInfo.Values.Get(roleAttribute); val != "" {
role = val
}
@@ -132,12 +131,7 @@ func (a *AuthN) ProviderInfo(ctx context.Context, authDomain *authtypes.AuthDoma
}
func (a *AuthN) serviceProvider(siteURL *url.URL, authDomain *authtypes.AuthDomain) (*saml2.SAMLServiceProvider, error) {
samlConfig, err := authDomain.Config().SamlConfig()
if err != nil {
return nil, err
}
certStore, err := a.getCertificateStore(samlConfig)
certStore, err := a.getCertificateStore(authDomain)
if err != nil {
return nil, err
}
@@ -148,32 +142,32 @@ func (a *AuthN) serviceProvider(siteURL *url.URL, authDomain *authtypes.AuthDoma
// The ServiceProviderIssuer is the client id in case of keycloak. Since we set it to the host here, we need to set the client id == host in keycloak.
// For AWSSSO, this is the value of Application SAML audience.
return &saml2.SAMLServiceProvider{
IdentityProviderSSOURL: samlConfig.Location,
IdentityProviderIssuer: samlConfig.EntityID,
IdentityProviderSSOURL: authDomain.AuthDomainConfig().SAML.SamlIdp,
IdentityProviderIssuer: authDomain.AuthDomainConfig().SAML.SamlEntity,
ServiceProviderIssuer: siteURL.Host,
AssertionConsumerServiceURL: acsURL.String(),
SignAuthnRequests: !samlConfig.InsecureSkipAuthNRequestsSigned,
SignAuthnRequests: !authDomain.AuthDomainConfig().SAML.InsecureSkipAuthNRequestsSigned,
AllowMissingAttributes: true,
IDPCertificateStore: certStore,
SPKeyStore: dsig.RandomKeyStoreForTest(),
}, nil
}
func (a *AuthN) getCertificateStore(samlConfig authtypes.SamlConfig) (dsig.X509CertificateStore, error) {
func (a *AuthN) getCertificateStore(authDomain *authtypes.AuthDomain) (dsig.X509CertificateStore, error) {
certStore := &dsig.MemoryX509CertificateStore{
Roots: []*x509.Certificate{},
}
var certBytes []byte
if strings.Contains(samlConfig.Certificate, "-----BEGIN CERTIFICATE-----") {
block, _ := pem.Decode([]byte(samlConfig.Certificate))
if strings.Contains(authDomain.AuthDomainConfig().SAML.SamlCert, "-----BEGIN CERTIFICATE-----") {
block, _ := pem.Decode([]byte(authDomain.AuthDomainConfig().SAML.SamlCert))
if block == nil {
return certStore, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "no valid pem cert found")
}
certBytes = block.Bytes
} else {
certData, err := base64.StdEncoding.DecodeString(samlConfig.Certificate)
certData, err := base64.StdEncoding.DecodeString(authDomain.AuthDomainConfig().SAML.SamlCert)
if err != nil {
return certStore, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to read certificate: %s", err.Error())
}

View File

@@ -119,6 +119,10 @@ func (provider *provider) CheckTransactions(ctx context.Context, subject string,
return results, nil
}
func (provider *provider) ListObjects(ctx context.Context, subject string, relation authtypes.Relation, objectType coretypes.Type) ([]*coretypes.Object, error) {
return provider.openfgaServer.ListObjects(ctx, subject, relation, objectType)
}
func (provider *provider) Write(ctx context.Context, additions []*openfgav1.TupleKey, deletions []*openfgav1.TupleKey) error {
return provider.openfgaServer.Write(ctx, additions, deletions)
}
@@ -127,6 +131,10 @@ func (provider *provider) ReadTuples(ctx context.Context, tupleKey *openfgav1.Re
return provider.openfgaServer.ReadTuples(ctx, tupleKey)
}
func (provider *provider) Get(ctx context.Context, orgID valuer.UUID, id valuer.UUID) (*authtypes.Role, error) {
return provider.pkgAuthzService.Get(ctx, orgID, id)
}
func (provider *provider) GetByOrgIDAndName(ctx context.Context, orgID valuer.UUID, name string) (*authtypes.Role, error) {
return provider.pkgAuthzService.GetByOrgIDAndName(ctx, orgID, name)
}
@@ -175,7 +183,7 @@ func (provider *provider) CreateManagedUserRoleTransactions(ctx context.Context,
return provider.Write(ctx, tuples, nil)
}
func (provider *provider) Create(ctx context.Context, orgID valuer.UUID, role *authtypes.Role) error {
func (provider *provider) Create(ctx context.Context, orgID valuer.UUID, role *authtypes.RoleWithTransactionGroups) error {
_, err := provider.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
@@ -200,42 +208,153 @@ func (provider *provider) Create(ctx context.Context, orgID valuer.UUID, role *a
return err
}
return provider.store.Create(ctx, role)
if err := provider.store.Create(ctx, role.Role); err != nil {
return err
}
return nil
}
func (provider *provider) Get(ctx context.Context, orgID valuer.UUID, id valuer.UUID) (*authtypes.Role, error) {
return provider.store.Get(ctx, orgID, id)
func (provider *provider) GetOrCreate(ctx context.Context, orgID valuer.UUID, role *authtypes.Role) (*authtypes.Role, error) {
_, err := provider.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
existingRole, err := provider.store.GetByOrgIDAndName(ctx, role.OrgID, role.Name)
if err != nil {
if !errors.Ast(err, errors.TypeNotFound) {
return nil, err
}
}
if existingRole != nil {
return existingRole, nil
}
err = provider.store.Create(ctx, role)
if err != nil {
return nil, err
}
return role, nil
}
func (provider *provider) Update(ctx context.Context, orgID valuer.UUID, updatedRole *authtypes.Role) error {
func (provider *provider) GetWithTransactionGroups(ctx context.Context, orgID valuer.UUID, id valuer.UUID) (*authtypes.RoleWithTransactionGroups, error) {
_, err := provider.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
role, err := provider.store.Get(ctx, orgID, id)
if err != nil {
return nil, err
}
tuples, err := provider.readAllTuplesForRole(ctx, role.Name, orgID)
if err != nil {
return nil, err
}
transactionGroups := authtypes.MustNewTransactionGroupsFromTuples(tuples)
return authtypes.MakeRoleWithTransactionGroups(role, transactionGroups), nil
}
func (provider *provider) GetObjects(ctx context.Context, orgID valuer.UUID, id valuer.UUID, relation authtypes.Relation) ([]*coretypes.Object, error) {
_, err := provider.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableRole, err := provider.store.Get(ctx, orgID, id)
if err != nil {
return nil, err
}
objects := make([]*coretypes.Object, 0)
for _, objectType := range provider.registry.Types() {
if coretypes.ErrIfVerbNotValidForType(relation.Verb, objectType) != nil {
continue
}
resourceObjects, err := provider.
ListObjects(
ctx,
authtypes.MustNewSubject(coretypes.NewResourceRole(), storableRole.Name, orgID, &coretypes.VerbAssignee),
relation,
objectType,
)
if err != nil {
return nil, err
}
objects = append(objects, resourceObjects...)
}
return objects, nil
}
func (provider *provider) Update(ctx context.Context, orgID valuer.UUID, updatedRole *authtypes.RoleWithTransactionGroups) error {
_, err := provider.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
existingRole, err := provider.Get(ctx, orgID, updatedRole.ID)
existingRole, err := provider.GetWithTransactionGroups(ctx, orgID, updatedRole.ID)
if err != nil {
return err
}
existingTuples, err := provider.readAllTuplesForRole(ctx, existingRole.Name, orgID)
additions, deletions := existingRole.TransactionGroups.Diff(updatedRole.TransactionGroups)
additionTuples, err := authtypes.NewTuplesFromTransactionGroups(existingRole.Name, orgID, additions)
if err != nil {
return err
}
desiredTuples, err := authtypes.NewTuplesFromTransactionGroups(existingRole.Name, orgID, updatedRole.TransactionGroups)
deletionTuples, err := authtypes.NewTuplesFromTransactionGroups(existingRole.Name, orgID, deletions)
if err != nil {
return err
}
additionTuples, deletionTuples := authtypes.DiffTuples(existingTuples, desiredTuples)
err = provider.Write(ctx, additionTuples, deletionTuples)
if err != nil {
return err
}
return provider.store.Update(ctx, orgID, updatedRole)
return provider.store.Update(ctx, orgID, updatedRole.Role)
}
func (provider *provider) Patch(ctx context.Context, orgID valuer.UUID, role *authtypes.Role) error {
_, err := provider.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
return provider.store.Update(ctx, orgID, role)
}
func (provider *provider) PatchObjects(ctx context.Context, orgID valuer.UUID, name string, relation authtypes.Relation, additions, deletions []*coretypes.Object) error {
_, err := provider.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
additionTuples, err := authtypes.GetAdditionTuples(name, orgID, relation, additions)
if err != nil {
return err
}
deletionTuples, err := authtypes.GetDeletionTuples(name, orgID, relation, deletions)
if err != nil {
return err
}
err = provider.Write(ctx, additionTuples, deletionTuples)
if err != nil {
return err
}
return nil
}
func (provider *provider) Delete(ctx context.Context, orgID valuer.UUID, id valuer.UUID) error {
@@ -244,7 +363,7 @@ func (provider *provider) Delete(ctx context.Context, orgID valuer.UUID, id valu
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
role, err := provider.Get(ctx, orgID, id)
role, err := provider.GetWithTransactionGroups(ctx, orgID, id)
if err != nil {
return err
}
@@ -260,7 +379,7 @@ func (provider *provider) Delete(ctx context.Context, orgID valuer.UUID, id valu
}
}
tuples, err := provider.readAllTuplesForRole(ctx, role.Name, orgID)
tuples, err := authtypes.NewTuplesFromTransactionGroups(role.Name, orgID, role.TransactionGroups)
if err != nil {
return err
}
@@ -272,24 +391,6 @@ func (provider *provider) Delete(ctx context.Context, orgID valuer.UUID, id valu
return provider.store.Delete(ctx, orgID, id)
}
func (provider *provider) readAllTuplesForRole(ctx context.Context, roleName string, orgID valuer.UUID) ([]*openfgav1.TupleKey, error) {
subject := authtypes.MustNewSubject(coretypes.NewResourceRole(), roleName, orgID, &coretypes.VerbAssignee)
tuples := make([]*openfgav1.TupleKey, 0)
for _, objectType := range provider.registry.Types() {
typeTuples, err := provider.openfgaServer.ReadTuples(ctx, &openfgav1.ReadRequestTupleKey{
User: subject,
Object: objectType.StringValue() + ":",
})
if err != nil {
return nil, err
}
tuples = append(tuples, typeTuples...)
}
return tuples, nil
}
func (provider *provider) getManagedRoleGrantTuples(orgID valuer.UUID, userID valuer.UUID) []*openfgav1.TupleKey {
tuples := []*openfgav1.TupleKey{}
@@ -341,3 +442,21 @@ func (provider *provider) getManagedRoleTransactionTuples(orgID valuer.UUID) []*
return tuples
}
func (provider *provider) readAllTuplesForRole(ctx context.Context, roleName string, orgID valuer.UUID) ([]*openfgav1.TupleKey, error) {
subject := authtypes.MustNewSubject(coretypes.NewResourceRole(), roleName, orgID, &coretypes.VerbAssignee)
tuples := make([]*openfgav1.TupleKey, 0)
for _, objectType := range provider.registry.Types() {
typeTuples, err := provider.ReadTuples(ctx, &openfgav1.ReadRequestTupleKey{
User: subject,
Object: objectType.StringValue() + ":",
})
if err != nil {
return nil, err
}
tuples = append(tuples, typeTuples...)
}
return tuples, nil
}

View File

@@ -105,6 +105,10 @@ func (server *Server) BatchCheck(ctx context.Context, tupleReq map[string]*openf
return server.pkgAuthzService.BatchCheck(ctx, tupleReq)
}
func (server *Server) ListObjects(ctx context.Context, subject string, relation authtypes.Relation, objectType coretypes.Type) ([]*coretypes.Object, error) {
return server.pkgAuthzService.ListObjects(ctx, subject, relation, objectType)
}
func (server *Server) Write(ctx context.Context, additions []*openfgav1.TupleKey, deletions []*openfgav1.TupleKey) error {
return server.pkgAuthzService.Write(ctx, additions, deletions)
}

View File

@@ -108,20 +108,6 @@ func (provider *Provider) SearchIngestionKeysByName(ctx context.Context, orgID v
}, nil
}
func (provider *Provider) GetIngestionKey(ctx context.Context, orgID valuer.UUID, keyID string) (*gatewaytypes.IngestionKey, error) {
responseBody, err := provider.do(ctx, orgID, http.MethodGet, "/v1/workspaces/me/keys/"+keyID, nil, nil)
if err != nil {
return nil, err
}
var ingestionKey gatewaytypes.IngestionKey
if err := json.Unmarshal([]byte(gjson.GetBytes(responseBody, "data").String()), &ingestionKey); err != nil {
return nil, err
}
return &ingestionKey, nil
}
func (provider *Provider) CreateIngestionKey(ctx context.Context, orgID valuer.UUID, name string, tags []string, expiresAt time.Time) (*gatewaytypes.GettableCreatedIngestionKey, error) {
requestBody := gatewaytypes.PostableIngestionKey{
Name: name,
@@ -175,7 +161,7 @@ func (provider *Provider) DeleteIngestionKey(ctx context.Context, orgID valuer.U
}
func (provider *Provider) CreateIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, keyID string, signal string, limitConfig gatewaytypes.LimitConfig, tags []string) (*gatewaytypes.GettableCreatedIngestionKeyLimit, error) {
requestBody := gatewaytypes.DeprecatedPostableIngestionKeyLimit{
requestBody := gatewaytypes.PostableIngestionKeyLimit{
Signal: signal,
Config: limitConfig,
Tags: tags,
@@ -198,34 +184,6 @@ func (provider *Provider) CreateIngestionKeyLimit(ctx context.Context, orgID val
return &createdIngestionKeyLimitResponse, nil
}
func (provider *Provider) GetIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, limitID string) (*gatewaytypes.Limit, error) {
responseBody, err := provider.do(ctx, orgID, http.MethodGet, "/v1/workspaces/me/limits/"+limitID, nil, nil)
if err != nil {
return nil, err
}
var limit gatewaytypes.Limit
if err := json.Unmarshal([]byte(gjson.GetBytes(responseBody, "data").String()), &limit); err != nil {
return nil, err
}
return &limit, nil
}
func (provider *Provider) GetIngestionKeyLimits(ctx context.Context, orgID valuer.UUID, keyID string) ([]gatewaytypes.Limit, error) {
responseBody, err := provider.do(ctx, orgID, http.MethodGet, "/v1/workspaces/me/keys/"+keyID+"/limits", nil, nil)
if err != nil {
return nil, err
}
var limits []gatewaytypes.Limit
if err := json.Unmarshal([]byte(gjson.GetBytes(responseBody, "data").String()), &limits); err != nil {
return nil, err
}
return limits, nil
}
func (provider *Provider) UpdateIngestionKeyLimit(ctx context.Context, orgID valuer.UUID, limitID string, limitConfig gatewaytypes.LimitConfig, tags []string) error {
requestBody := gatewaytypes.UpdatableIngestionKeyLimit{
Config: limitConfig,

View File

@@ -0,0 +1,168 @@
package httplicensing
import (
"context"
"encoding/json"
"net/http"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/licensetypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type licensingAPI struct {
licensing licensing.Licensing
}
func NewLicensingAPI(licensing licensing.Licensing) licensing.API {
return &licensingAPI{licensing: licensing}
}
func (api *licensingAPI) Activate(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(rw, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "orgId is invalid"))
return
}
req := new(licensetypes.PostableLicense)
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
render.Error(rw, err)
return
}
err = api.licensing.Activate(r.Context(), orgID, req.Key)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusAccepted, nil)
}
func (api *licensingAPI) GetActive(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(rw, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "orgId is invalid"))
return
}
license, err := api.licensing.GetActive(r.Context(), orgID)
if err != nil {
render.Error(rw, err)
return
}
gettableLicense := licensetypes.NewGettableLicense(license.Data, license.Key)
render.Success(rw, http.StatusOK, gettableLicense)
}
func (api *licensingAPI) Refresh(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(rw, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "orgId is invalid"))
return
}
err = api.licensing.Refresh(r.Context(), orgID)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusNoContent, nil)
}
func (api *licensingAPI) Checkout(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(rw, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "orgId is invalid"))
return
}
req := new(licensetypes.PostableSubscription)
if err := json.NewDecoder(r.Body).Decode(req); err != nil {
render.Error(rw, err)
return
}
gettableSubscription, err := api.licensing.Checkout(ctx, orgID, req)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusCreated, gettableSubscription)
}
func (api *licensingAPI) Portal(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(rw, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "orgId is invalid"))
return
}
req := new(licensetypes.PostableSubscription)
if err := json.NewDecoder(r.Body).Decode(req); err != nil {
render.Error(rw, err)
return
}
gettableSubscription, err := api.licensing.Portal(ctx, orgID, req)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusCreated, gettableSubscription)
}

View File

@@ -2,9 +2,12 @@ package httplicensing
import (
"context"
"encoding/json"
"log/slog"
"time"
"github.com/tidwall/gjson"
"github.com/SigNoz/signoz/ee/licensing/licensingstore/sqllicensingstore"
"github.com/SigNoz/signoz/pkg/analytics"
"github.com/SigNoz/signoz/pkg/errors"
@@ -92,65 +95,24 @@ func (provider *provider) Validate(ctx context.Context) error {
return nil
}
func (provider *provider) Activate(ctx context.Context, organizationID valuer.UUID, key string) (*licensetypes.License, error) {
zeusLicense, err := provider.zeus.GetLicense(ctx, key)
func (provider *provider) Activate(ctx context.Context, organizationID valuer.UUID, key string) error {
data, err := provider.zeus.GetLicense(ctx, key)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "unable to fetch license data with upstream server")
return errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "unable to fetch license data with upstream server")
}
license, err := licensetypes.NewLicense(zeusLicense, organizationID)
license, err := licensetypes.NewLicense(data, organizationID)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to create license entity")
return errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to create license entity")
}
storableLicense := licensetypes.NewStorableLicenseFromLicense(license)
err = provider.store.Create(ctx, storableLicense)
if err != nil {
return nil, err
}
return license, nil
}
func (provider *provider) Get(ctx context.Context, organizationID valuer.UUID, licenseID valuer.UUID) (*licensetypes.License, error) {
storableLicense, err := provider.store.Get(ctx, organizationID, licenseID)
if err != nil {
return nil, err
}
return licensetypes.NewLicenseFromStorableLicense(storableLicense)
}
func (provider *provider) List(ctx context.Context, organizationID valuer.UUID) ([]*licensetypes.License, error) {
storableLicenses, err := provider.store.GetAll(ctx, organizationID)
if err != nil {
return nil, err
}
licenses := make([]*licensetypes.License, 0, len(storableLicenses))
for _, storableLicense := range storableLicenses {
license, err := licensetypes.NewLicenseFromStorableLicense(storableLicense)
if err != nil {
return nil, err
}
licenses = append(licenses, license)
}
return licenses, nil
}
func (provider *provider) Delete(ctx context.Context, organizationID valuer.UUID, licenseID valuer.UUID) error {
license, err := provider.Get(ctx, organizationID, licenseID)
if err != nil {
return err
}
if err := license.ErrIfCloud(); err != nil {
return errors.WithAdditionalf(err, "license %s cannot be deleted", licenseID.StringValue())
}
return provider.store.Delete(ctx, organizationID, licenseID)
return nil
}
func (provider *provider) GetActive(ctx context.Context, organizationID valuer.UUID) (*licensetypes.License, error) {
@@ -177,7 +139,7 @@ func (provider *provider) Refresh(ctx context.Context, organizationID valuer.UUI
return err
}
zeusLicense, err := provider.zeus.GetLicense(ctx, activeLicense.Key)
data, err := provider.zeus.GetLicense(ctx, activeLicense.Key)
if err != nil {
if time.Since(activeLicense.LastValidatedAt) > time.Duration(provider.config.FailureThreshold)*provider.config.PollInterval {
activeLicense.UpdateFeatures(licensetypes.BasicPlan)
@@ -192,7 +154,7 @@ func (provider *provider) Refresh(ctx context.Context, organizationID valuer.UUI
return err
}
err = activeLicense.Update(zeusLicense)
err = activeLicense.Update(data)
if err != nil {
return errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "failed to create license entity from license data")
}
@@ -225,6 +187,47 @@ func (provider *provider) Refresh(ctx context.Context, organizationID valuer.UUI
return nil
}
func (provider *provider) Checkout(ctx context.Context, organizationID valuer.UUID, postableSubscription *licensetypes.PostableSubscription) (*licensetypes.GettableSubscription, error) {
activeLicense, err := provider.GetActive(ctx, organizationID)
if err != nil {
return nil, err
}
body, err := json.Marshal(postableSubscription)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to marshal checkout payload")
}
response, err := provider.zeus.GetCheckoutURL(ctx, activeLicense.Key, body)
if err != nil {
if errors.Ast(err, errors.TypeAlreadyExists) {
return nil, errors.WithAdditionalf(err, "checkout has already been completed for this account. Please click 'Refresh Status' to sync your subscription")
}
return nil, err
}
return &licensetypes.GettableSubscription{RedirectURL: gjson.GetBytes(response, "url").String()}, nil
}
func (provider *provider) Portal(ctx context.Context, organizationID valuer.UUID, postableSubscription *licensetypes.PostableSubscription) (*licensetypes.GettableSubscription, error) {
activeLicense, err := provider.GetActive(ctx, organizationID)
if err != nil {
return nil, err
}
body, err := json.Marshal(postableSubscription)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to marshal portal payload")
}
response, err := provider.zeus.GetPortalURL(ctx, activeLicense.Key, body)
if err != nil {
return nil, err
}
return &licensetypes.GettableSubscription{RedirectURL: gjson.GetBytes(response, "url").String()}, nil
}
func (provider *provider) GetFeatureFlags(ctx context.Context, organizationID valuer.UUID) ([]*licensetypes.Feature, error) {
license, err := provider.GetActive(ctx, organizationID)
if err != nil {

View File

@@ -64,22 +64,6 @@ func (store *store) GetAll(ctx context.Context, organizationID valuer.UUID) ([]*
return storableLicenses, nil
}
func (store *store) Delete(ctx context.Context, organizationID valuer.UUID, licenseID valuer.UUID) error {
_, err := store.
sqlstore.
BunDB().
NewDelete().
Model(new(licensetypes.StorableLicense)).
Where("org_id = ?", organizationID).
Where("id = ?", licenseID).
Exec(ctx)
if err != nil {
return errors.Wrapf(err, errors.TypeInternal, errors.CodeInternal, "unable to delete license with ID: %s", licenseID)
}
return nil
}
func (store *store) Update(ctx context.Context, organizationID valuer.UUID, storableLicense *licensetypes.StorableLicense) error {
_, err := store.
sqlstore.

View File

@@ -16,7 +16,7 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/metercollector"
"github.com/SigNoz/signoz/pkg/modules/retention"
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrymeter"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/licensetypes"
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
@@ -153,7 +153,7 @@ func (provider *Provider) Collect(
func buildOriginQuery(meterName string) (string, []any) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select("toInt64(ifNull(min(unix_milli), 0))")
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
sb.Where(sb.Equal("metric_name", meterName))
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
@@ -171,7 +171,7 @@ func buildQuery(meterName string, segment *retentiontypes.RetentionPolicySegment
sb := sqlbuilder.NewSelectBuilder()
sb.Select(selects...)
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
sb.Where(
sb.Equal("metric_name", meterName),
sb.GTE("unix_milli", segment.StartMs),

View File

@@ -1,36 +0,0 @@
package implcloudprovider
import (
"context"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
)
type gcpcloudprovider struct {
serviceDefinitions cloudintegrationtypes.ServiceDefinitionStore
}
func NewGCPCloudProvider(defStore cloudintegrationtypes.ServiceDefinitionStore) cloudintegration.CloudProviderModule {
return &gcpcloudprovider{
serviceDefinitions: defStore,
}
}
func (g *gcpcloudprovider) BuildIntegrationConfig(ctx context.Context, account *cloudintegrationtypes.Account, services []*cloudintegrationtypes.StorableCloudIntegrationService) (*cloudintegrationtypes.ProviderIntegrationConfig, error) {
// for manual flow we don't have any integration config to return, so returning empty config for now.
return &cloudintegrationtypes.ProviderIntegrationConfig{}, nil
}
func (g *gcpcloudprovider) GetConnectionArtifact(ctx context.Context, account *cloudintegrationtypes.Account, req *cloudintegrationtypes.GetConnectionArtifactRequest) (*cloudintegrationtypes.ConnectionArtifact, error) {
// for manual flow we don't have any connection artifact to return, so returning empty artifact for now.
return &cloudintegrationtypes.ConnectionArtifact{}, nil
}
func (g *gcpcloudprovider) GetServiceDefinition(ctx context.Context, serviceID cloudintegrationtypes.ServiceID) (*cloudintegrationtypes.ServiceDefinition, error) {
return g.serviceDefinitions.Get(ctx, cloudintegrationtypes.CloudProviderTypeGCP, serviceID)
}
func (g *gcpcloudprovider) ListServiceDefinitions(ctx context.Context) ([]*cloudintegrationtypes.ServiceDefinition, error) {
return g.serviceDefinitions.List(ctx, cloudintegrationtypes.CloudProviderTypeGCP)
}

View File

@@ -519,7 +519,7 @@ func (module *module) getOrCreateAPIKey(ctx context.Context, orgID valuer.UUID,
if err != nil {
return "", err
}
_, err = module.serviceAccount.SetRoleByName(ctx, orgID, serviceAccount.ID, authtypes.SigNozViewerRoleName)
err = module.serviceAccount.SetRoleByName(ctx, orgID, serviceAccount.ID, authtypes.SigNozViewerRoleName)
if err != nil {
return "", err
}
@@ -551,12 +551,12 @@ func (module *module) provisionDashboards(ctx context.Context, orgID valuer.UUID
continue
}
createdDashboard, err := module.dashboardModule.CreateV2(ctx, orgID, createdBy, creator, dashboardtypes.SourceIntegration, dashboard.Definition)
createdDashboard, err := module.dashboardModule.Create(ctx, orgID, createdBy, creator, dashboardtypes.SourceIntegration, dashboardtypes.PostableDashboard(dashboard.Definition))
if err != nil {
return err
}
integrationDashboard := cloudintegrationtypes.NewStorableIntegrationDashboard(createdDashboard.ID.StringValue(), cloudintegrationtypes.IntegrationDashboardProviderCloudIntegration, slug)
integrationDashboard := cloudintegrationtypes.NewStorableIntegrationDashboard(createdDashboard.ID, cloudintegrationtypes.IntegrationDashboardProviderCloudIntegration, slug)
if err := module.store.CreateIntegrationDashboard(ctx, integrationDashboard); err != nil {
return err
}

View File

@@ -29,12 +29,11 @@ type module struct {
settings factory.ScopedProviderSettings
querier querier.Querier
licensing licensing.Licensing
tagModule tag.Module
}
func NewModule(store dashboardtypes.Store, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, querier querier.Querier, licensing licensing.Licensing, tagModule tag.Module, systemDashboardRegistry dashboardtypes.SystemDashboardRegistry) dashboard.Module {
func NewModule(store dashboardtypes.Store, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, querier querier.Querier, licensing licensing.Licensing, tagModule tag.Module) dashboard.Module {
scopedProviderSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/ee/modules/dashboard/impldashboard")
pkgDashboardModule := pkgimpldashboard.NewModule(store, settings, analytics, orgGetter, queryParser, tagModule, systemDashboardRegistry)
pkgDashboardModule := pkgimpldashboard.NewModule(store, settings, analytics, orgGetter, queryParser, tagModule)
return &module{
pkgDashboardModule: pkgDashboardModule,
@@ -42,7 +41,6 @@ func NewModule(store dashboardtypes.Store, settings factory.ProviderSettings, an
settings: scopedProviderSettings,
querier: querier,
licensing: licensing,
tagModule: tagModule,
}
}
@@ -134,55 +132,6 @@ func (module *module) GetPublicWidgetQueryRange(ctx context.Context, id valuer.U
return module.querier.QueryRange(ctx, dashboard.OrgID, query)
}
func (module *module) GetDashboardByPublicIDV2(ctx context.Context, id valuer.UUID) (*dashboardtypes.DashboardV2, error) {
storableDashboard, err := module.store.GetDashboardByPublicID(ctx, id.StringValue())
if err != nil {
return nil, err
}
tags, err := module.tagModule.ListForResource(ctx, storableDashboard.OrgID, coretypes.KindDashboard, storableDashboard.ID)
if err != nil {
return nil, err
}
return storableDashboard.ToDashboardV2(tags)
}
func (module *module) GetPublicWidgetQueryRangeV2(ctx context.Context, id valuer.UUID, panelKey, startTimeRaw, endTimeRaw string) (*querybuildertypesv5.QueryRangeResponse, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "dashboard",
instrumentationtypes.CodeFunctionName: "GetPublicWidgetQueryRangeV2",
})
storableDashboard, err := module.store.GetDashboardByPublicID(ctx, id.StringValue())
if err != nil {
return nil, err
}
// tags are not needed for query range.
dashboard, err := storableDashboard.ToDashboardV2(nil)
if err != nil {
return nil, err
}
publicDashboard, err := module.GetPublic(ctx, dashboard.OrgID, dashboard.ID)
if err != nil {
return nil, err
}
startTime, endTime, err := publicDashboard.ResolveTimeRange(startTimeRaw, endTimeRaw)
if err != nil {
return nil, err
}
query, err := dashboard.GetPanelQuery(startTime, endTime, panelKey)
if err != nil {
return nil, err
}
return module.querier.QueryRange(ctx, dashboard.OrgID, query)
}
func (module *module) UpdatePublic(ctx context.Context, orgID valuer.UUID, publicDashboard *dashboardtypes.PublicDashboard) error {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
@@ -276,10 +225,6 @@ func (module *module) GetV2(ctx context.Context, orgID valuer.UUID, id valuer.UU
return module.pkgDashboardModule.GetV2(ctx, orgID, id)
}
func (module *module) MigrateV2(ctx context.Context, orgID valuer.UUID, id valuer.UUID) (*dashboardtypes.DashboardV2, error) {
return module.pkgDashboardModule.MigrateV2(ctx, orgID, id)
}
func (module *module) UpdateV2(ctx context.Context, orgID valuer.UUID, id valuer.UUID, updatedBy string, updatable dashboardtypes.UpdatableDashboardV2) (*dashboardtypes.DashboardV2, error) {
return module.pkgDashboardModule.UpdateV2(ctx, orgID, id, updatedBy, updatable)
}
@@ -341,7 +286,7 @@ func (module *module) Get(ctx context.Context, orgID valuer.UUID, id valuer.UUID
return module.pkgDashboardModule.Get(ctx, orgID, id)
}
func (module *module) GetByMetricNames(ctx context.Context, orgID valuer.UUID, metricNames []string) (map[string][]dashboardtypes.DashboardPanelRef, error) {
func (module *module) GetByMetricNames(ctx context.Context, orgID valuer.UUID, metricNames []string) (map[string][]map[string]string, error) {
return module.pkgDashboardModule.GetByMetricNames(ctx, orgID, metricNames)
}
@@ -361,14 +306,6 @@ func (module *module) LockUnlock(ctx context.Context, orgID valuer.UUID, id valu
return module.pkgDashboardModule.LockUnlock(ctx, orgID, id, updatedBy, isAdmin, lock)
}
func (module *module) ReconcileSystemDashboards(ctx context.Context, orgID valuer.UUID) error {
return module.pkgDashboardModule.ReconcileSystemDashboards(ctx, orgID)
}
func (module *module) GetSystemDashboard(ctx context.Context, orgID valuer.UUID, name string) (*dashboardtypes.DashboardV2, error) {
return module.pkgDashboardModule.GetSystemDashboard(ctx, orgID, name)
}
func (module *module) delete(ctx context.Context, orgID, id valuer.UUID) error {
return module.store.RunInTx(ctx, func(ctx context.Context) error {
if err := module.store.DeletePublic(ctx, id.String()); err != nil && !errors.Ast(err, errors.TypeNotFound) {

View File

@@ -8,22 +8,20 @@ import (
sqlbuilder "github.com/huandu/go-sqlbuilder"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/metricreductionruletypes"
)
var (
reductionRulesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.ReductionRulesTableName
metadataTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.AttributesMetadataTableName
bufferSeriesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.TimeseriesV4BufferTableName
reductionRulesTable = telemetrymetrics.DBName + "." + telemetrymetrics.ReductionRulesTableName
metadataTable = telemetrymetrics.DBName + "." + telemetrymetrics.AttributesMetadataTableName
bufferSeriesTable = telemetrymetrics.DBName + "." + telemetrymetrics.TimeseriesV4BufferTableName
)
const timeSeriesBucketMilli = int64(time.Hour / time.Millisecond)
const sampleBucketExpr = "toInt64(toUnixTimestamp(toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalMinute(10)))) * 1000 AS bucket"
type volumeRow struct {
MetricName string
Ingested uint64
@@ -192,7 +190,7 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
sb := sqlbuilder.NewSelectBuilder()
sb.Select("metric_name", "uniq(fingerprint)")
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
conds := []string{
sb.In("metric_name", names...),
sb.GE("unix_milli", startMs),
@@ -229,8 +227,8 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
// reduced sample tables.
func (c *clickhouse) reducedSeriesCount(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[string]uint64, error) {
out := make(map[string]uint64, len(metricNames))
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
counts, err := c.reducedSeriesCountForTable(ctx, metricstelemetryschema.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
counts, err := c.reducedSeriesCountForTable(ctx, telemetrymetrics.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
@@ -291,35 +289,38 @@ func (c *clickhouse) RankByVolume(ctx context.Context, metricNames []string, eff
}
ctx = c.withThreads(ctx)
orderExpr := "ifNull(i.samples, 0)"
if orderBy == metricreductionruletypes.OrderByReducedVolume {
orderExpr = "if(ifNull(d.samples, 0) = 0 OR ifNull(d.samples, 0) > ifNull(i.samples, 0), ifNull(i.samples, 0), ifNull(d.samples, 0))"
orderExpr := "ingested"
switch orderBy {
case metricreductionruletypes.OrderByReducedVolume:
orderExpr = "reduced"
case metricreductionruletypes.OrderByReduction:
orderExpr = "if(ingested = 0, 0, (toFloat64(ingested) - toFloat64(reduced)) / toFloat64(ingested))"
}
direction := "ASC"
if order == metricreductionruletypes.OrderDesc {
direction = "DESC"
}
ingestedTable := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName
reducedLast := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedLastTableName
reducedSum := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedSumTableName
ingestedTable := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName
reducedLast := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedLastTableName
reducedSum := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedSumTableName
sb := sqlbuilder.NewSelectBuilder()
sb.Select("base.metric_name AS metric_name", "ifNull(i.cnt, 0) AS ingested", "ifNull(d.cnt, 0) AS reduced")
sb.From("(SELECT arrayJoin(" + sb.Var(metricNames) + ") AS metric_name) AS base")
sb.JoinWithOption(
sqlbuilder.LeftJoin,
"(SELECT metric_name, uniq(fingerprint) AS cnt, count() AS samples FROM "+ingestedTable+" WHERE has("+sb.Var(metricNames)+", metric_name) AND unix_milli >= "+sb.Var(startMs)+" AND unix_milli < "+sb.Var(endMs)+" AND "+strictEffectiveFrom(sb, metricNames, effectiveFrom)+" GROUP BY metric_name) AS i",
"(SELECT metric_name, uniq(fingerprint) AS cnt FROM "+ingestedTable+" WHERE has("+sb.Var(metricNames)+", metric_name) AND unix_milli >= "+sb.Var(startMs)+" AND unix_milli < "+sb.Var(endMs)+" AND "+strictEffectiveFrom(sb, metricNames, effectiveFrom)+" GROUP BY metric_name) AS i",
"base.metric_name = i.metric_name",
)
// Reduced series are spread across two type-specific tables; union the per-table distinct
// reduced_fingerprints and sum per metric (a metric only lands in the table matching its type).
sb.JoinWithOption(
sqlbuilder.LeftJoin,
"(SELECT metric_name, sum(cnt) AS cnt, sum(samples) AS samples FROM ("+
"SELECT metric_name, uniq(reduced_fingerprint) AS cnt, uniq(reduced_fingerprint, unix_milli) AS samples FROM "+reducedLast+" WHERE has("+sb.Var(metricNames)+", metric_name) AND unix_milli >= "+sb.Var(startMs)+" AND unix_milli < "+sb.Var(endMs)+" AND "+strictEffectiveFrom(sb, metricNames, effectiveFrom)+" GROUP BY metric_name"+
"(SELECT metric_name, sum(cnt) AS cnt FROM ("+
"SELECT metric_name, uniq(reduced_fingerprint) AS cnt FROM "+reducedLast+" WHERE has("+sb.Var(metricNames)+", metric_name) AND unix_milli >= "+sb.Var(startMs)+" AND unix_milli < "+sb.Var(endMs)+" AND "+strictEffectiveFrom(sb, metricNames, effectiveFrom)+" GROUP BY metric_name"+
" UNION ALL "+
"SELECT metric_name, uniq(reduced_fingerprint) AS cnt, uniq(reduced_fingerprint, unix_milli) AS samples FROM "+reducedSum+" WHERE has("+sb.Var(metricNames)+", metric_name) AND unix_milli >= "+sb.Var(startMs)+" AND unix_milli < "+sb.Var(endMs)+" AND "+strictEffectiveFrom(sb, metricNames, effectiveFrom)+" GROUP BY metric_name"+
"SELECT metric_name, uniq(reduced_fingerprint) AS cnt FROM "+reducedSum+" WHERE has("+sb.Var(metricNames)+", metric_name) AND unix_milli >= "+sb.Var(startMs)+" AND unix_milli < "+sb.Var(endMs)+" AND "+strictEffectiveFrom(sb, metricNames, effectiveFrom)+" GROUP BY metric_name"+
") GROUP BY metric_name) AS d",
"base.metric_name = d.metric_name",
)
@@ -346,184 +347,120 @@ func (c *clickhouse) RankByVolume(ctx context.Context, metricNames []string, eff
return out, rows.Err()
}
func (c *clickhouse) SampleVolumeByMetric(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[string]volumeRow, error) {
func (c *clickhouse) SampleVolume(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (uint64, uint64, error) {
if len(metricNames) == 0 {
return map[string]volumeRow{}, nil
return 0, 0, nil
}
ctx = c.withThreads(ctx)
ingested, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4BufferTableName, "count()", metricNames, effectiveFrom, startMs, endMs)
ingested, err := c.countRawSamples(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4BufferTableName, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
return 0, 0, err
}
last, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedLastTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
last, err := c.countReducedSamples(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedLastTableName, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
return 0, 0, err
}
sum, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedSumTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
sum, err := c.countReducedSamples(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedSumTableName, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
return 0, 0, err
}
out := make(map[string]volumeRow, len(metricNames))
for _, name := range metricNames {
out[name] = volumeRow{MetricName: name, Ingested: ingested[name], Reduced: last[name] + sum[name]}
}
return out, nil
return ingested, min(last+sum, ingested), nil
}
func (c *clickhouse) countSamplesByMetric(ctx context.Context, table, countExpr string, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[string]uint64, error) {
func (c *clickhouse) countRawSamples(ctx context.Context, table string, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (uint64, error) {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select("metric_name", countExpr)
sb.Select("count()")
sb.From(table)
conds := []string{
sb.In("metric_name", names...),
sb.GE("unix_milli", startMs),
sb.LT("unix_milli", endMs),
}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
}
sb.Where(conds...)
sb.GroupBy("metric_name")
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
rows, err := c.telemetryStore.ClickhouseDB().Query(ctx, query, args...)
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "failed to count samples")
}
defer rows.Close()
out := make(map[string]uint64, len(metricNames))
for rows.Next() {
var (
metricName string
count uint64
)
if err := rows.Scan(&metricName, &count); err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "failed to scan series count")
}
out[metricName] = count
}
return out, rows.Err()
}
func (c *clickhouse) TotalVolume(ctx context.Context, startMs, endMs int64) (uint64, uint64, error) {
ctx = c.withThreads(ctx)
sb := sqlbuilder.NewSelectBuilder()
sb.Select("uniq(fingerprint)", "count()")
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
var series, samples uint64
if err := c.telemetryStore.ClickhouseDB().QueryRow(ctx, query, args...).Scan(&series, &samples); err != nil {
return 0, 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to count total ingested volume")
}
return series, samples, nil
}
func (c *clickhouse) SampleTimeseries(ctx context.Context, ruledMetrics []string, effectiveFrom map[string]int64, startMs, endMs int64) ([]volumePoint, error) {
ctx = c.withThreads(ctx)
ingested, err := c.totalSamplesByBucket(ctx, startMs, endMs)
if err != nil {
return nil, err
}
ruledIngested := make(map[int64]uint64)
ruledRetained := make(map[int64]uint64)
if len(ruledMetrics) > 0 {
ruledIngested, err = c.ruledIngestedSamplesByBucket(ctx, ruledMetrics, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
ruledRetained, err = c.ruledRetainedSamplesByBucket(ctx, ruledMetrics, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
}
retained := make(map[int64]uint64, len(ingested))
for ts, total := range ingested {
shed := uint64(0)
if ri := ruledIngested[ts]; ri > ruledRetained[ts] {
shed = ri - ruledRetained[ts]
}
if total > shed {
retained[ts] = total - shed
} else {
retained[ts] = 0
}
}
return mergeVolumePoints(ingested, retained), nil
}
func (c *clickhouse) totalSamplesByBucket(ctx context.Context, startMs, endMs int64) (map[int64]uint64, error) {
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "count()")
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
sb.GroupBy("bucket")
return c.scanBuckets(ctx, sb)
}
func (c *clickhouse) ruledIngestedSamplesByBucket(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[int64]uint64, error) {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "count()")
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
}
sb.Where(conds...)
sb.GroupBy("bucket")
return c.scanBuckets(ctx, sb)
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
var count uint64
if err := c.telemetryStore.ClickhouseDB().QueryRow(ctx, query, args...).Scan(&count); err != nil {
return 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to count ingested samples")
}
return count, nil
}
// reduced 60s rows are versioned by computed_at, so count distinct buckets.
func (c *clickhouse) ruledRetainedSamplesByBucket(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[int64]uint64, error) {
out := make(map[int64]uint64)
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
}
func (c *clickhouse) countReducedSamples(ctx context.Context, table string, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (uint64, error) {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select(sampleBucketExpr, "uniq(reduced_fingerprint, unix_milli)")
sb.From(metricstelemetryschema.DBName + "." + table)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
}
sb.Where(conds...)
sb.GroupBy("bucket")
sb := sqlbuilder.NewSelectBuilder()
// Reduced tables key the series on reduced_fingerprint (not fingerprint); dedupe ReplacingMergeTree recomputes.
sb.Select("uniq(reduced_fingerprint, unix_milli)")
sb.From(table)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
}
sb.Where(conds...)
counts, err := c.scanBuckets(ctx, sb)
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
var count uint64
if err := c.telemetryStore.ClickhouseDB().QueryRow(ctx, query, args...).Scan(&count); err != nil {
return 0, errors.WrapInternalf(err, errors.CodeInternal, "failed to count reduced samples")
}
return count, nil
}
// SeriesTimeseries returns ingested vs reduced series per 60s bucket from the samples tables, gated
// to each metric's strict effective_from (see strictEffectiveFrom).
func (c *clickhouse) SeriesTimeseries(ctx context.Context, allMetrics, reducedMetrics []string, effectiveFrom map[string]int64, startMs, endMs int64) ([]volumePoint, error) {
if len(allMetrics) == 0 {
return []volumePoint{}, nil
}
ctx = c.withThreads(ctx)
ingested, err := c.ingestedSeriesByBucket(ctx, allMetrics, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
retained := make(map[int64]uint64)
if len(reducedMetrics) > 0 {
reduced, err := c.reducedSeriesByBucket(ctx, reducedMetrics, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
for ts, count := range counts {
out[ts] += count
for ts, count := range reduced {
retained[ts] += count
}
}
return out, nil
reducedSet := make(map[string]struct{}, len(reducedMetrics))
for _, name := range reducedMetrics {
reducedSet[name] = struct{}{}
}
nonReduced := make([]string, 0, len(allMetrics))
for _, name := range allMetrics {
if _, ok := reducedSet[name]; !ok {
nonReduced = append(nonReduced, name)
}
}
if len(nonReduced) > 0 {
nonReducedIngested, err := c.ingestedSeriesByBucket(ctx, nonReduced, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
for ts, count := range nonReducedIngested {
retained[ts] += count
}
}
return mergeVolumePoints(ingested, retained), nil
}
func mergeVolumePoints(ingested, reduced map[int64]uint64) []volumePoint {
@@ -551,6 +488,60 @@ func mergeVolumePoints(ingested, reduced map[int64]uint64) []volumePoint {
return points
}
// ingestedSeriesByBucket counts distinct raw fingerprints per hourly bucket from the samples buffer.
func (c *clickhouse) ingestedSeriesByBucket(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[int64]uint64, error) {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
}
sb := sqlbuilder.NewSelectBuilder()
bucketExpr := "toInt64(toUnixTimestamp(toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalHour(1)))) * 1000 AS bucket"
sb.Select(bucketExpr, "uniq(fingerprint)")
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
}
sb.Where(conds...)
sb.GroupBy("bucket")
return c.scanBuckets(ctx, sb)
}
// reducedSeriesByBucket counts distinct reduced_fingerprints per hourly bucket, summed across the two
// reduced sample tables (a metric only lands in the table matching its type, so per-bucket sums are
// exact).
func (c *clickhouse) reducedSeriesByBucket(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[int64]uint64, error) {
out := make(map[int64]uint64)
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
names := make([]any, len(metricNames))
for i, name := range metricNames {
names[i] = name
}
sb := sqlbuilder.NewSelectBuilder()
bucketExpr := "toInt64(toUnixTimestamp(toStartOfInterval(toDateTime(intDiv(unix_milli, 1000)), toIntervalHour(1)))) * 1000 AS bucket"
sb.Select(bucketExpr, "uniq(reduced_fingerprint)")
sb.From(telemetrymetrics.DBName + "." + table)
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
if len(effectiveFrom) > 0 {
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
}
sb.Where(conds...)
sb.GroupBy("bucket")
counts, err := c.scanBuckets(ctx, sb)
if err != nil {
return nil, err
}
for ts, count := range counts {
out[ts] += count
}
}
return out, nil
}
func (c *clickhouse) scanBuckets(ctx context.Context, sb *sqlbuilder.SelectBuilder) (map[int64]uint64, error) {
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
rows, err := c.telemetryStore.ClickhouseDB().Query(ctx, query, args...)

View File

@@ -4,6 +4,7 @@ import (
"context"
"log/slog"
"sort"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/errors"
@@ -27,16 +28,9 @@ import (
const (
// effectiveFromMargin delays effective_from so the collector picks up the synced rule before it
// goes live; it must be >= the collector's rule-refresh interval (~2m worst case,
// see signoz-otel-collector#839).
effectiveFromMargin = 2 * time.Minute
// uiActivationDelay keeps a rule shown as "pending" in the UI for a while after it goes live to
// the collector, so the user doesn't see "active" before reduced data is actually flowing. The
// user-facing pending window is effectiveFromMargin + uiActivationDelay (~5m).
uiActivationDelay = 3 * time.Minute
defaultPreviewLookback = 1 * time.Hour
statsLookback = 1 * time.Hour
timeseriesLookback = 6 * time.Hour
// goes live; it must be >= the collector's rule-refresh interval (see signoz-otel-collector#839).
effectiveFromMargin = 5 * time.Minute
defaultPreviewLookback = 24 * time.Hour
pricePerMillionSamplesUSD = 0.1
monthDuration = 30 * 24 * time.Hour
@@ -86,7 +80,7 @@ func (m *module) List(ctx context.Context, orgID valuer.UUID, params *metricredu
}
now := time.Now()
startMs := now.Add(-statsLookback).UnixMilli()
startMs := now.Add(-defaultPreviewLookback).UnixMilli()
endMs := now.UnixMilli()
switch params.OrderBy {
@@ -113,14 +107,10 @@ func (m *module) listSortedByColumn(ctx context.Context, orgID valuer.UUID, para
if err != nil {
return nil, err
}
sampleVolumes, err := m.ch.SampleVolumeByMetric(ctx, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
rules := make([]metricreductionruletypes.GettableReductionRule, 0, len(domainRules))
for _, rule := range domainRules {
rules = append(rules, withVolume(toGettableReductionRule(rule), volumes[rule.MetricName], sampleVolumes[rule.MetricName]))
rules = append(rules, withVolume(toGettableReductionRule(rule), volumes[rule.MetricName]))
}
return &metricreductionruletypes.GettableReductionRules{Rules: rules, Total: total}, nil
@@ -149,24 +139,13 @@ func (m *module) listSortedByVolume(ctx context.Context, orgID valuer.UUID, para
return nil, err
}
pageMetricNames := make([]string, 0, len(ranked))
for _, row := range ranked {
pageMetricNames = append(pageMetricNames, row.MetricName)
}
// TODO(srikanthccv): do we need to run this query? can we just get the same from RankByVolume?
sampleVolumes, err := m.ch.SampleVolumeByMetric(ctx, pageMetricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
rules := make([]metricreductionruletypes.GettableReductionRule, 0, len(ranked))
for _, row := range ranked {
rule, ok := ruleByMetric[row.MetricName]
if !ok {
continue
}
rules = append(rules, withVolume(toGettableReductionRule(rule), row, sampleVolumes[row.MetricName]))
rules = append(rules, withVolume(toGettableReductionRule(rule), row))
}
return &metricreductionruletypes.GettableReductionRules{Rules: rules, Total: total}, nil
@@ -309,17 +288,20 @@ func (m *module) Stats(ctx context.Context, orgID valuer.UUID) (*metricreduction
}
now := time.Now()
startMs := now.Add(-statsLookback).UnixMilli()
startMs := now.Add(-defaultPreviewLookback).UnixMilli()
endMs := now.UnixMilli()
rules, _, err := m.store.List(ctx, orgID, &metricreductionruletypes.ListReductionRulesParams{})
allRules, total, err := m.store.List(ctx, orgID, &metricreductionruletypes.ListReductionRulesParams{})
if err != nil {
return nil, err
}
if total == 0 {
return &metricreductionruletypes.GettableReductionRuleStats{}, nil
}
metricNames := make([]string, len(rules))
effectiveFrom := make(map[string]int64, len(rules))
for i, rule := range rules {
metricNames := make([]string, len(allRules))
effectiveFrom := make(map[string]int64, len(allRules))
for i, rule := range allRules {
metricNames[i] = rule.MetricName
effectiveFrom[rule.MetricName] = rule.EffectiveFrom.UnixMilli()
}
@@ -328,43 +310,31 @@ func (m *module) Stats(ctx context.Context, orgID valuer.UUID) (*metricreduction
if err != nil {
return nil, err
}
var ruledIngestedSeries, ruledRetainedSeries uint64
for _, volume := range volumes {
ruledIngestedSeries += volume.Ingested
ruledRetainedSeries += effectiveRetained(volume.Ingested, volume.Reduced)
var ingestedSeries, retainedSeries uint64
reducedMetricNames := make([]string, 0, len(volumes))
reducedEffectiveFrom := make(map[string]int64, len(volumes))
for name, volume := range volumes {
ingestedSeries += volume.Ingested
retained := effectiveRetained(volume.Ingested, volume.Reduced)
retainedSeries += retained
if retained < volume.Ingested {
reducedMetricNames = append(reducedMetricNames, name)
reducedEffectiveFrom[name] = effectiveFrom[name]
}
}
sampleVolumes, err := m.ch.SampleVolumeByMetric(ctx, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
var ruledIngestedSamples, ruledRetainedSamples uint64
for _, sv := range sampleVolumes {
ruledIngestedSamples += sv.Ingested
ruledRetainedSamples += effectiveRetained(sv.Ingested, sv.Reduced)
}
totalSeries, totalSamples, err := m.ch.TotalVolume(ctx, startMs, endMs)
ingestedSamples, reducedSamples, err := m.ch.SampleVolume(ctx, reducedMetricNames, reducedEffectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
return &metricreductionruletypes.GettableReductionRuleStats{
IngestedSeries: totalSeries,
RetainedSeries: clampSub(totalSeries, ruledIngestedSeries-ruledRetainedSeries),
IngestedSamples: totalSamples,
RetainedSamples: clampSub(totalSamples, ruledIngestedSamples-ruledRetainedSamples),
EstimatedMonthlySavingsUsd: monthlySavingsUSD(ruledIngestedSamples, ruledRetainedSamples, startMs, endMs),
IngestedSeries: ingestedSeries,
RetainedSeries: retainedSeries,
EstimatedMonthlySavingsUsd: monthlySavingsUSD(ingestedSamples, reducedSamples, startMs, endMs),
}, nil
}
func clampSub(a, b uint64) uint64 {
if a < b {
return 0
}
return a - b
}
// monthlySavingsUSD extrapolates the windowed sample reduction to a monthly figure at the per-sample
// list price. Ingested is gated to effective_from upstream, so pre-activation hours don't inflate it.
func monthlySavingsUSD(ingestedSamples, reducedSamples uint64, startMs, endMs int64) float64 {
@@ -382,7 +352,7 @@ func (m *module) Timeseries(ctx context.Context, orgID valuer.UUID) (*querybuild
}
now := time.Now()
startMs := now.Add(-timeseriesLookback).UnixMilli()
startMs := now.Add(-defaultPreviewLookback).UnixMilli()
endMs := now.UnixMilli()
allRules, _, err := m.store.List(ctx, orgID, &metricreductionruletypes.ListReductionRulesParams{})
@@ -396,7 +366,18 @@ func (m *module) Timeseries(ctx context.Context, orgID valuer.UUID) (*querybuild
effectiveFrom[rule.MetricName] = rule.EffectiveFrom.UnixMilli()
}
points, err := m.ch.SampleTimeseries(ctx, metricNames, effectiveFrom, startMs, endMs)
volumes, err := m.ch.VolumeByMetric(ctx, metricNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
reducedNames := make([]string, 0, len(volumes))
for name, volume := range volumes {
if effectiveRetained(volume.Ingested, volume.Reduced) < volume.Ingested {
reducedNames = append(reducedNames, name)
}
}
points, err := m.ch.SeriesTimeseries(ctx, metricNames, reducedNames, effectiveFrom, startMs, endMs)
if err != nil {
return nil, err
}
@@ -433,7 +414,7 @@ func buildVolumeTimeseries(points []volumePoint) *querybuildertypesv5.QueryRange
}
func (m *module) validateMetricForReduction(ctx context.Context, orgID valuer.UUID, metricName string) error {
lastSeen, err := m.metadataStore.FetchLastSeenInfoMulti(ctx, orgID, metricName)
lastSeen, err := m.metadataStore.FetchLastSeenInfoMulti(ctx, metricName)
if err != nil {
return err
}
@@ -466,12 +447,12 @@ func (m *module) relatedAssetImpact(ctx context.Context, orgID valuer.UUID, metr
m.logger.WarnContext(ctx, "failed to fetch related dashboards for reduction preview", slog.String("metric_name", metricName), errors.Attr(err))
} else {
for _, item := range dashboards[metricName] {
usedLabels := append(append([]string{}, item.GroupBy...), item.FilterBy...)
usedLabels := append(splitCSV(item["group_by"]), splitCSV(item["filter_by"])...)
affected = append(affected, metricreductionruletypes.AffectedAsset{
Type: metricreductionruletypes.AssetTypeDashboard,
ID: item.DashboardID,
Name: item.DashboardName,
Widget: &metricreductionruletypes.AffectedWidget{ID: item.PanelID, Name: item.PanelName},
ID: item["dashboard_id"],
Name: item["dashboard_name"],
Widget: &metricreductionruletypes.AffectedWidget{ID: item["widget_id"], Name: item["widget_name"]},
ImpactedLabels: intersectLabels(usedLabels, droppedSet),
})
}
@@ -501,7 +482,7 @@ func toGettableReductionRule(rule *metricreductionruletypes.ReductionRule) metri
MatchType: rule.MatchType,
Labels: rule.Labels,
EffectiveFrom: rule.EffectiveFrom,
Active: !rule.EffectiveFrom.Add(uiActivationDelay).After(time.Now()),
Active: !rule.EffectiveFrom.After(time.Now()),
}
}
@@ -512,11 +493,12 @@ func effectiveRetained(ingested, reduced uint64) uint64 {
return reduced
}
func withVolume(rule metricreductionruletypes.GettableReductionRule, series volumeRow, samples volumeRow) metricreductionruletypes.GettableReductionRule {
rule.IngestedSeries = series.Ingested
rule.RetainedSeries = effectiveRetained(series.Ingested, series.Reduced)
rule.IngestedSamples = samples.Ingested
rule.RetainedSamples = effectiveRetained(samples.Ingested, samples.Reduced)
func withVolume(rule metricreductionruletypes.GettableReductionRule, volume volumeRow) metricreductionruletypes.GettableReductionRule {
rule.IngestedSeries = volume.Ingested
rule.RetainedSeries = effectiveRetained(volume.Ingested, volume.Reduced)
if volume.Ingested > 0 {
rule.ReductionPercent = (1 - float64(rule.RetainedSeries)/float64(volume.Ingested)) * 100
}
return rule
}
@@ -536,6 +518,13 @@ func intersectLabels(keys []string, droppedSet map[string]struct{}) []string {
return out
}
func splitCSV(s string) []string {
if s == "" {
return nil
}
return strings.Split(s, ",")
}
func resolveDroppedKept(matchType metricreductionruletypes.MatchType, ruleLabels, keys []string) (dropped, kept []string) {
ruleSet := make(map[string]struct{}, len(ruleLabels))
for _, l := range ruleLabels {

View File

@@ -35,7 +35,7 @@ func (s *store) List(ctx context.Context, orgID valuer.UUID, params *metricreduc
Where("org_id = ?", orgID).
Order(column + " " + direction)
if params.Search != "" {
query = query.Where("metric_name LIKE ? ESCAPE '\\'", "%"+s.sqlstore.Formatter().EscapeLikePattern(params.Search)+"%")
query = query.Where("metric_name LIKE ?", "%"+params.Search+"%")
}
if params.MetricName != "" {
query = query.Where("metric_name = ?", params.MetricName)

View File

@@ -101,10 +101,6 @@ func (h *handler) QueryRange(rw http.ResponseWriter, req *http.Request) {
h.community.QueryRange(rw, req)
}
func (h *handler) QueryRangePreview(rw http.ResponseWriter, req *http.Request) {
h.community.QueryRangePreview(rw, req)
}
func (h *handler) QueryRawStream(rw http.ResponseWriter, req *http.Request) {
h.community.QueryRawStream(rw, req)
}

View File

@@ -4,6 +4,7 @@ import (
"net/http"
"time"
"github.com/SigNoz/signoz/ee/licensing/httplicensing"
"github.com/SigNoz/signoz/ee/query-service/usage"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/http/middleware"
@@ -41,6 +42,7 @@ func NewAPIHandler(opts APIHandlerOptions, signoz *signoz.SigNoz, config signoz.
IntegrationsController: opts.IntegrationsController,
LogsParsingPipelineController: opts.LogsParsingPipelineController,
FluxInterval: opts.FluxInterval,
LicensingAPI: httplicensing.NewLicensingAPI(signoz.Licensing),
Signoz: signoz,
QueryParserAPI: queryparser.NewAPI(signoz.Instrumentation.ToProviderSettings(), signoz.QueryParser),
}, config)
@@ -65,11 +67,20 @@ func (ah *APIHandler) RegisterRoutes(router *mux.Router, am *middleware.AuthZ) {
// note: add ee override methods first
// routes available only in ee version
router.HandleFunc("/api/v1/features", am.OpenAccess(ah.getFeatureFlags)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/features", am.ViewAccess(ah.getFeatureFlags)).Methods(http.MethodGet)
// base overrides
router.HandleFunc("/api/v1/version", am.OpenAccess(ah.getVersion)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/checkout", am.AdminAccess(ah.LicensingAPI.Checkout)).Methods(http.MethodPost)
router.HandleFunc("/api/v1/billing", am.AdminAccess(ah.getBilling)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/portal", am.AdminAccess(ah.LicensingAPI.Portal)).Methods(http.MethodPost)
// v3
router.HandleFunc("/api/v3/licenses", am.AdminAccess(ah.LicensingAPI.Activate)).Methods(http.MethodPost)
router.HandleFunc("/api/v3/licenses", am.AdminAccess(ah.LicensingAPI.Refresh)).Methods(http.MethodPut)
router.HandleFunc("/api/v3/licenses/active", am.ViewAccess(ah.LicensingAPI.GetActive)).Methods(http.MethodGet)
// v4
router.HandleFunc("/api/v4/query_range", am.ViewAccess(ah.queryRangeV4)).Methods(http.MethodPost)

View File

@@ -141,7 +141,7 @@ func (ah *APIHandler) getOrCreateCloudIntegrationServiceAccount(ctx context.Cont
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration service account: %w", err))
}
_, err = ah.Signoz.Modules.ServiceAccount.SetRoleByName(ctx, orgId, cloudIntegrationServiceAccount.ID, authtypes.SigNozViewerRoleName)
err = ah.Signoz.Modules.ServiceAccount.SetRoleByName(ctx, orgId, cloudIntegrationServiceAccount.ID, authtypes.SigNozViewerRoleName)
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration service account: %w", err))
}

View File

@@ -80,6 +80,24 @@ func (ah *APIHandler) getFeatureFlags(w http.ResponseWriter, r *http.Request) {
Route: "",
})
fineGrainedAuthz := ah.Signoz.Flagger.BooleanOrEmpty(ctx, flagger.FeatureUseFineGrainedAuthz, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureUseFineGrainedAuthz.String()),
Active: fineGrainedAuthz,
Usage: 0,
UsageLimit: -1,
Route: "",
})
useDashboardV2 := ah.Signoz.Flagger.BooleanOrEmpty(ctx, flagger.FeatureUseDashboardV2, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureUseDashboardV2.String()),
Active: useDashboardV2,
Usage: 0,
UsageLimit: -1,
Route: "",
})
aiObservability := ah.Signoz.Flagger.BooleanOrEmpty(ctx, flagger.FeatureEnableAIObservability, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureEnableAIObservability.String()),
@@ -98,6 +116,14 @@ func (ah *APIHandler) getFeatureFlags(w http.ResponseWriter, r *http.Request) {
Route: "",
})
if constants.IsDotMetricsEnabled {
for idx, feature := range featureSet {
if feature.Name == licensetypes.DotMetricsEnabled {
featureSet[idx].Active = true
}
}
}
ah.Respond(w, featureSet)
}

View File

@@ -0,0 +1,122 @@
package api
import (
"encoding/json"
"fmt"
"net/http"
"github.com/SigNoz/signoz/ee/query-service/constants"
"github.com/SigNoz/signoz/ee/query-service/model"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type DayWiseBreakdown struct {
Type string `json:"type"`
Breakdown []DayWiseData `json:"breakdown"`
}
type DayWiseData struct {
Timestamp int64 `json:"timestamp"`
Count float64 `json:"count"`
Size float64 `json:"size"`
UnitPrice float64 `json:"unitPrice"`
Quantity float64 `json:"quantity"`
Total float64 `json:"total"`
}
type tierBreakdown struct {
UnitPrice float64 `json:"unitPrice"`
Quantity float64 `json:"quantity"`
TierStart int64 `json:"tierStart"`
TierEnd int64 `json:"tierEnd"`
TierCost float64 `json:"tierCost"`
}
type usageResponse struct {
Type string `json:"type"`
Unit string `json:"unit"`
Tiers []tierBreakdown `json:"tiers"`
DayWiseBreakdown DayWiseBreakdown `json:"dayWiseBreakdown"`
}
type details struct {
Total float64 `json:"total"`
Breakdown []usageResponse `json:"breakdown"`
BaseFee float64 `json:"baseFee"`
BillTotal float64 `json:"billTotal"`
}
type billingData struct {
BillingPeriodStart int64 `json:"billingPeriodStart"`
BillingPeriodEnd int64 `json:"billingPeriodEnd"`
Details details `json:"details"`
Discount float64 `json:"discount"`
SubscriptionStatus string `json:"subscriptionStatus"`
}
type billingDetails struct {
Status string `json:"status"`
Data billingData `json:"data"`
}
func (ah *APIHandler) getBilling(w http.ResponseWriter, r *http.Request) {
licenseKey := r.URL.Query().Get("licenseKey")
if licenseKey == "" {
RespondError(w, model.BadRequest(fmt.Errorf("license key is required")), nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)
useZeus := ah.Signoz.Flagger.BooleanOrEmpty(r.Context(), flagger.FeatureGetMetersFromZeus, evalCtx)
if useZeus {
data, err := ah.Signoz.Zeus.GetMeters(r.Context(), licenseKey)
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
var billing billingData
if err := json.Unmarshal(data, &billing); err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
ah.Respond(w, billing)
return
}
billingURL := fmt.Sprintf("%s/usage?licenseKey=%s", constants.LicenseSignozIo, licenseKey)
hClient := &http.Client{}
req, err := http.NewRequest("GET", billingURL, nil)
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
req.Header.Add("X-SigNoz-SecretKey", constants.LicenseAPIKey)
billingResp, err := hClient.Do(req)
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
var billingResponse billingDetails
if err := json.NewDecoder(billingResp.Body).Decode(&billingResponse); err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
ah.Respond(w, billingResponse.Data)
}

View File

@@ -64,7 +64,6 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
signoz.Prometheus,
signoz.TelemetryStore.Cluster(),
signoz.Cache,
signoz.Flagger,
nil,
)
@@ -183,7 +182,9 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
apiHandler.RegisterLogsRoutes(r, am)
apiHandler.RegisterIntegrationRoutes(r, am)
apiHandler.RegisterQueryRangeV3Routes(r, am)
apiHandler.RegisterInfraMetricsRoutes(r, am)
apiHandler.RegisterQueryRangeV4Routes(r, am)
apiHandler.RegisterWebSocketPaths(r, am)
apiHandler.RegisterMessagingQueuesRoutes(r, am)
apiHandler.RegisterThirdPartyApiRoutes(r, am)
apiHandler.RegisterTraceFunnelsRoutes(r, am)
@@ -196,7 +197,7 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
c := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET", "DELETE", "POST", "PUT", "PATCH", "OPTIONS"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control"},
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "cache-control", "X-SIGNOZ-QUERY-ID", "Sec-WebSocket-Protocol"},
})
handler := c.Handler(r)

View File

@@ -4,6 +4,8 @@ import (
"os"
)
var LicenseSignozIo = "https://license.signoz.io/api/v1"
var LicenseAPIKey = GetOrDefaultEnv("SIGNOZ_LICENSE_API_KEY", "")
var SaasSegmentKey = GetOrDefaultEnv("SIGNOZ_SAAS_SEGMENT_KEY", "")
var FetchFeatures = GetOrDefaultEnv("FETCH_FEATURES", "false")
var ZeusFeaturesURL = GetOrDefaultEnv("ZEUS_FEATURES_URL", "ZeusFeaturesURL")
@@ -15,3 +17,15 @@ func GetOrDefaultEnv(key string, fallback string) string {
}
return v
}
// constant functions that override env vars
const DotMetricsEnabled = "DOT_METRICS_ENABLED"
var IsDotMetricsEnabled = false
func init() {
if GetOrDefaultEnv(DotMetricsEnabled, "true") == "true" {
IsDotMetricsEnabled = true
}
}

View File

@@ -240,17 +240,17 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
mock := mockStore.Mock()
// Mock the fingerprint query (for Prometheus label matching)
// args: $1=metric_name (the __name__ matcher maps onto the column)
mock.ExpectQuery("SELECT fingerprint, any").
WithArgs("test_metric").
WithArgs("test_metric", "__name__", "test_metric").
WillReturnRows(fingerprintRows)
// Mock the samples query (for Prometheus metric data)
// args: metric_name IN (discovered names), subquery metric_name, start, end
mock.ExpectQuery("SELECT metric_name, fingerprint, unix_milli").
WithArgs(
"test_metric",
"test_metric",
"__name__",
"test_metric",
queryStart,
queryEnd,
).

View File

@@ -3,7 +3,6 @@ package postgressqlschema
import (
"context"
"database/sql"
"log/slog"
"github.com/uptrace/bun"
@@ -215,20 +214,6 @@ WHERE
}
func (provider *provider) GetIndices(ctx context.Context, name sqlschema.TableName) ([]sqlschema.Index, error) {
// A key is one entry an index is built on: a column (plain) or an expression
// like lower(x) (functional). Returns one row per key of each non-constraint-
// backed index on the table, ordered by position. Keys are enumerated with
// generate_series over indnkeyatts (count of key columns, excluding non-key
// INCLUDE columns), so expression keys are covered too. Reconstruction uses:
//
// index_name | index the key belongs to
// is_expression | true if the key is an expression, not a plain column
// key_def | rendered key text, e.g. org_id or lower(key)
// column_name | column name for plain keys; NULL for expression keys
// predicate | partial-index WHERE; NULL if not partial (same every row)
//
// table_name/unique/primary/column_position are carried over from the prior
// query (unique gates the loop, column_position orders the keys).
rows, err := provider.
sqlstore.
BunDB().
@@ -239,21 +224,19 @@ SELECT
i.indisunique AS unique,
i.indisprimary AS primary,
a.attname AS column_name,
n AS column_position,
(i.indkey[n - 1] = 0) AS is_expression,
pg_get_indexdef(i.indexrelid, n, true) AS key_def,
array_position(i.indkey, a.attnum) AS column_position,
pg_get_expr(i.indpred, i.indrelid) AS predicate
FROM
pg_index i
LEFT JOIN pg_class ct ON ct.oid = i.indrelid
LEFT JOIN pg_class ci ON ci.oid = i.indexrelid
LEFT JOIN pg_attribute a ON a.attrelid = ct.oid
LEFT JOIN pg_constraint con ON con.conindid = i.indexrelid
CROSS JOIN generate_series(1, i.indnkeyatts) AS n
LEFT JOIN pg_attribute a ON a.attrelid = ct.oid AND a.attnum = i.indkey[n - 1]
WHERE
ct.relname = ?
AND ct.relkind = 'r'
a.attnum = ANY(i.indkey)
AND con.oid IS NULL
AND ct.relkind = 'r'
AND ct.relname = ?
ORDER BY index_name, column_position`, string(name))
if err != nil {
return nil, provider.sqlstore.WrapNotFoundErrf(err, errors.CodeNotFound, "no indices for table (%s) found", name)
@@ -268,10 +251,6 @@ ORDER BY index_name, column_position`, string(name))
type indexEntry struct {
columns []sqlschema.ColumnName
predicate *string
// keyDefs holds every key rendered by pg_get_indexdef, in key order; used
// for functional indexes where a key may be an expression like lower(x).
keyDefs []string
hasExpression bool
}
uniqueIndicesMap := make(map[string]*indexEntry)
@@ -281,45 +260,30 @@ ORDER BY index_name, column_position`, string(name))
indexName string
unique bool
primary bool
columnName *string
// starts from 1 and is unused in this function, this is to ensure that the column names are in the correct order
columnName string
// starts from 0 and is unused in this function, this is to ensure that the column names are in the correct order
columnPosition int
isExpression bool
keyDef string
predicate *string
)
if err := rows.Scan(&tableName, &indexName, &unique, &primary, &columnName, &columnPosition, &isExpression, &keyDef, &predicate); err != nil {
if err := rows.Scan(&tableName, &indexName, &unique, &primary, &columnName, &columnPosition, &predicate); err != nil {
return nil, err
}
if !unique {
continue
}
entry, ok := uniqueIndicesMap[indexName]
if !ok {
entry = &indexEntry{predicate: predicate}
uniqueIndicesMap[indexName] = entry
}
entry.keyDefs = append(entry.keyDefs, keyDef)
if isExpression {
entry.hasExpression = true
} else if columnName != nil {
entry.columns = append(entry.columns, sqlschema.ColumnName(*columnName))
if unique {
if _, ok := uniqueIndicesMap[indexName]; !ok {
uniqueIndicesMap[indexName] = &indexEntry{
columns: []sqlschema.ColumnName{sqlschema.ColumnName(columnName)},
predicate: predicate,
}
} else {
uniqueIndicesMap[indexName].columns = append(uniqueIndicesMap[indexName].columns, sqlschema.ColumnName(columnName))
}
}
}
indices := make([]sqlschema.Index, 0)
for indexName, entry := range uniqueIndicesMap {
// functional partial indexes aren't representable (PartialUniqueIndex has no
// expressions); skip rather than misrepresent.
if entry.hasExpression && entry.predicate != nil {
provider.settings.Logger().WarnContext(ctx, "skipping functional partial unique index; not representable by sqlschema", slog.String("index", indexName), slog.String("table", string(name)))
continue
}
if entry.predicate != nil {
index := &sqlschema.PartialUniqueIndex{
TableName: name,
@@ -327,17 +291,6 @@ ORDER BY index_name, column_position`, string(name))
Where: *entry.predicate,
}
if index.Name() == indexName {
indices = append(indices, index)
} else {
indices = append(indices, index.Named(indexName))
}
} else if entry.hasExpression {
index := &sqlschema.UniqueIndexWithExpressions{
TableName: name,
Expressions: entry.keyDefs,
}
if index.Name() == indexName {
indices = append(indices, index)
} else {

View File

@@ -1,116 +0,0 @@
package postgressqlschema
import (
"context"
"database/sql/driver"
"testing"
"github.com/DATA-DOG/go-sqlmock"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstoretest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestGetIndices feeds GetIndices the exact rows the catalog query returns for a
// real table (captured from devenv postgres) and checks each unique-index shape
// is reconstructed: expression keys, partial (predicate), and plain columns.
func TestGetIndices(t *testing.T) {
// column order must match the SELECT in GetIndices.
columns := []string{"table_name", "index_name", "unique", "primary", "column_name", "column_position", "is_expression", "key_def", "predicate"}
testCases := []struct {
name string
table sqlschema.TableName
rows [][]driver.Value
want []sqlschema.Index
}{
{
name: "UniqueIndexWithExpressions",
table: "tag",
rows: [][]driver.Value{
{"tag", "uq_tag_a36c51df", true, false, "org_id", 1, false, "org_id", nil},
{"tag", "uq_tag_a36c51df", true, false, "kind", 2, false, "kind", nil},
{"tag", "uq_tag_a36c51df", true, false, nil, 3, true, "lower(key)", nil},
{"tag", "uq_tag_a36c51df", true, false, nil, 4, true, "lower(value)", nil},
},
want: []sqlschema.Index{
&sqlschema.UniqueIndexWithExpressions{
TableName: "tag",
Expressions: []string{"org_id", "kind", "lower(key)", "lower(value)"},
},
},
},
{
name: "PartialUniqueIndexes",
table: "service_account",
rows: [][]driver.Value{
{"service_account", "puq_service_account_email_org_id_471d6134", true, false, "email", 1, false, "email", "(status <> 'deleted'::text)"},
{"service_account", "puq_service_account_email_org_id_471d6134", true, false, "org_id", 2, false, "org_id", "(status <> 'deleted'::text)"},
{"service_account", "puq_service_account_name_org_id_471d6134", true, false, "name", 1, false, "name", "(status <> 'deleted'::text)"},
{"service_account", "puq_service_account_name_org_id_471d6134", true, false, "org_id", 2, false, "org_id", "(status <> 'deleted'::text)"},
},
want: []sqlschema.Index{
(&sqlschema.PartialUniqueIndex{
TableName: "service_account",
ColumnNames: []sqlschema.ColumnName{"email", "org_id"},
Where: "(status <> 'deleted'::text)",
}).Named("puq_service_account_email_org_id_471d6134"),
(&sqlschema.PartialUniqueIndex{
TableName: "service_account",
ColumnNames: []sqlschema.ColumnName{"name", "org_id"},
Where: "(status <> 'deleted'::text)",
}).Named("puq_service_account_name_org_id_471d6134"),
},
},
{
name: "PlainUniqueIndex",
table: "tag_relation",
rows: [][]driver.Value{
{"tag_relation", "uq_tag_relation_kind_resource_id_tag_id", true, false, "kind", 1, false, "kind", nil},
{"tag_relation", "uq_tag_relation_kind_resource_id_tag_id", true, false, "resource_id", 2, false, "resource_id", nil},
{"tag_relation", "uq_tag_relation_kind_resource_id_tag_id", true, false, "tag_id", 3, false, "tag_id", nil},
},
want: []sqlschema.Index{
&sqlschema.UniqueIndex{
TableName: "tag_relation",
ColumnNames: []sqlschema.ColumnName{"kind", "resource_id", "tag_id"},
},
},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
store := sqlstoretest.New(sqlstore.Config{Provider: "postgres"}, sqlmock.QueryMatcherRegexp)
rows := sqlmock.NewRows(columns)
for _, row := range testCase.rows {
rows.AddRow(row...)
}
store.Mock().ExpectQuery("pg_index").WillReturnRows(rows)
schema, err := New(context.Background(), instrumentationtest.New().ToProviderSettings(), sqlschema.Config{}, store)
require.NoError(t, err)
indices, err := schema.GetIndices(context.Background(), testCase.table)
require.NoError(t, err)
assert.Len(t, indices, len(testCase.want))
// GetIndices iterates a map, so match by name rather than slice order.
got := make(map[string]sqlschema.Index, len(indices))
for _, index := range indices {
got[index.Name()] = index
}
for _, want := range testCase.want {
actual, ok := got[want.Name()]
if !assert.True(t, ok, "expected index %s not returned", want.Name()) {
// so that other items in the for loop can also be checked.
continue
}
assert.True(t, want.Equals(actual), "index %s reconstructed differently", want.Name())
}
})
}
}

View File

@@ -1,95 +0,0 @@
package httpsubscription
import (
"context"
"encoding/json"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/subscription"
"github.com/SigNoz/signoz/pkg/types/subscriptiontypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/SigNoz/signoz/pkg/zeus"
"github.com/tidwall/gjson"
)
const upstreamTimeout = 10 * time.Second
type provider struct {
zeus zeus.Zeus
licensing licensing.Licensing
}
func New(zeus zeus.Zeus, licensing licensing.Licensing) subscription.Subscription {
return &provider{
zeus: zeus,
licensing: licensing,
}
}
func (provider *provider) Create(ctx context.Context, organizationID valuer.UUID, postableSubscription *subscriptiontypes.PostableSubscription) (*subscriptiontypes.GettableSubscription, error) {
ctx, cancel := context.WithTimeout(ctx, upstreamTimeout)
defer cancel()
license, err := provider.licensing.GetActive(ctx, organizationID)
if err != nil {
return nil, err
}
body, err := json.Marshal(postableSubscription)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to marshal subscription payload")
}
response, err := provider.zeus.GetCheckoutURL(ctx, license.Key, body)
if err != nil {
if errors.Ast(err, errors.TypeAlreadyExists) {
return nil, errors.WithAdditionalf(err, "checkout has already been completed for this account. Please click 'Refresh Status' to sync your subscription")
}
return nil, err
}
return &subscriptiontypes.GettableSubscription{RedirectURL: gjson.GetBytes(response, "url").String()}, nil
}
func (provider *provider) Update(ctx context.Context, organizationID valuer.UUID, postableSubscription *subscriptiontypes.PostableSubscription) (*subscriptiontypes.GettableSubscription, error) {
ctx, cancel := context.WithTimeout(ctx, upstreamTimeout)
defer cancel()
license, err := provider.licensing.GetActive(ctx, organizationID)
if err != nil {
return nil, err
}
body, err := json.Marshal(postableSubscription)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to marshal subscription payload")
}
response, err := provider.zeus.GetPortalURL(ctx, license.Key, body)
if err != nil {
return nil, err
}
return &subscriptiontypes.GettableSubscription{RedirectURL: gjson.GetBytes(response, "url").String()}, nil
}
func (provider *provider) Get(ctx context.Context, organizationID valuer.UUID) (*subscriptiontypes.GettableSubscriptionUsage, error) {
license, err := provider.licensing.GetActive(ctx, organizationID)
if err != nil {
return nil, err
}
data, err := provider.zeus.GetMeters(ctx, license.Key)
if err != nil {
return nil, err
}
usage, err := subscriptiontypes.NewGettableSubscriptionUsage(data)
if err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, zeus.ErrCodeResponseMalformed, "failed to unmarshal subscription usage")
}
return usage, nil
}

View File

@@ -51,7 +51,7 @@ func New(ctx context.Context, providerSettings factory.ProviderSettings, config
}, nil
}
func (provider *Provider) GetLicense(ctx context.Context, key string) (*zeustypes.License, error) {
func (provider *Provider) GetLicense(ctx context.Context, key string) ([]byte, error) {
response, err := provider.do(
ctx,
provider.config.URL.JoinPath("/v2/licenses/me"),
@@ -63,12 +63,7 @@ func (provider *Provider) GetLicense(ctx context.Context, key string) (*zeustype
return nil, err
}
license := new(zeustypes.License)
if err := json.Unmarshal([]byte(gjson.GetBytes(response, "data").String()), license); err != nil {
return nil, errors.Wrapf(err, errors.TypeInternal, zeus.ErrCodeResponseMalformed, "failed to unmarshal license data")
}
return license, nil
return []byte(gjson.GetBytes(response, "data").String()), nil
}
func (provider *Provider) GetCheckoutURL(ctx context.Context, key string, body []byte) ([]byte, error) {

9
frontend/.gitignore vendored
View File

@@ -28,11 +28,4 @@ e2e/test-plan/saved-views/
e2e/test-plan/service-map/
e2e/test-plan/services/
e2e/test-plan/traces/
e2e/test-plan/user-preferences/
# Storybook
/storybook-static/
debug-storybook.log
# Storybook screenshot sweeps (scripts/story-shots.mjs)
/.story-shots/
e2e/test-plan/user-preferences/

View File

@@ -293,8 +293,6 @@
// Forces subpath imports (@signozhq/ui/<component>) instead of the eagerly-loaded barrel
"signoz/no-css-module-bracket-access": "warn",
// Prevents bracket access on CSS modules (styles['kebab-case']) which fails with camelCaseOnly config
"signoz/no-dashboard-fetch-outside-root": "error",
// Forces useDashboardFetchRequired() outside the root V2 pages (allowlisted in overrides below)
"no-restricted-globals": [
"error",
{
@@ -323,19 +321,13 @@
"name": "react",
"importNames": [
"createContext",
"useContext",
"useSyncExternalStore"
"useContext"
],
"message": "[State mgmt] React Context and hand-rolled external stores are deprecated. Migrate shared state to Zustand."
"message": "[State mgmt] React Context is deprecated. Migrate shared state to Zustand."
},
{
"name": "immer",
"message": "[State mgmt] Direct immer usage is deprecated. Use Zustand (which integrates immer via the immer middleware) instead."
},
{
"name": "api/generated/services/dashboard",
"importNames": ["patchDashboardV2", "usePatchDashboardV2"],
"message": "[dashboard-v2] Don't call patchDashboardV2/usePatchDashboardV2 directly — use useOptimisticPatch().patchAsync so spec edits update the react-query cache optimistically and reconcile on settle."
}
]
}
@@ -488,11 +480,8 @@
// Simplifies boolean returns
"sonarjs/prefer-while": "error",
// Suggests while loops over for loops
"sonarjs/elseif-without-else": "off",
"sonarjs/elseif-without-else": "off"
// Requires final else in if-else-if chains (was disabled)
"signoz/no-conditional-text-nodes-with-siblings": "warn",
// Vendored from eslint-plugin-react-google-translate
"signoz/no-return-text-nodes": "warn"
},
"ignorePatterns": [
"src/parser/*.ts",
@@ -564,29 +553,6 @@
"rules": {
"signoz/no-zustand-getstate-in-hooks": "off"
}
},
{
// Root dashboard pages own the fetch lifecycle; useDashboardFetchRequired wraps it.
// Everywhere else must use useDashboardFetchRequired().
"files": [
"src/pages/DashboardPage/DashboardPage.tsx",
"src/pages/DashboardPage/PanelEditorPage/PanelEditorPage.tsx",
"src/pages/DashboardPage/DashboardContainer/hooks/useDashboardFetchRequired.ts"
],
"rules": {
"signoz/no-dashboard-fetch-outside-root": "off"
}
},
{
// Dev-tooling CLIs: stdout is their output, and they carry ported pixel/heap
// algorithms that read worse when split up.
"files": [
"scripts/**"
],
"rules": {
"no-console": "off",
"sonarjs/cognitive-complexity": "off"
}
}
]
}

View File

@@ -1,89 +0,0 @@
import { dirname, resolve } from 'path';
import { fileURLToPath } from 'url';
import type { StorybookConfig } from '@storybook/react-vite';
import type { Plugin, PluginOption } from 'vite';
const srcPath = resolve(dirname(fileURLToPath(import.meta.url)), '../src');
/**
* Modules replaced for every story. Same idea as `moduleNameMapper` in
* `jest.config.ts`: the app keeps importing its own paths, Storybook resolves
* them to a mock. Regexes so only exact specifiers match: `lib/history` must
* not catch `lib/historyUtils`.
*
* Each replacement is typed as the module it stands in for, so drift is a
* compile error rather than a story that fails at render. The `jest` note on
* each entry is where the same import lands under the other runner. The two
* only diverge where the runner needs them to.
*/
const mockAliases = [
{
// jest: not replaced, jsdom drives a real browser history.
find: /^(?:src\/)?lib\/history$/,
replacement: `${srcPath}/storybook/navigation/history.alias.ts`,
},
{
// jest: src/__tests__/logEventMock.ts
find: /^(?:src\/)?api\/common\/logEvent$/,
replacement: `${srcPath}/storybook/mocks/logEvent.mock.ts`,
},
{
// jest: __mocks__/env.ts, which leaves `baseURL` empty because jsdom already
// resolves a relative `/api/...` against `http://localhost`.
find: /^(?:src\/)?constants\/env$/,
replacement: `${srcPath}/storybook/mocks/env.mock.ts`,
},
];
/**
* Plugins from `vite.config.ts` that either target the app's `index.html` or
* only pay off in a production build.
*/
const EXCLUDED_PLUGINS = [
'vite-plugin-checker',
'dev-base-path',
'dev-boot-data',
'vite-plugin-image-optimizer',
'vite-plugin-compression',
];
const isExcluded = (plugin: PluginOption): boolean =>
!!plugin &&
typeof plugin === 'object' &&
'name' in plugin &&
EXCLUDED_PLUGINS.includes((plugin as Plugin).name);
const config: StorybookConfig = {
framework: '@storybook/react-vite',
stories: ['../src/**/*.stories.@(ts|tsx)'],
// `../public` carries the fonts, icons and i18n bundles the app expects at
// the root; `./public` carries the msw worker, which must not ship in a
// production build.
staticDirs: ['../public', './public'],
addons: ['@storybook/addon-a11y'],
core: { disableTelemetry: true },
viteFinal: async (viteConfig) => {
const plugins = (viteConfig.plugins ?? [])
.flat(Infinity as 1)
.filter((plugin) => !isExcluded(plugin as PluginOption));
const existingAlias = viteConfig.resolve?.alias;
const normalizedAlias = Array.isArray(existingAlias)
? existingAlias
: Object.entries(existingAlias ?? {}).map(([find, replacement]) => ({
find,
replacement: replacement as string,
}));
return {
...viteConfig,
plugins,
resolve: {
...viteConfig.resolve,
alias: [...mockAliases, ...normalizedAlias],
},
};
},
};
export default config;

View File

@@ -1,10 +0,0 @@
/**
* Chromatic modes: one snapshot per entry, per story. The globals in a mode are
* Storybook's own, so `theme` is the toolbar's theme and the story renders the
* way it does locally. The width matches `scripts/story-shots.mjs` (`--width`),
* so a cloud snapshot and a local shot frame the same page.
*/
export const allModes = {
dark: { theme: 'dark', viewport: { width: 1680, height: 1200 } },
light: { theme: 'light', viewport: { width: 1680, height: 1200 } },
} as const;

View File

@@ -1,24 +0,0 @@
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Inter:ital,opsz,wght@0,14..32,100..900;1,14..32,100..900&display=swap"
rel="stylesheet"
/>
<link rel="stylesheet" href="css/uPlot.min.css" />
<script>
// i18n's language detector would pick the browser locale (e.g. `en-US`), which
// /public/locales does not ship; pin it to the bundled language instead.
window.localStorage.setItem('i18nextLng', 'en');
// The Go backend injects this at boot; every integration it enables is off in
// Storybook so no third-party script loads inside the iframe.
window.signozBootData = {
settings: {
posthog: { enabled: false, apiHost: '', key: '', uiHost: '' },
appcues: { enabled: false, appId: '' },
sentry: { enabled: false, dsn: '', tunnel: '' },
pylon: { enabled: false, appId: '', identitySecret: '' },
},
};
</script>

View File

@@ -1,133 +0,0 @@
import type { Preview } from '@storybook/react-vite';
import type { SetupWorker } from 'msw';
import { setupWorker } from 'msw';
import { settleForCapture } from '../src/storybook/visual/settleForCapture';
import { withProviders } from '../src/storybook/decorators/withProviders';
import { globalMocks } from '../src/storybook/globals';
import { resetStoryHistory } from '../src/storybook/navigation/containment';
import { clearBlockedNavigations } from '../src/storybook/navigation/blockedNavigationStore';
import {
resolveStory,
type StoryRuntimeContext,
} from '../src/storybook/runtime/resolveStory';
import { allModes } from './modes';
import '../src/ReactI18';
import '../src/styles.scss';
import '../src/storybook/storybook-root.scss';
interface StorybookWorkerHolder {
__signozStorybookWorker?: StorybookWorker;
}
const holder = window as unknown as StorybookWorkerHolder;
/**
* One worker per page, even if this module is re-executed by HMR. Two live
* workers both answer the service worker and the story gets whichever replies
* first.
*/
interface StorybookWorker {
worker: SetupWorker;
ready: Promise<unknown>;
}
const { worker, ready } = (holder.__signozStorybookWorker ??=
((): StorybookWorker => {
const instance = setupWorker();
return {
worker: instance,
ready: instance.start({
serviceWorker: { url: './mockServiceWorker.js' },
// Storybook's own traffic (index.json, HMR, telemetry) goes unhandled by
// design; only flag the app's API calls so a missing handler is obvious.
onUnhandledRequest: (request, print): void => {
const url = new URL(request.url.href);
const isStaticAsset =
/\.(?:woff2?|ttf|otf|css|js|map|png|jpe?g|svg|webp|ico)$/.test(
url.pathname,
);
const isAppRequest =
!isStaticAsset &&
(url.pathname.startsWith('/api/') || url.host !== window.location.host);
if (isAppRequest) {
print.warning();
}
},
}),
};
})());
const preview: Preview = {
parameters: {
layout: 'fullscreen',
controls: { expanded: true },
// One cloud snapshot per theme, for every story. A mode carries Storybook
// globals, so `theme` here is the same toolbar global the app reads out of
// localStorage. Widths are Chromatic's only real dimension, as they are
// locally: the app shell sizes itself to the viewport, so the height is the
// one it is given.
chromatic: { modes: allModes },
},
globalTypes: {
theme: {
description: 'SigNoz color scheme',
toolbar: {
title: 'Theme',
icon: 'paintbrush',
items: [
{ value: 'dark', title: 'Dark' },
{ value: 'light', title: 'Light' },
],
dynamicTitle: true,
},
},
motion: {
description:
'Park every animation on its last frame once the story has settled. Still is what both capture stacks shoot; Live is for watching a transition.',
toolbar: {
title: 'Motion',
icon: 'play',
items: [
{ value: 'still', title: 'Still' },
{ value: 'live', title: 'Live' },
],
dynamicTitle: true,
},
},
},
initialGlobals: { theme: 'dark', motion: 'still' },
// Controls every story carries: permissions, banners, and whether the page's
// own endpoints answer, hang or fail.
args: globalMocks.args,
argTypes: globalMocks.argTypes,
decorators: [withProviders],
loaders: [
// Runs on every render, args changes included, and ahead of the decorators:
// the whole story world is put in place here, so the provider tree only has
// to read it. Re-registering the handlers per render also means an edit to a
// handler module takes effect on the next render instead of leaving the
// worker on the set it was created with.
async (context): Promise<void> => {
const world = resolveStory(context as unknown as StoryRuntimeContext);
world.apply();
world.install(worker);
await ready;
},
],
beforeEach: () => {
clearBlockedNavigations();
resetStoryHistory();
},
// After `play`, which is the moment both capture stacks shoot at.
afterEach: settleForCapture,
};
export default preview;

View File

@@ -1,303 +0,0 @@
/* eslint-disable */
/* tslint:disable */
/**
* Mock Service Worker (1.3.2).
* @see https://github.com/mswjs/msw
* - Please do NOT modify this file.
* - Please do NOT serve this file on production.
*/
const INTEGRITY_CHECKSUM = '3d6b9f06410d179a7f7404d4bf4c3c70'
const activeClientIds = new Set()
self.addEventListener('install', function () {
self.skipWaiting()
})
self.addEventListener('activate', function (event) {
event.waitUntil(self.clients.claim())
})
self.addEventListener('message', async function (event) {
const clientId = event.source.id
if (!clientId || !self.clients) {
return
}
const client = await self.clients.get(clientId)
if (!client) {
return
}
const allClients = await self.clients.matchAll({
type: 'window',
})
switch (event.data) {
case 'KEEPALIVE_REQUEST': {
sendToClient(client, {
type: 'KEEPALIVE_RESPONSE',
})
break
}
case 'INTEGRITY_CHECK_REQUEST': {
sendToClient(client, {
type: 'INTEGRITY_CHECK_RESPONSE',
payload: INTEGRITY_CHECKSUM,
})
break
}
case 'MOCK_ACTIVATE': {
activeClientIds.add(clientId)
sendToClient(client, {
type: 'MOCKING_ENABLED',
payload: true,
})
break
}
case 'MOCK_DEACTIVATE': {
activeClientIds.delete(clientId)
break
}
case 'CLIENT_CLOSED': {
activeClientIds.delete(clientId)
const remainingClients = allClients.filter((client) => {
return client.id !== clientId
})
// Unregister itself when there are no more clients
if (remainingClients.length === 0) {
self.registration.unregister()
}
break
}
}
})
self.addEventListener('fetch', function (event) {
const { request } = event
const accept = request.headers.get('accept') || ''
// Bypass server-sent events.
if (accept.includes('text/event-stream')) {
return
}
// Bypass navigation requests.
if (request.mode === 'navigate') {
return
}
// Opening the DevTools triggers the "only-if-cached" request
// that cannot be handled by the worker. Bypass such requests.
if (request.cache === 'only-if-cached' && request.mode !== 'same-origin') {
return
}
// Bypass all requests when there are no active clients.
// Prevents the self-unregistered worked from handling requests
// after it's been deleted (still remains active until the next reload).
if (activeClientIds.size === 0) {
return
}
// Generate unique request ID.
const requestId = Math.random().toString(16).slice(2)
event.respondWith(
handleRequest(event, requestId).catch((error) => {
if (error.name === 'NetworkError') {
console.warn(
'[MSW] Successfully emulated a network error for the "%s %s" request.',
request.method,
request.url,
)
return
}
// At this point, any exception indicates an issue with the original request/response.
console.error(
`\
[MSW] Caught an exception from the "%s %s" request (%s). This is probably not a problem with Mock Service Worker. There is likely an additional logging output above.`,
request.method,
request.url,
`${error.name}: ${error.message}`,
)
}),
)
})
async function handleRequest(event, requestId) {
const client = await resolveMainClient(event)
const response = await getResponse(event, client, requestId)
// Send back the response clone for the "response:*" life-cycle events.
// Ensure MSW is active and ready to handle the message, otherwise
// this message will pend indefinitely.
if (client && activeClientIds.has(client.id)) {
;(async function () {
const clonedResponse = response.clone()
sendToClient(client, {
type: 'RESPONSE',
payload: {
requestId,
type: clonedResponse.type,
ok: clonedResponse.ok,
status: clonedResponse.status,
statusText: clonedResponse.statusText,
body:
clonedResponse.body === null ? null : await clonedResponse.text(),
headers: Object.fromEntries(clonedResponse.headers.entries()),
redirected: clonedResponse.redirected,
},
})
})()
}
return response
}
// Resolve the main client for the given event.
// Client that issues a request doesn't necessarily equal the client
// that registered the worker. It's with the latter the worker should
// communicate with during the response resolving phase.
async function resolveMainClient(event) {
const client = await self.clients.get(event.clientId)
if (client?.frameType === 'top-level') {
return client
}
const allClients = await self.clients.matchAll({
type: 'window',
})
return allClients
.filter((client) => {
// Get only those clients that are currently visible.
return client.visibilityState === 'visible'
})
.find((client) => {
// Find the client ID that's recorded in the
// set of clients that have registered the worker.
return activeClientIds.has(client.id)
})
}
async function getResponse(event, client, requestId) {
const { request } = event
const clonedRequest = request.clone()
function passthrough() {
// Clone the request because it might've been already used
// (i.e. its body has been read and sent to the client).
const headers = Object.fromEntries(clonedRequest.headers.entries())
// Remove MSW-specific request headers so the bypassed requests
// comply with the server's CORS preflight check.
// Operate with the headers as an object because request "Headers"
// are immutable.
delete headers['x-msw-bypass']
return fetch(clonedRequest, { headers })
}
// Bypass mocking when the client is not active.
if (!client) {
return passthrough()
}
// Bypass initial page load requests (i.e. static assets).
// The absence of the immediate/parent client in the map of the active clients
// means that MSW hasn't dispatched the "MOCK_ACTIVATE" event yet
// and is not ready to handle requests.
if (!activeClientIds.has(client.id)) {
return passthrough()
}
// Bypass requests with the explicit bypass header.
// Such requests can be issued by "ctx.fetch()".
if (request.headers.get('x-msw-bypass') === 'true') {
return passthrough()
}
// Notify the client that a request has been intercepted.
const clientMessage = await sendToClient(client, {
type: 'REQUEST',
payload: {
id: requestId,
url: request.url,
method: request.method,
headers: Object.fromEntries(request.headers.entries()),
cache: request.cache,
mode: request.mode,
credentials: request.credentials,
destination: request.destination,
integrity: request.integrity,
redirect: request.redirect,
referrer: request.referrer,
referrerPolicy: request.referrerPolicy,
body: await request.text(),
bodyUsed: request.bodyUsed,
keepalive: request.keepalive,
},
})
switch (clientMessage.type) {
case 'MOCK_RESPONSE': {
return respondWithMock(clientMessage.data)
}
case 'MOCK_NOT_FOUND': {
return passthrough()
}
case 'NETWORK_ERROR': {
const { name, message } = clientMessage.data
const networkError = new Error(message)
networkError.name = name
// Rejecting a "respondWith" promise emulates a network error.
throw networkError
}
}
return passthrough()
}
function sendToClient(client, message) {
return new Promise((resolve, reject) => {
const channel = new MessageChannel()
channel.port1.onmessage = (event) => {
if (event.data && event.data.error) {
return reject(event.data.error)
}
resolve(event.data)
}
client.postMessage(message, [channel.port2])
})
}
function sleep(timeMs) {
return new Promise((resolve) => {
setTimeout(resolve, timeMs)
})
}
async function respondWithMock(response) {
await sleep(response.delay)
return new Response(response.body, response)
}

View File

@@ -25,7 +25,6 @@ You are operating within a constrained context window and strict system prompts.
- Never create barrel files.
- When writing new css, prefer CSS Modules
- Use ./docs/css-modules-guide.md as reference on how to write good CSS Modules.
- When writing code that could need authorization checks, read ./src/lib/authz/README.md
3. FORCED VERIFICATION: Your internal tools mark file writes as successful even if the code does not compile. You are FORBIDDEN from reporting a task as complete until you have:
- Run `pnpm tsgo --noEmit`

View File

@@ -1,127 +1,105 @@
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="../docs/readme-assets/signoz-hero-dark.png" width="700">
<source media="(prefers-color-scheme: light)" srcset="../docs/readme-assets/signoz-hero-light.png" width="700">
<img alt="SigNoz - Observability on Your Terms" src="../docs/readme-assets/signoz-hero-light.png" width="700">
</picture>
</p>
# Configuring Over Local
1. Docker
1. Without Docker
<p align="center">
<a href="https://github.com/SigNoz/signoz/issues"><img alt="GitHub issues" src="https://img.shields.io/github/issues/SigNoz/signoz"></a>
<a href="https://signoz.io/slack"><img alt="Slack community" src="https://img.shields.io/badge/slack-community-4A154B?logo=slack&logoColor=white"></a>
</p>
## With Docker
# SigNoz Frontend
**Building image**
React-based web interface for [SigNoz](https://signoz.io), the open-source observability platform.
``docker compose up`
/ This will also run
## Tech Stack
or
`docker build . -t tagname`
- **Framework:** React 18 + TypeScript
- **Build:** Vite
- **State:** React Query, Zustand, Redux Toolkit (legacy)
- **Styling:** CSS Modules, Ant Design (legacy)
- **Charts:** uPlot
- **Testing:** Jest
## Local Development Setup
1. Run SigNoz backend locally — see [Self-Host Docs](https://signoz.io/docs/install/self-host/)
2. Configure environment:
```bash
cp example.env .env
```
Key variables in `.env`:
```bash
# Backend API endpoint (required)
VITE_FRONTEND_API_ENDPOINT="http://localhost:8080"
# Enable bundle analyzer (optional)
BUNDLE_ANALYSER="true"
```
3. Install and run:
```bash
pnpm install
pnpm dev
```
## Development
```bash
pnpm dev
```
Opens [http://localhost:3301](http://localhost:3301).
## Build
```bash
pnpm build
```
Output in `build/` folder.
## Bundle Size Analysis
Set in `.env`:
```bash
BUNDLE_ANALYSER="true"
```
Then run build:
```bash
pnpm build
```
Opens bundle analyzer visualization automatically.
## Testing
```bash
# Unit tests
pnpm test
# Type checking
pnpm tsgo --noEmit
```
## Storybook
```bash
pnpm storybook
```
Opens [http://localhost:6006](http://localhost:6006). Pages run against msw
mocks with no backend; query-param navigation works inside a story, leaving the
page is blocked. See [`src/storybook/README.md`](src/storybook/README.md) for the
override surface.
## Linting
```bash
# Run all linters (oxlint + stylelint)
pnpm lint
```
## Project Structure
**Tag to remote url- Introduce versioning later on**
```
src/
├── api/ # API clients and react-query hooks
├── components/ # Shared UI components
├── container/ # Page-level containers
├── hooks/ # Custom React hooks
├── pages/ # Route pages
├── providers/ # React context providers
├── store/ # Redux store
└── types/ # TypeScript definitions
docker tag signoz/frontend:latest 7296823551/signoz:latest
```
## Contributing
```
docker compose up
```
See [CONTRIBUTING.md](../CONTRIBUTING.md) in the root repo.
## Without Docker
Follow the steps below
Questions? Join our [Slack community](https://signoz.io/slack).
1. ```git clone https://github.com/SigNoz/signoz.git && cd signoz/frontend```
1. change baseURL to ```<test environment URL>``` in file ```src/constants/env.ts```
1. ```pnpm install```
1. ```pnpm dev```
```Note: Please ping us in #contributing channel in our slack community and we will DM you with <test environment URL>```
# Getting Started with Create React App
This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
## Available Scripts
In the project directory, you can run:
### `pnpm start`
Runs the app in the development mode.\
Open [http://localhost:3301](http://localhost:3301) to view it in the browser.
The page will reload if you make edits.\
You will also see any lint errors in the console.
### `pnpm test`
Launches the test runner in the interactive watch mode.\
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
### `pnpm build`
Builds the app for production to the `build` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.
The build is minified and the filenames include the hashes.\
Your app is ready to be deployed!
See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
### `pnpm eject`
**Note: this is a one-way operation. Once you `eject`, you cant go back!**
If you arent satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point youre on your own.
You dont have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldnt feel obligated to use this feature. However we understand that this tool wouldnt be useful if you couldnt customize it when you are ready for it.
## Learn More
You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
To learn React, check out the [React documentation](https://reactjs.org/).
### Code Splitting
This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting)
### Analyzing the Bundle Size
This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size)
### Making a Progressive Web App
This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app)
### Advanced Configuration
This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration)
### Deployment
This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment)
### `pnpm build` fails to minify
This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify)

View File

@@ -1,11 +0,0 @@
// Test stub for the Vite-only icon glob module: `import.meta.glob` can't be
// parsed by jest, so every test that transitively imports it is redirected here
// (see moduleNameMapper in jest.config.ts). Provides a minimal name → URL map so
// the resolver stays deterministic under test.
export const ICON_URLS: Record<string, string> = {
'eight-ball': 'mock-eight-ball-url',
};
export const LOGO_URLS: Record<string, string> = {
'aws-dark': 'mock-logo-url',
};

View File

@@ -1,3 +0,0 @@
export const IS_DEV = false;
export const IS_PROD = true;
export const MODE = 'test';

View File

@@ -1,88 +0,0 @@
// oxlint-disable-next-line no-restricted-imports
import * as React from 'react';
// In jsdom, AnimatePresence from motion/react keeps children in DOM during exit
// animations (awaiting rAF-driven completion that never fully runs in jsdom).
// This mock makes AnimatePresence render children immediately and makes motion.*
// elements render as their plain HTML equivalents without animation side-effects.
//
// IMPORTANT: motion component references are cached so React sees a stable
// component identity across re-renders and does not enter an infinite remount loop.
const MOTION_PROPS_TO_STRIP = new Set([
'initial',
'animate',
'exit',
'variants',
'transition',
'whileHover',
'whileTap',
'whileFocus',
'whileInView',
'layout',
'layoutId',
'onAnimationStart',
'onAnimationComplete',
]);
const cache = new Map<string, React.ComponentType>();
function getMotionComponent(tag: string): React.ComponentType {
if (!cache.has(tag)) {
const Component = React.forwardRef<HTMLElement, Record<string, unknown>>(
(props, ref) => {
const domProps: Record<string, unknown> = {};
for (const [k, v] of Object.entries(props)) {
if (!MOTION_PROPS_TO_STRIP.has(k)) {
domProps[k] = v;
}
}
return React.createElement(tag, { ...domProps, ref });
},
);
Component.displayName = `motion.${tag}`;
cache.set(tag, Component as unknown as React.ComponentType);
}
return cache.get(tag) as React.ComponentType;
}
const motionHandler: ProxyHandler<Record<string, React.ComponentType>> = {
get(_target, prop: string) {
return getMotionComponent(prop);
},
};
export const AnimatePresence: React.FC<{
children?: React.ReactNode;
mode?: string;
}> = ({ children }) => React.createElement(React.Fragment, null, children);
export const motion = new Proxy(
{} as Record<string, React.ComponentType>,
motionHandler,
);
export const useAnimation = (): Record<string, unknown> => ({
start: (): unknown => Promise.resolve(),
stop: (): unknown => undefined,
set: (): unknown => undefined,
});
export const useMotionValue = (
initial: unknown,
): { get: () => unknown; set: () => void } => ({
get: (): unknown => initial,
set: (): unknown => undefined,
});
export const useTransform = (): { get: () => number } => ({
get: (): number => 0,
});
export const useSpring = (v: unknown): unknown => v;
export const useScroll = (): { scrollY: { get: () => number } } => ({
scrollY: { get: (): number => 0 },
});
export default { motion, AnimatePresence };

View File

@@ -0,0 +1,29 @@
import { PropsWithChildren } from 'react';
type CommonProps = PropsWithChildren<{
className?: string;
minSize?: number;
maxSize?: number;
defaultSize?: number;
direction?: 'horizontal' | 'vertical';
autoSaveId?: string;
withHandle?: boolean;
}>;
export function ResizablePanelGroup({
children,
className,
}: CommonProps): JSX.Element {
return <div className={className}>{children}</div>;
}
export function ResizablePanel({
children,
className,
}: CommonProps): JSX.Element {
return <div className={className}>{children}</div>;
}
export function ResizableHandle({ className }: CommonProps): JSX.Element {
return <div className={className} />;
}

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 139 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 49 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 86 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 64 KiB

Some files were not shown because too many files have changed in this diff Show More