mirror of
https://github.com/SigNoz/signoz.git
synced 2026-07-28 16:50:40 +01:00
Compare commits
1 Commits
refactor/q
...
chore/clau
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
64f9d94e3e |
142
.claude/skills/authz/SKILL.md
Normal file
142
.claude/skills/authz/SKILL.md
Normal file
@@ -0,0 +1,142 @@
|
||||
---
|
||||
name: authz
|
||||
description: Add authorization (authz) to a SigNoz backend feature end-to-end — register kinds/resources, grant managed-role permissions, wire routes with ResourceDefs, backfill tuples for existing orgs, and write the authz integration tests. Use this whenever the user mentions authz, authorization, permissions, access control, managed roles (admin/editor/viewer), OpenFGA, ResourceDef, CheckResources, 403/forbidden behavior on an endpoint, wiring or protecting a new route, registering a new resource/kind, or writing integration tests that assert role-based allow/deny. Also use it when a new API endpoint is being added and its access control hasn't been decided yet — every route needs an authz decision even if the user didn't say the word.
|
||||
---
|
||||
|
||||
# SigNoz Authz Workflow
|
||||
|
||||
This skill automates the procedure documented in `docs/contributing/go/authz.md`. Read that doc first — it is the canonical explanation of the concepts (types, verbs, kinds, resources, selectors, tuples, the community/enterprise split). This skill tells you what to do; the doc tells you why it works.
|
||||
|
||||
The core model in one paragraph: permissions are never attached to users. They are attached to **roles** as OpenFGA tuples, and users/service accounts are made **assignees** of roles. You declare everything in code registries under `pkg/types/coretypes/`; tuples for new organizations are derived from those registries at org bootstrap. Routes declare what resource+verb they touch via `ResourceDef`s, and middleware turns that into a check that works in both community (role gate) and enterprise (per-resource FGA) editions.
|
||||
|
||||
## Before you start
|
||||
|
||||
Figure out what shape the work is. Ask the user if unclear:
|
||||
|
||||
- **New resource** (new kind + routes) → all steps below.
|
||||
- **New route on an existing resource** → step 4 only (maybe step 3 if roles need new verbs).
|
||||
- **Changing what a role can do** → steps 3, 5, 6, 7.
|
||||
- **Just tests** → steps 6, 7.
|
||||
|
||||
Almost every feature resource is a `metaresource` (dashboards, rules, pipelines) or a `telemetryresource` (logs, traces, metrics). If the feature genuinely needs a brand-new FGA **type**, stop and flag it to the user — that requires editing both `pkg/authz/openfgaschema/base.fga` and `ee/authz/openfgaschema/base.fga` together and is almost never the right call. A new **kind** never touches the schema.
|
||||
|
||||
## Step 1: Register the kind
|
||||
|
||||
Edit `pkg/types/coretypes/registry_kind.go`. Add a var and append it to the `Kinds` slice (both are required — the slice is what gets validated and iterated):
|
||||
|
||||
```go
|
||||
KindThing = MustNewKind("thing")
|
||||
```
|
||||
|
||||
Kind names are lowercase, hyphen-separated (`quick-filter`, `trace-funnel`, `notification-channel`).
|
||||
|
||||
Also add the kind to `Kind.Enum()` in `pkg/types/coretypes/kind.go` — a second hand-maintained list mirroring `Kinds` that feeds the OpenAPI enum. It is easy to miss because nothing fails at build time without it; the drift only shows up in the generated spec.
|
||||
|
||||
## Step 2: Register the resource
|
||||
|
||||
Edit `pkg/types/coretypes/registry_resource.go`. Add a var and append it to the `Resources` slice:
|
||||
|
||||
```go
|
||||
ResourceMetaResourceThing = NewResourceMetaResource(KindThing)
|
||||
```
|
||||
|
||||
Use `NewResourceTelemetryResource(KindThing)` for telemetry data resources. Pass an explicit verb list only when the resource supports fewer verbs than the type default — see `ResourceMetaResourceFactorAPIKey` in that file for the shape.
|
||||
|
||||
## Step 3: Grant permissions to managed roles
|
||||
|
||||
Edit `pkg/types/coretypes/registry_managed_role.go`. Add `Transaction` entries to `ManagedRoleToTransactions` for each role that may act on the resource:
|
||||
|
||||
```go
|
||||
{Verb: VerbCreate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindThing}, WildCardSelectorString)},
|
||||
```
|
||||
|
||||
The convention: **admin** gets everything; **editor** gets full CRUD only on day-to-day observability resources (dashboards, rules, saved views — not access control or org settings); **viewer** gets `read`/`list`; **anonymous** gets nothing (public dashboards are the only exception). Match the file's existing style, including its one-line grouping comment per resource block (e.g. `// thing — editors manage, viewers read`) — that file uses grouping comments as structure; do not add comments anywhere else.
|
||||
|
||||
This map is the single source of truth: changing it affects only organizations created afterwards (see step 5 for existing orgs), and the integration suite asserts its contents via a golden file (see step 6).
|
||||
|
||||
## Step 4: Wire the route
|
||||
|
||||
Register routes in the existing domain file under `pkg/apiserver/signozapiserver/` — do not create new per-sub-resource route files. Each route needs three coordinated pieces:
|
||||
|
||||
1. **`provider.authzMiddleware.CheckResources(handlerFn, roles...)`** — the role list is the community-edition fallback gate (which managed roles may call this route when per-resource checks are unavailable).
|
||||
2. **`handler.WithResourceDefs(...)`** — declares resource, verb, audit category, ID extractor, and selector.
|
||||
3. **`SecuritySchemes: newScopedSecuritySchemes([]string{resource.Scope(verb)})`** — advertises the scope in the OpenAPI spec.
|
||||
|
||||
Selector rules (getting these wrong silently over- or under-authorizes):
|
||||
|
||||
| Operation | Selector | ID extractor |
|
||||
|---|---|---|
|
||||
| create | `WildcardSelector` | `ResponseJSONPath("data.id")` (ID exists only after the handler runs; recorded for audit) |
|
||||
| list | `WildcardSelector` | none |
|
||||
| read/update/delete of one instance | `IDSelector` | `PathParam("id")` or `BodyJSONPath(...)` |
|
||||
| request ID ≠ FGA selector (e.g. route has a role UUID, FGA objects use role names) | custom `SelectorFunc` | as appropriate |
|
||||
|
||||
Routes that link two resources check **both** sides. Read `references/route-wiring.md` before wiring any create route, linking route (attach/detach, parent-child), or custom selector — it has complete real examples from `serviceaccount.go`.
|
||||
|
||||
Prefer `CheckResources` + `ResourceDef` for anything resource-shaped. The legacy gates `ViewAccess`/`EditAccess`/`AdminAccess` are coarse role-only checks — do not use them for new routes. `OpenAccess` means no authorization (authentication still applies); `CheckWithoutClaims` serves anonymous routes.
|
||||
|
||||
## Step 5: Backfill existing organizations (only if needed)
|
||||
|
||||
New organizations derive their tuples from the registries at creation — no migration needed for them. If **existing** organizations must get the new permissions, add a migration in `pkg/sqlmigration/` that does two things per org:
|
||||
|
||||
1. **Insert the FGA tuples** — follow `083_add_role_crud_tuples.go`: one `tuple` row (+ `changelog` row) per role/verb/object, with `ON CONFLICT ... DO NOTHING` so it stays idempotent, handling both the Postgres and SQLite tuple column layouts as that file does.
|
||||
2. **Refresh the managed roles' JSON record** — since `099_add_role_transaction_groups.go`, each role row carries a `transaction_groups` JSON column that mirrors its permissions. Tuples and JSON must move together or the stored record drifts from the registry. Follow 099's managed-role loop: marshal `authtypes.NewTransactionGroupsFromTransactions(coretypes.ManagedRoleToTransactions[roleName])` and `UPDATE role SET transaction_groups = ? WHERE org_id = ? AND type = 'managed' AND name = ?`. Serializing from the registry (which already contains your new grants) makes this step self-updating. Custom roles need no changes — only managed roles mirror the registry.
|
||||
|
||||
Register the new migration factory in `NewSQLMigrationProviderFactories` in `pkg/signoz/provider.go`, appended after the existing entries. The factory name passed to `factory.MustNewName` must match `^[a-z][a-z0-9_-]{0,30}$` — at most 31 characters, or it panics at startup; prefer a short name like `notification_template_tuples` over `add_notification_template_tuples`.
|
||||
|
||||
Ask the user whether backfill is needed rather than assuming — shipping a feature to existing orgs almost always means yes.
|
||||
|
||||
## Step 6: Update the golden testdata and regenerate CI-checked artifacts
|
||||
|
||||
If you changed `ManagedRoleToTransactions`, update `tests/integration/testdata/role/managed_role_transactions.json` to match — the `role/01_register.py` integration test compares each managed role's verb/type/kind matrix against this file and will fail otherwise.
|
||||
|
||||
If you touched the registries or `Kind.Enum()`, regenerate the derived artifacts. The `goci` GitHub workflow reruns each of these and fails the PR on any uncommitted diff:
|
||||
|
||||
```bash
|
||||
go run cmd/enterprise/*.go generate openapi # docs/api/openapi.yml
|
||||
go run cmd/enterprise/*.go generate authz # frontend/src/lib/authz/hooks/useAuthZ/permissions.type.ts
|
||||
go run cmd/enterprise/*.go generate config transaction-groups # frontend/src/schemas/generated/transactionGroups.schema.json
|
||||
```
|
||||
|
||||
Commit whatever they change; never hand-edit these generated files.
|
||||
|
||||
`generate authz` is allowlist-based: it only emits resources listed in the `allowedResources` map inside `runGenerateAuthz` in `cmd/authz.go`. For the new resource to surface in the UI's permission gating (AuthZButton and friends), add it there —
|
||||
|
||||
```go
|
||||
coretypes.NewResourceRef(coretypes.ResourceMetaResourceThing).String(): true,
|
||||
```
|
||||
|
||||
— then rerun `generate authz` and commit the regenerated `permissions.type.ts`. If the resource has no UI surface yet, skip the allowlist entry and the command produces no diff; that is expected.
|
||||
|
||||
## Step 7: Write integration tests
|
||||
|
||||
Read `references/integration-tests.md` for the suite layout, fixtures, and canonical allow/deny test shapes. The minimum bar for a new authz'd resource: each managed role's expected 200-vs-403 matrix on the new routes, and (if enterprise/FGA behavior matters) per-object scoping with a custom role — granted instance succeeds, non-granted instance gets 403, revoke flips it back to 403.
|
||||
|
||||
Python test conventions that are hard rules in this repo: inline `requests.*` calls directly (never wrap single API calls in helpers), always pass `timeout=`, and run `make py-fmt && make py-lint` before finishing.
|
||||
|
||||
## Verification
|
||||
|
||||
Run these before declaring the work done:
|
||||
|
||||
```bash
|
||||
make go-build-community
|
||||
```
|
||||
|
||||
Consistency greps — every one of these must hold:
|
||||
|
||||
- the new `Kind` var appears in BOTH the `Kinds` slice (`registry_kind.go`) and `Kind.Enum()` (`kind.go`)
|
||||
- the three `generate` commands from step 6 have been run and leave `git diff` clean when re-run
|
||||
- if the resource is UI-gated, it appears in `allowedResources` in `cmd/authz.go` and `permissions.type.ts` was regenerated
|
||||
- the new `Resource` var appears in the `Resources` slice (`registry_resource.go`)
|
||||
- every `ResourceDef` in the route file references the registered resource and one of its allowed verbs
|
||||
- every route's `SecuritySchemes` scope matches its `ResourceDef` verb
|
||||
- if `ManagedRoleToTransactions` changed, `managed_role_transactions.json` changed with it
|
||||
|
||||
Then run the relevant integration suites (`make py-test-setup` from the repo root boots the environment; pytest runs from `tests/`):
|
||||
|
||||
```bash
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/role/
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<your-suite>/
|
||||
```
|
||||
|
||||
General repo conventions apply throughout: no code comments (the grouping comments in `registry_managed_role.go` are the sole exception), complete descriptive names, and never commit without an explicit ask.
|
||||
114
.claude/skills/authz/references/integration-tests.md
Normal file
114
.claude/skills/authz/references/integration-tests.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Authz Integration Tests Reference
|
||||
|
||||
Existing authz suites to imitate: `integration/tests/role/` (managed roles, custom-role CRUD, FGA scoping), `integration/tests/serviceaccount/` (API-key auth by role, role assignment), `integration/tests/passwordauthn/04_role.py` (user role changes, editor denied on admin endpoints).
|
||||
|
||||
## Where a new test goes
|
||||
|
||||
- Suite package: `tests/integration/tests/<suite>/` — pick the existing suite matching the feature's domain, or create a new package when the feature is its own domain.
|
||||
- File name: two-digit ordered prefix, `NN_<topic>.py` (`03_fga.py`, `04_roles.py`). Files in a suite run in order; setup tests early in the file establish state that later tests use.
|
||||
- Golden/testdata JSON: `tests/integration/testdata/<suite>/`, loaded via `fixtures.fs.get_testdata_file_path`.
|
||||
- If the change touched `ManagedRoleToTransactions`, update `tests/integration/testdata/role/managed_role_transactions.json` — `role/01_register.py` asserts every managed role's verb/type/kind matrix against it.
|
||||
|
||||
## Environment and running
|
||||
|
||||
`make` targets run from the repo root; direct `uv run pytest` commands run from `tests/`:
|
||||
|
||||
```bash
|
||||
make py-test-setup # repo root — boot the stack once (~4 min), keep it with --reuse
|
||||
cd tests
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
|
||||
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/role/03_fga.py::test_read_scoped_to_granted_role
|
||||
make py-test-teardown # repo root — destroy when done; never mix --reuse and --teardown
|
||||
make py-fmt && make py-lint # repo root — mandatory before finishing
|
||||
```
|
||||
|
||||
## Fixtures you'll need
|
||||
|
||||
From `tests/fixtures/auth.py` (registered via `pytest_plugins` in `tests/conftest.py`):
|
||||
|
||||
- `signoz: types.SigNoz` — the running stack; build URLs with `signoz.self.host_configs["8080"].get("/api/v1/...")`.
|
||||
- `create_user_admin` — package-scoped; registers the first admin (`USER_ADMIN_EMAIL` / `USER_ADMIN_PASSWORD`). Take it as an unused argument in every test that needs the org to exist.
|
||||
- `get_token(email, password) -> str` — bearer token for a user.
|
||||
- `add_license(signoz, make_http_mocks, get_token)` — custom roles are enterprise-only; call it in a `test_apply_license` at the top of any suite that creates custom roles.
|
||||
- `create_active_user(signoz, admin_token, email=..., role="VIEWER", password=..., name=...) -> user_id` — invite + activate.
|
||||
- `change_user_role(signoz, admin_token, user_id, old_role, new_role)` — e.g. `"signoz-viewer"` → your custom role name.
|
||||
|
||||
From `tests/fixtures/role.py`:
|
||||
|
||||
- `create_role(token, name, transaction_groups=None, description="") -> role_id` (fixture).
|
||||
- `find_role_id(token, name) -> role_id` (fixture).
|
||||
- `transaction_group(relation, type_name, kind_name, selectors)` — builds one permission entry, e.g. `transaction_group("read", "metaresource", "thing", ["*"])` or with a specific instance name/ID as selector.
|
||||
|
||||
From `tests/fixtures/serviceaccount.py`:
|
||||
|
||||
- `create_service_account_with_key(signoz, token, name, role="signoz-admin") -> (sa_id, api_key)`.
|
||||
- Service accounts authenticate with the header `SIGNOZ-API-KEY: <key>`; users with `Authorization: Bearer <token>`.
|
||||
|
||||
Hard rule: inline `requests.get/post/put/delete` calls directly in tests — do not wrap single API calls in helper functions or fixtures. Always pass `timeout=5`. Include the response text in assertion messages so failures are diagnosable.
|
||||
|
||||
## Shape 1: managed-role matrix (community-safe)
|
||||
|
||||
For each role, assert the expected status on each new route. Pattern from `serviceaccount/03_auth.py`:
|
||||
|
||||
```python
|
||||
def test_editor_forbidden_on_admin_endpoint(signoz, create_user_admin, get_token):
|
||||
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
|
||||
_, api_key = create_service_account_with_key(signoz, token, "sa-role-editor", role="signoz-editor")
|
||||
|
||||
resp = requests.get(
|
||||
signoz.self.host_configs["8080"].get("/api/v1/service_accounts"),
|
||||
headers={"SIGNOZ-API-KEY": api_key},
|
||||
timeout=5,
|
||||
)
|
||||
assert resp.status_code == HTTPStatus.FORBIDDEN, f"expected 403, got {resp.status_code}: {resp.text}"
|
||||
```
|
||||
|
||||
The user-token variant creates an editor/viewer via `create_active_user` and asserts 403 on the route, 200 for admin.
|
||||
|
||||
## Shape 2: per-object FGA scoping (enterprise, custom role)
|
||||
|
||||
Grant instance verbs on one object only; assert the other stays forbidden. Structure from `role/03_fga.py`:
|
||||
|
||||
```python
|
||||
def test_apply_license(signoz, create_user_admin, make_http_mocks, get_token):
|
||||
add_license(signoz, make_http_mocks, get_token)
|
||||
|
||||
|
||||
def test_setup_actor(signoz, create_user_admin, get_token, create_role):
|
||||
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
|
||||
create_role(
|
||||
admin_token,
|
||||
"thing-fga-actor",
|
||||
[
|
||||
transaction_group("read", "metaresource", "thing", [_TARGET_A_ID]),
|
||||
transaction_group("list", "metaresource", "thing", ["*"]),
|
||||
],
|
||||
)
|
||||
user_id = create_active_user(signoz, admin_token, email=_ACTOR_EMAIL, role="VIEWER", password=_ACTOR_PASSWORD, name="thing fga actor")
|
||||
change_user_role(signoz, admin_token, user_id, "signoz-viewer", "thing-fga-actor")
|
||||
|
||||
|
||||
def test_read_scoped_to_grant(signoz, create_user_admin, get_token):
|
||||
token = get_token(_ACTOR_EMAIL, _ACTOR_PASSWORD)
|
||||
|
||||
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v1/things/{_TARGET_A_ID}"), headers={"Authorization": f"Bearer {token}"}, timeout=5)
|
||||
assert resp.status_code == HTTPStatus.OK, resp.text
|
||||
|
||||
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v1/things/{_TARGET_B_ID}"), headers={"Authorization": f"Bearer {token}"}, timeout=5)
|
||||
assert resp.status_code == HTTPStatus.FORBIDDEN, f"expected 403, got {resp.status_code}: {resp.text}"
|
||||
```
|
||||
|
||||
Key semantics to test (they mirror the selector rules on the route side):
|
||||
|
||||
- `read`/`update`/`delete` granted on a specific selector authorize only that instance; `*` authorizes all.
|
||||
- `create` and `list` are collection-scoped — only a `*` grant makes them pass, and `list` on `*` returns every instance including ones the caller can't `read` individually.
|
||||
- Revoking the grant (update the role's transaction groups, or `change_user_role` back) flips the previously-allowed call to 403.
|
||||
|
||||
Module-level constants for names/emails (see `role/03_fga.py` — `_ACTOR_ROLE_NAME`, `_TARGET_A`) keep the setup test and the assertion tests in sync.
|
||||
|
||||
## What "done" looks like
|
||||
|
||||
- Every new route appears in at least one allow case and one deny case.
|
||||
- Suites that create custom roles start with `test_apply_license`.
|
||||
- `make py-fmt && make py-lint` pass.
|
||||
- The suite passes against a `--reuse` stack: `uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/`.
|
||||
155
.claude/skills/authz/references/route-wiring.md
Normal file
155
.claude/skills/authz/references/route-wiring.md
Normal file
@@ -0,0 +1,155 @@
|
||||
# Route Wiring Reference
|
||||
|
||||
All examples below are real code from `pkg/apiserver/signozapiserver/serviceaccount.go`. Match this style exactly.
|
||||
|
||||
## Basic route: create (wildcard selector, response-phase ID)
|
||||
|
||||
```go
|
||||
if err := router.Handle("/api/v1/service_accounts", handler.New(
|
||||
provider.authzMiddleware.CheckResources(provider.serviceAccountHandler.Create, authtypes.SigNozAdminRoleName),
|
||||
handler.OpenAPIDef{
|
||||
ID: "CreateServiceAccount",
|
||||
Tags: []string{"serviceaccount"},
|
||||
Summary: "Create service account",
|
||||
Description: "This endpoint creates a service account",
|
||||
Request: new(serviceaccounttypes.PostableServiceAccount),
|
||||
Response: new(types.Identifiable),
|
||||
ResponseContentType: "application/json",
|
||||
SuccessStatusCode: http.StatusCreated,
|
||||
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusConflict},
|
||||
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceServiceAccount.Scope(coretypes.VerbCreate)}),
|
||||
},
|
||||
handler.WithResourceDefs(handler.BasicResourceDef{
|
||||
Resource: coretypes.ResourceServiceAccount,
|
||||
Verb: coretypes.VerbCreate,
|
||||
Category: coretypes.ActionCategoryAccessControl,
|
||||
ID: coretypes.ResponseJSONPath("data.id"),
|
||||
Selector: coretypes.WildcardSelector,
|
||||
}),
|
||||
)).Methods(http.MethodPost).GetError(); err != nil {
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
Create checks the wildcard (can the caller create *any* instance?) but still records the new instance's ID for audit via `ResponseJSONPath` — the ID only exists after the handler runs.
|
||||
|
||||
Instance routes (read/update/delete) differ only in verb, `ID: coretypes.PathParam("id")`, and `Selector: coretypes.IDSelector`. List routes use `WildcardSelector` and omit `ID` entirely.
|
||||
|
||||
Pick the `Category` matching the resource's domain — grep `ActionCategory` in `pkg/types/coretypes/` for the registered values (`ActionCategoryAccessControl` is for access-control resources; observability resources use their own categories).
|
||||
|
||||
## ID extractors (`pkg/types/coretypes/extractor.go`)
|
||||
|
||||
| Extractor | Phase | Use for |
|
||||
|---|---|---|
|
||||
| `PathParam("id")` | request | ID in the URL path |
|
||||
| `BodyJSONPath("data.id")` | request | ID in the request body (gjson path) |
|
||||
| `BodyJSONArray("ids")` | request | multiple IDs in a body array |
|
||||
| `ResponseJSONPath("data.id")` | response | ID known only after the handler (create) |
|
||||
| `NewResourceIDExtractor(phase, fn)` | either | ID needs a lookup (see below) |
|
||||
| `OneID(extractor)` | — | lifts a single-ID extractor where a multi-ID one is required (`SourceIDs`/`TargetIDs`/`ChildIDs`) |
|
||||
|
||||
Custom extractor example — the route path carries a service-account-role ID but the check needs the underlying role ID, so the extractor resolves it through the module:
|
||||
|
||||
```go
|
||||
func (provider *provider) roleIDExtractor() coretypes.ResourceIDExtractor {
|
||||
return coretypes.NewResourceIDExtractor(coretypes.PhaseRequest, func(ec coretypes.ExtractorContext) (string, error) {
|
||||
if ec.Request == nil {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
claims, err := authtypes.ClaimsFromContext(ec.Request.Context())
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
serviceAccountRoleID, err := valuer.NewUUID(mux.Vars(ec.Request)["id"])
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
serviceAccountRole, err := provider.serviceAccountGetter.GetServiceAccountRole(ec.Request.Context(), valuer.MustNewUUID(claims.OrgID), serviceAccountRoleID)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
return serviceAccountRole.RoleID.String(), nil
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Custom SelectorFunc
|
||||
|
||||
Use one when the extracted ID is not what FGA objects use. Roles are the canonical case: routes receive a role UUID, but FGA role objects are keyed by role *name*. Always include the wildcard alongside the specific selector — authorization on `*` implies authorization on every instance:
|
||||
|
||||
```go
|
||||
func (provider *provider) roleSelector(ctx context.Context, resource coretypes.Resource, id string, orgID valuer.UUID) ([]coretypes.Selector, error) {
|
||||
roleID, err := valuer.NewUUID(id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
role, err := provider.authzService.Get(ctx, orgID, roleID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return []coretypes.Selector{
|
||||
resource.Type().MustSelector(role.Name),
|
||||
resource.Type().MustSelector(coretypes.WildCardSelectorString),
|
||||
}, nil
|
||||
}
|
||||
```
|
||||
|
||||
## Linking routes: siblings (peer-to-peer)
|
||||
|
||||
Attaching a role to a service account requires `attach` on **both** resources — the same verb checked on each side. Advertise both scopes:
|
||||
|
||||
```go
|
||||
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceServiceAccount.Scope(coretypes.VerbAttach), coretypes.ResourceRole.Scope(coretypes.VerbAttach)}),
|
||||
// ...
|
||||
handler.WithResourceDefs(handler.AttachDetachSiblingResourceDef{
|
||||
Verb: coretypes.VerbAttach,
|
||||
Category: coretypes.ActionCategoryAccessControl,
|
||||
SourceResource: coretypes.ResourceServiceAccount,
|
||||
SourceIDs: coretypes.OneID(coretypes.PathParam("id")),
|
||||
SourceSelector: coretypes.IDSelector,
|
||||
TargetResource: coretypes.ResourceRole,
|
||||
TargetIDs: coretypes.OneID(coretypes.BodyJSONPath("id")),
|
||||
TargetSelector: provider.roleSelector,
|
||||
}),
|
||||
```
|
||||
|
||||
The detach route is identical with `VerbDetach` and the target ID coming from the path.
|
||||
|
||||
## Linking routes: parent-child
|
||||
|
||||
Creating an API key *under* a service account checks two different things: `create` on the child resource, and `attach` on the parent. Declare both defs in the same `WithResourceDefs`:
|
||||
|
||||
```go
|
||||
handler.WithResourceDefs(
|
||||
handler.BasicResourceDef{
|
||||
Resource: coretypes.ResourceMetaResourceFactorAPIKey,
|
||||
Verb: coretypes.VerbCreate,
|
||||
Category: coretypes.ActionCategoryAccessControl,
|
||||
ID: coretypes.ResponseJSONPath("data.id"),
|
||||
Selector: coretypes.WildcardSelector,
|
||||
},
|
||||
handler.AttachDetachParentChildResourceDef{
|
||||
Verb: coretypes.VerbAttach,
|
||||
Category: coretypes.ActionCategoryAccessControl,
|
||||
ParentResource: coretypes.ResourceServiceAccount,
|
||||
ParentID: coretypes.PathParam("id"),
|
||||
ParentSelector: coretypes.IDSelector,
|
||||
ChildResource: coretypes.ResourceMetaResourceFactorAPIKey,
|
||||
ChildIDs: coretypes.OneID(coretypes.ResponseJSONPath("data.id")),
|
||||
},
|
||||
),
|
||||
```
|
||||
|
||||
Within the parent-child def, the child is only recorded for audit — the authz check on the child comes from its own `BasicResourceDef`. Deletion mirrors this with `VerbDelete` on the child and `VerbDetach` on the parent (see the `/api/v1/service_accounts/{id}/keys/{fid}` DELETE route).
|
||||
|
||||
## Legacy gates — do not use for new routes
|
||||
|
||||
`ViewAccess`/`EditAccess`/`AdminAccess` only check "does the caller hold one of these roles". They give up per-resource granularity, produce no audit resources, and don't advertise scopes. They exist for old routes; new routes use `CheckResources` + `ResourceDef`. `OpenAccess` performs no authorization at all (authentication still applies — used for `/me`-style routes); `CheckWithoutClaims` is for anonymous routes such as public dashboards.
|
||||
|
||||
Full def types live in `pkg/http/handler/resourcedef.go`; middleware behavior in `pkg/http/middleware/authz.go` and `pkg/http/middleware/resource.go`.
|
||||
@@ -2,9 +2,9 @@ package main
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/metercollector"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/zeustypes"
|
||||
)
|
||||
@@ -25,8 +25,8 @@ var meterConfigs = []metercollector.Config{
|
||||
Name: zeustypes.MeterLogSize,
|
||||
Unit: zeustypes.MeterUnitBytes,
|
||||
Aggregation: zeustypes.MeterAggregationSum,
|
||||
DBName: logstelemetryschema.DBName,
|
||||
TableName: logstelemetryschema.LogsV2LocalTableName,
|
||||
DBName: telemetrylogs.DBName,
|
||||
TableName: telemetrylogs.LogsV2LocalTableName,
|
||||
DefaultRetentionDays: retentiontypes.DefaultLogsRetentionDays,
|
||||
},
|
||||
},
|
||||
@@ -36,8 +36,8 @@ var meterConfigs = []metercollector.Config{
|
||||
Name: zeustypes.MeterSpanSize,
|
||||
Unit: zeustypes.MeterUnitBytes,
|
||||
Aggregation: zeustypes.MeterAggregationSum,
|
||||
DBName: tracestelemetryschema.DBName,
|
||||
TableName: tracestelemetryschema.SpanIndexV3LocalTableName,
|
||||
DBName: telemetrytraces.DBName,
|
||||
TableName: telemetrytraces.SpanIndexV3LocalTableName,
|
||||
DefaultRetentionDays: retentiontypes.DefaultTracesRetentionDays,
|
||||
},
|
||||
},
|
||||
@@ -47,8 +47,8 @@ var meterConfigs = []metercollector.Config{
|
||||
Name: zeustypes.MeterDatapointCount,
|
||||
Unit: zeustypes.MeterUnitCount,
|
||||
Aggregation: zeustypes.MeterAggregationSum,
|
||||
DBName: metricstelemetryschema.DBName,
|
||||
TableName: metricstelemetryschema.SamplesV4LocalTableName,
|
||||
DBName: telemetrymetrics.DBName,
|
||||
TableName: telemetrymetrics.SamplesV4LocalTableName,
|
||||
DefaultRetentionDays: retentiontypes.DefaultMetricsRetentionDays,
|
||||
},
|
||||
},
|
||||
|
||||
@@ -16,7 +16,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/metercollector"
|
||||
"github.com/SigNoz/signoz/pkg/modules/retention"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymeter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/licensetypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/retentiontypes"
|
||||
@@ -153,7 +153,7 @@ func (provider *Provider) Collect(
|
||||
func buildOriginQuery(meterName string) (string, []any) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("toInt64(ifNull(min(unix_milli), 0))")
|
||||
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
|
||||
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
|
||||
sb.Where(sb.Equal("metric_name", meterName))
|
||||
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
}
|
||||
@@ -171,7 +171,7 @@ func buildQuery(meterName string, segment *retentiontypes.RetentionPolicySegment
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select(selects...)
|
||||
sb.From(metertelemetryschema.DBName + "." + metertelemetryschema.SamplesTableName)
|
||||
sb.From(telemetrymeter.DBName + "." + telemetrymeter.SamplesTableName)
|
||||
sb.Where(
|
||||
sb.Equal("metric_name", meterName),
|
||||
sb.GTE("unix_milli", segment.StartMs),
|
||||
|
||||
@@ -8,16 +8,16 @@ import (
|
||||
sqlbuilder "github.com/huandu/go-sqlbuilder"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/metricreductionruletypes"
|
||||
)
|
||||
|
||||
var (
|
||||
reductionRulesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.ReductionRulesTableName
|
||||
metadataTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.AttributesMetadataTableName
|
||||
bufferSeriesTable = metricstelemetryschema.DBName + "." + metricstelemetryschema.TimeseriesV4BufferTableName
|
||||
reductionRulesTable = telemetrymetrics.DBName + "." + telemetrymetrics.ReductionRulesTableName
|
||||
metadataTable = telemetrymetrics.DBName + "." + telemetrymetrics.AttributesMetadataTableName
|
||||
bufferSeriesTable = telemetrymetrics.DBName + "." + telemetrymetrics.TimeseriesV4BufferTableName
|
||||
)
|
||||
|
||||
const timeSeriesBucketMilli = int64(time.Hour / time.Millisecond)
|
||||
@@ -192,7 +192,7 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("metric_name", "uniq(fingerprint)")
|
||||
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
|
||||
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
|
||||
conds := []string{
|
||||
sb.In("metric_name", names...),
|
||||
sb.GE("unix_milli", startMs),
|
||||
@@ -229,8 +229,8 @@ func (c *clickhouse) ingestedSeriesCount(ctx context.Context, metricNames []stri
|
||||
// reduced sample tables.
|
||||
func (c *clickhouse) reducedSeriesCount(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[string]uint64, error) {
|
||||
out := make(map[string]uint64, len(metricNames))
|
||||
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
|
||||
counts, err := c.reducedSeriesCountForTable(ctx, metricstelemetryschema.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
|
||||
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
|
||||
counts, err := c.reducedSeriesCountForTable(ctx, telemetrymetrics.DBName+"."+table, metricNames, effectiveFrom, startMs, endMs)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -300,9 +300,9 @@ func (c *clickhouse) RankByVolume(ctx context.Context, metricNames []string, eff
|
||||
direction = "DESC"
|
||||
}
|
||||
|
||||
ingestedTable := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName
|
||||
reducedLast := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedLastTableName
|
||||
reducedSum := metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4ReducedSumTableName
|
||||
ingestedTable := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName
|
||||
reducedLast := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedLastTableName
|
||||
reducedSum := telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4ReducedSumTableName
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("base.metric_name AS metric_name", "ifNull(i.cnt, 0) AS ingested", "ifNull(d.cnt, 0) AS reduced")
|
||||
@@ -352,16 +352,16 @@ func (c *clickhouse) SampleVolumeByMetric(ctx context.Context, metricNames []str
|
||||
}
|
||||
ctx = c.withThreads(ctx)
|
||||
|
||||
ingested, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4BufferTableName, "count()", metricNames, effectiveFrom, startMs, endMs)
|
||||
ingested, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4BufferTableName, "count()", metricNames, effectiveFrom, startMs, endMs)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
last, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedLastTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
|
||||
last, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedLastTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sum, err := c.countSamplesByMetric(ctx, metricstelemetryschema.DBName+"."+metricstelemetryschema.SamplesV4ReducedSumTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
|
||||
sum, err := c.countSamplesByMetric(ctx, telemetrymetrics.DBName+"."+telemetrymetrics.SamplesV4ReducedSumTableName, "uniq(reduced_fingerprint, unix_milli)", metricNames, effectiveFrom, startMs, endMs)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -419,7 +419,7 @@ func (c *clickhouse) TotalVolume(ctx context.Context, startMs, endMs int64) (uin
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("uniq(fingerprint)", "count()")
|
||||
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
|
||||
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
|
||||
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
@@ -470,7 +470,7 @@ func (c *clickhouse) SampleTimeseries(ctx context.Context, ruledMetrics []string
|
||||
func (c *clickhouse) totalSamplesByBucket(ctx context.Context, startMs, endMs int64) (map[int64]uint64, error) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select(sampleBucketExpr, "count()")
|
||||
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
|
||||
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
|
||||
sb.Where(sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs))
|
||||
sb.GroupBy("bucket")
|
||||
|
||||
@@ -485,7 +485,7 @@ func (c *clickhouse) ruledIngestedSamplesByBucket(ctx context.Context, metricNam
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select(sampleBucketExpr, "count()")
|
||||
sb.From(metricstelemetryschema.DBName + "." + metricstelemetryschema.SamplesV4BufferTableName)
|
||||
sb.From(telemetrymetrics.DBName + "." + telemetrymetrics.SamplesV4BufferTableName)
|
||||
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
|
||||
if len(effectiveFrom) > 0 {
|
||||
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
|
||||
@@ -499,7 +499,7 @@ func (c *clickhouse) ruledIngestedSamplesByBucket(ctx context.Context, metricNam
|
||||
// reduced 60s rows are versioned by computed_at, so count distinct buckets.
|
||||
func (c *clickhouse) ruledRetainedSamplesByBucket(ctx context.Context, metricNames []string, effectiveFrom map[string]int64, startMs, endMs int64) (map[int64]uint64, error) {
|
||||
out := make(map[int64]uint64)
|
||||
for _, table := range []string{metricstelemetryschema.SamplesV4ReducedLastTableName, metricstelemetryschema.SamplesV4ReducedSumTableName} {
|
||||
for _, table := range []string{telemetrymetrics.SamplesV4ReducedLastTableName, telemetrymetrics.SamplesV4ReducedSumTableName} {
|
||||
names := make([]any, len(metricNames))
|
||||
for i, name := range metricNames {
|
||||
names[i] = name
|
||||
@@ -507,7 +507,7 @@ func (c *clickhouse) ruledRetainedSamplesByBucket(ctx context.Context, metricNam
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select(sampleBucketExpr, "uniq(reduced_fingerprint, unix_milli)")
|
||||
sb.From(metricstelemetryschema.DBName + "." + table)
|
||||
sb.From(telemetrymetrics.DBName + "." + table)
|
||||
conds := []string{sb.In("metric_name", names...), sb.GE("unix_milli", startMs), sb.LT("unix_milli", endMs)}
|
||||
if len(effectiveFrom) > 0 {
|
||||
conds = append(conds, strictEffectiveFrom(sb, metricNames, effectiveFrom))
|
||||
|
||||
@@ -2,29 +2,6 @@ import { RenderErrorResponseDTO } from 'api/generated/services/sigNoz.schemas';
|
||||
import { AxiosError } from 'axios';
|
||||
import APIError from 'types/api/error';
|
||||
|
||||
// The wire shape these handlers can actually rely on. The generated
|
||||
// RenderErrorResponseDTO marks code/message/url/errors as required, but the
|
||||
// server omits any of them even on valid errors (e.g. a 400 with just a
|
||||
// message), so a present `error` object is all the guard can guarantee.
|
||||
type ErrorEnvelope = {
|
||||
error: {
|
||||
code?: string;
|
||||
message?: string;
|
||||
url?: string;
|
||||
errors?: { message?: string }[];
|
||||
};
|
||||
};
|
||||
|
||||
function isErrorEnvelope(data: unknown): data is ErrorEnvelope {
|
||||
return (
|
||||
typeof data === 'object' &&
|
||||
data !== null &&
|
||||
'error' in data &&
|
||||
typeof (data as ErrorEnvelope).error === 'object' &&
|
||||
(data as ErrorEnvelope).error !== null
|
||||
);
|
||||
}
|
||||
|
||||
// @deprecated Use convertToApiError instead
|
||||
export function ErrorResponseHandlerForGeneratedAPIs(
|
||||
error: AxiosError<RenderErrorResponseDTO>,
|
||||
@@ -33,29 +10,15 @@ export function ErrorResponseHandlerForGeneratedAPIs(
|
||||
// The request was made and the server responded with a status code
|
||||
// that falls out of the range of 2xx
|
||||
if (response) {
|
||||
// The body isn't guaranteed to be an error envelope — e.g. a gateway 5xx
|
||||
// with an HTML/empty body during a deploy. Verify the shape before reading
|
||||
// it; otherwise synthesize a consistent error from the status.
|
||||
const data: unknown = response.data;
|
||||
if (isErrorEnvelope(data)) {
|
||||
const { code, message, url, errors } = data.error;
|
||||
throw new APIError({
|
||||
httpStatusCode: response.status || 500,
|
||||
error: {
|
||||
code: code ?? '',
|
||||
message: message ?? '',
|
||||
url: url ?? '',
|
||||
errors: (errors ?? []).map((e) => ({ message: e.message ?? '' })),
|
||||
},
|
||||
});
|
||||
}
|
||||
throw new APIError({
|
||||
httpStatusCode: response.status || 500,
|
||||
error: {
|
||||
code: 'UPSTREAM_UNAVAILABLE',
|
||||
message: error.message || 'Something went wrong',
|
||||
url: '',
|
||||
errors: [],
|
||||
code: response.data.error.code,
|
||||
message: response.data.error.message,
|
||||
url: response.data.error.url ?? '',
|
||||
errors: (response.data.error.errors ?? []).map((e) => ({
|
||||
message: e.message ?? '',
|
||||
})),
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -99,7 +62,9 @@ export function convertToApiError(
|
||||
return new APIError({
|
||||
httpStatusCode: response?.status || error.status || 500,
|
||||
error: {
|
||||
code: errorData?.code || 'UPSTREAM_UNAVAILABLE',
|
||||
code:
|
||||
errorData?.code ||
|
||||
String(response?.status || error.code || 'unknown_error'),
|
||||
message:
|
||||
errorData?.message ||
|
||||
response?.statusText ||
|
||||
|
||||
@@ -2,38 +2,19 @@ import { AxiosError } from 'axios';
|
||||
import { ErrorV2Resp } from 'types/api';
|
||||
import APIError from 'types/api/error';
|
||||
|
||||
function isErrorV2Resp(data: unknown): data is ErrorV2Resp {
|
||||
return (
|
||||
typeof data === 'object' &&
|
||||
data !== null &&
|
||||
'error' in data &&
|
||||
typeof (data as ErrorV2Resp).error?.code === 'string'
|
||||
);
|
||||
}
|
||||
|
||||
// reference - https://axios-http.com/docs/handling_errors
|
||||
export function ErrorResponseHandlerV2(error: AxiosError<ErrorV2Resp>): never {
|
||||
const { response, request } = error;
|
||||
// The request was made and the server responded with a status code
|
||||
// that falls out of the range of 2xx
|
||||
if (response) {
|
||||
// response.data isn't guaranteed to be a V2 envelope (e.g. a gateway 5xx
|
||||
// with an HTML/empty body during a deploy), so verify the shape first.
|
||||
const data: unknown = response.data;
|
||||
if (isErrorV2Resp(data)) {
|
||||
const { code, message, url, errors } = data.error;
|
||||
throw new APIError({
|
||||
httpStatusCode: response.status || 500,
|
||||
error: { code, message, url, errors },
|
||||
});
|
||||
}
|
||||
throw new APIError({
|
||||
httpStatusCode: response.status || 500,
|
||||
error: {
|
||||
code: 'UPSTREAM_UNAVAILABLE',
|
||||
message: error.message || 'Something went wrong',
|
||||
url: '',
|
||||
errors: [],
|
||||
code: response.data.error.code,
|
||||
message: response.data.error.message,
|
||||
url: response.data.error.url,
|
||||
errors: response.data.error.errors,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,126 +0,0 @@
|
||||
import { ErrorResponseHandlerV2 } from 'api/ErrorResponseHandlerV2';
|
||||
import { AxiosError } from 'axios';
|
||||
import { ErrorV2Resp } from 'types/api';
|
||||
import APIError from 'types/api/error';
|
||||
|
||||
function asAxiosError(partial: Partial<AxiosError>): AxiosError<ErrorV2Resp> {
|
||||
return partial as AxiosError<ErrorV2Resp>;
|
||||
}
|
||||
|
||||
// ErrorResponseHandlerV2 always throws — capture the APIError so assertions stay
|
||||
// unconditional.
|
||||
function runHandler(error: AxiosError<ErrorV2Resp>): APIError {
|
||||
try {
|
||||
ErrorResponseHandlerV2(error);
|
||||
} catch (thrown) {
|
||||
return thrown as APIError;
|
||||
}
|
||||
throw new Error('expected ErrorResponseHandlerV2 to throw');
|
||||
}
|
||||
|
||||
type ExpectedError = {
|
||||
httpStatusCode: number;
|
||||
code: string;
|
||||
message: string;
|
||||
errors: { message: string }[];
|
||||
};
|
||||
|
||||
// One row per response shape the handler must normalize. New shapes (with
|
||||
// different bodies) can be added here without a new test block.
|
||||
const cases: {
|
||||
name: string;
|
||||
error: AxiosError<ErrorV2Resp>;
|
||||
expected: ExpectedError;
|
||||
}[] = [
|
||||
{
|
||||
name: 'well-formed V2 error envelope',
|
||||
error: asAxiosError({
|
||||
message: 'Request failed with status code 400',
|
||||
response: {
|
||||
status: 400,
|
||||
data: {
|
||||
error: {
|
||||
code: 'bad_request',
|
||||
message: 'Invalid dashboard payload',
|
||||
url: 'https://signoz.io/docs',
|
||||
errors: [{ message: 'name is required' }, { message: 'name too long' }],
|
||||
},
|
||||
},
|
||||
} as AxiosError['response'],
|
||||
}),
|
||||
expected: {
|
||||
httpStatusCode: 400,
|
||||
code: 'bad_request',
|
||||
message: 'Invalid dashboard payload',
|
||||
errors: [{ message: 'name is required' }, { message: 'name too long' }],
|
||||
},
|
||||
},
|
||||
{
|
||||
// Regression: during a deployment the gateway returns a 5xx with a
|
||||
// non-envelope body. Reading response.data.error.code used to throw a
|
||||
// TypeError from inside the handler itself. See engineering-pod#5760.
|
||||
name: '5xx with a non-envelope HTML body',
|
||||
error: asAxiosError({
|
||||
message: 'Request failed with status code 503',
|
||||
response: {
|
||||
status: 503,
|
||||
data: '<html><body>503 Service Temporarily Unavailable</body></html>',
|
||||
} as AxiosError['response'],
|
||||
}),
|
||||
expected: {
|
||||
httpStatusCode: 503,
|
||||
code: 'UPSTREAM_UNAVAILABLE',
|
||||
message: 'Request failed with status code 503',
|
||||
errors: [],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: '5xx with an empty body',
|
||||
error: asAxiosError({
|
||||
message: 'Request failed with status code 502',
|
||||
response: {
|
||||
status: 502,
|
||||
data: undefined,
|
||||
} as AxiosError['response'],
|
||||
}),
|
||||
expected: {
|
||||
httpStatusCode: 502,
|
||||
code: 'UPSTREAM_UNAVAILABLE',
|
||||
message: 'Request failed with status code 502',
|
||||
errors: [],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'no response received (network error)',
|
||||
error: asAxiosError({
|
||||
message: 'Network Error',
|
||||
code: 'ERR_NETWORK',
|
||||
name: 'AxiosError',
|
||||
request: {},
|
||||
}),
|
||||
expected: {
|
||||
httpStatusCode: 500,
|
||||
code: 'ERR_NETWORK',
|
||||
message: 'Network Error',
|
||||
errors: [],
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
describe('ErrorResponseHandlerV2', () => {
|
||||
it.each(cases)(
|
||||
'normalizes $name into a consistent APIError',
|
||||
({ error, expected }) => {
|
||||
const apiError = runHandler(error);
|
||||
|
||||
expect(apiError).toBeInstanceOf(APIError);
|
||||
expect(apiError.getHttpStatusCode()).toBe(expected.httpStatusCode);
|
||||
expect(apiError.getErrorCode()).toBe(expected.code);
|
||||
expect(apiError.getErrorMessage()).toBe(expected.message);
|
||||
// The sub-error messages feed several parts of the UI, so assert them.
|
||||
expect(apiError.getErrorDetails().error.errors).toStrictEqual(
|
||||
expected.errors,
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -92,76 +92,52 @@
|
||||
color: var(--l3-foreground);
|
||||
}
|
||||
|
||||
.only-btn,
|
||||
.only-btn {
|
||||
cursor: not-allowed;
|
||||
color: var(--l3-foreground);
|
||||
}
|
||||
|
||||
.toggle-btn {
|
||||
cursor: not-allowed;
|
||||
color: var(--l3-foreground);
|
||||
}
|
||||
}
|
||||
|
||||
// Stack "Only"/"All" and "Toggle" in a single cell so revealing
|
||||
// one swaps in place instead of shifting the row layout.
|
||||
.value-actions {
|
||||
display: grid;
|
||||
align-items: center;
|
||||
justify-items: end;
|
||||
|
||||
> * {
|
||||
grid-area: 1 / 1;
|
||||
}
|
||||
.only-btn {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.only-btn,
|
||||
.toggle-btn {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.toggle-btn:hover {
|
||||
background-color: unset;
|
||||
}
|
||||
|
||||
.only-btn:hover {
|
||||
background-color: unset;
|
||||
}
|
||||
}
|
||||
|
||||
.checkbox-value-section:hover {
|
||||
.toggle-btn {
|
||||
display: none;
|
||||
}
|
||||
.only-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 21px;
|
||||
opacity: 0;
|
||||
transform: translateX(4px);
|
||||
// `display` is animated as a discrete property so the button
|
||||
// stays mounted through its fade-out on exit.
|
||||
transition:
|
||||
opacity 0.16s ease,
|
||||
transform 0.16s ease,
|
||||
display 0.16s allow-discrete;
|
||||
|
||||
&:hover {
|
||||
background-color: unset;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Hovering the label/value area reveals the "Only"/"All" action.
|
||||
.checkbox-value-section:hover .only-btn {
|
||||
display: flex;
|
||||
opacity: 1;
|
||||
transform: translateX(0);
|
||||
|
||||
// Entry animation start state (fires on the display switch).
|
||||
@starting-style {
|
||||
opacity: 0;
|
||||
transform: translateX(4px);
|
||||
}
|
||||
}
|
||||
|
||||
// Hovering the checkbox reveals the "Toggle" action.
|
||||
.check-box:hover ~ .checkbox-value-section .toggle-btn {
|
||||
display: flex;
|
||||
opacity: 1;
|
||||
transform: translateX(0);
|
||||
|
||||
@starting-style {
|
||||
opacity: 0;
|
||||
transform: translateX(4px);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.only-btn,
|
||||
.value:hover {
|
||||
.toggle-btn {
|
||||
transition: none;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 21px;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -53,14 +53,12 @@ function CheckboxValueRow({
|
||||
</Typography.Text>
|
||||
</TooltipSimple>
|
||||
)}
|
||||
<div className="value-actions">
|
||||
<Button type="text" className="only-btn">
|
||||
{onlyButtonLabel}
|
||||
</Button>
|
||||
<Button type="text" className="toggle-btn">
|
||||
Toggle
|
||||
</Button>
|
||||
</div>
|
||||
<Button type="text" className="only-btn">
|
||||
{onlyButtonLabel}
|
||||
</Button>
|
||||
<Button type="text" className="toggle-btn">
|
||||
Toggle
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
|
||||
@@ -6,7 +6,7 @@ import { Skeleton } from 'antd';
|
||||
import { Typography } from '@signozhq/ui/typography';
|
||||
import {
|
||||
useGetMetricAlerts,
|
||||
useGetMetricDashboardsV2,
|
||||
useGetMetricDashboards,
|
||||
} from 'api/generated/services/metrics';
|
||||
import { QueryParams } from 'constants/query';
|
||||
import ROUTES from 'constants/routes';
|
||||
@@ -38,7 +38,7 @@ function DashboardsAndAlertsPopover({
|
||||
data: dashboardsData,
|
||||
isLoading: isLoadingDashboards,
|
||||
isError: isErrorDashboards,
|
||||
} = useGetMetricDashboardsV2(
|
||||
} = useGetMetricDashboards(
|
||||
{
|
||||
metricName,
|
||||
},
|
||||
@@ -55,8 +55,7 @@ function DashboardsAndAlertsPopover({
|
||||
|
||||
const dashboards = useMemo(() => {
|
||||
const currentDashboards = dashboardsData?.data.dashboards ?? [];
|
||||
// The API returns one entry per referencing panel, so a dashboard repeats
|
||||
// once per panel that uses the metric.
|
||||
// Remove duplicate dashboards
|
||||
return currentDashboards.filter(
|
||||
(dashboard, index, self) =>
|
||||
index === self.findIndex((t) => t.dashboardId === dashboard.dashboardId),
|
||||
|
||||
@@ -23,7 +23,7 @@ const useGetMetricAlertsMock = jest.spyOn(
|
||||
);
|
||||
const useGetMetricDashboardsMock = jest.spyOn(
|
||||
metricsExplorerHooks,
|
||||
'useGetMetricDashboardsV2',
|
||||
'useGetMetricDashboards',
|
||||
);
|
||||
|
||||
describe('DashboardsAndAlertsPopover', () => {
|
||||
@@ -153,15 +153,11 @@ describe('DashboardsAndAlertsPopover', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('collapses multiple panels of the same dashboard into one entry', async () => {
|
||||
it('renders unique dashboards even when there are duplicates', async () => {
|
||||
useGetMetricDashboardsMock.mockReturnValue(
|
||||
getMockDashboardsData({
|
||||
data: {
|
||||
dashboards: [
|
||||
MOCK_DASHBOARD_1,
|
||||
MOCK_DASHBOARD_2,
|
||||
{ ...MOCK_DASHBOARD_1, panelId: '3', panelName: 'Panel 3' },
|
||||
],
|
||||
dashboards: [MOCK_DASHBOARD_1, MOCK_DASHBOARD_2, MOCK_DASHBOARD_1],
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -2,7 +2,7 @@ import * as metricsExplorerHooks from 'api/generated/services/metrics';
|
||||
import {
|
||||
GetMetricAlerts200,
|
||||
GetMetricAttributes200,
|
||||
GetMetricDashboardsV2200,
|
||||
GetMetricDashboards200,
|
||||
GetMetricHighlights200,
|
||||
GetMetricMetadata200,
|
||||
MetrictypesTemporalityDTO,
|
||||
@@ -40,14 +40,14 @@ export function getMockMetricHighlightsData(
|
||||
export const MOCK_DASHBOARD_1 = {
|
||||
dashboardName: 'Dashboard 1',
|
||||
dashboardId: '1',
|
||||
panelId: '1',
|
||||
panelName: 'Panel 1',
|
||||
widgetId: '1',
|
||||
widgetName: 'Widget 1',
|
||||
};
|
||||
export const MOCK_DASHBOARD_2 = {
|
||||
dashboardName: 'Dashboard 2',
|
||||
dashboardId: '2',
|
||||
panelId: '2',
|
||||
panelName: 'Panel 2',
|
||||
widgetId: '2',
|
||||
widgetName: 'Widget 2',
|
||||
};
|
||||
export const MOCK_ALERT_1 = {
|
||||
alertName: 'Alert 1',
|
||||
@@ -59,7 +59,7 @@ export const MOCK_ALERT_2 = {
|
||||
};
|
||||
|
||||
export function getMockDashboardsData(
|
||||
overrides?: Partial<GetMetricDashboardsV2200>,
|
||||
overrides?: Partial<GetMetricDashboards200>,
|
||||
{
|
||||
isLoading = false,
|
||||
isError = false,
|
||||
@@ -67,7 +67,7 @@ export function getMockDashboardsData(
|
||||
isLoading?: boolean;
|
||||
isError?: boolean;
|
||||
} = {},
|
||||
): ReturnType<typeof metricsExplorerHooks.useGetMetricDashboardsV2> {
|
||||
): ReturnType<typeof metricsExplorerHooks.useGetMetricDashboards> {
|
||||
return {
|
||||
data: {
|
||||
data: {
|
||||
@@ -79,7 +79,7 @@ export function getMockDashboardsData(
|
||||
|
||||
isLoading,
|
||||
isError,
|
||||
} as ReturnType<typeof metricsExplorerHooks.useGetMetricDashboardsV2>;
|
||||
} as ReturnType<typeof metricsExplorerHooks.useGetMetricDashboards>;
|
||||
}
|
||||
|
||||
export function getMockAlertsData(
|
||||
|
||||
@@ -11,9 +11,3 @@
|
||||
border: 1px solid var(--l3-border) !important;
|
||||
box-shadow: none !important;
|
||||
}
|
||||
|
||||
/* Highlights the dashboard name inside the delete-confirm title. */
|
||||
.deleteName {
|
||||
color: var(--danger-background);
|
||||
font-weight: var(--font-weight-medium);
|
||||
}
|
||||
|
||||
@@ -27,13 +27,16 @@ import logEvent from 'api/common/logEvent';
|
||||
import { cloneDashboardV2 } from 'api/generated/services/dashboard';
|
||||
import type { DashboardtypesGettableDashboardV2DTO } from 'api/generated/services/sigNoz.schemas';
|
||||
import ROUTES from 'constants/routes';
|
||||
import { useDeleteDashboard } from 'hooks/dashboard/useDeleteDashboard';
|
||||
import { useSafeNavigate } from 'hooks/useSafeNavigate';
|
||||
import history from 'lib/history';
|
||||
import { DashboardDetailEvents } from 'pages/DashboardPageV2/constants/events';
|
||||
import { useAppContext } from 'providers/App/App';
|
||||
import { useErrorModal } from 'providers/ErrorModalProvider';
|
||||
import APIError from 'types/api/error';
|
||||
import { USER_ROLES } from 'types/roles';
|
||||
|
||||
import ConfirmDeleteDialog from '../../components/ConfirmDeleteDialog/ConfirmDeleteDialog';
|
||||
import DisabledControlTooltip from '../../components/DisabledControlTooltip/DisabledControlTooltip';
|
||||
import DisabledMenuItemLabel from '../../components/DisabledMenuItemLabel/DisabledMenuItemLabel';
|
||||
import DashboardSettings from '../../DashboardSettings';
|
||||
@@ -42,7 +45,6 @@ import SectionTitleModal from '../../PanelsAndSectionsLayout/Section/SectionTitl
|
||||
import JsonEditorDrawer from '../JsonEditorDrawer/JsonEditorDrawer';
|
||||
import SettingsDrawer from '../SettingsDrawer';
|
||||
import styles from './DashboardActions.module.scss';
|
||||
import { useDeleteDashboardAction } from './useDeleteDashboardAction';
|
||||
import { DASHBOARD_LOCKED_REASON } from '../../hooks/useDashboardEditGuard';
|
||||
import { useDashboardStore } from '../../store/useDashboardStore';
|
||||
|
||||
@@ -82,12 +84,8 @@ function DashboardActions({
|
||||
const [isCloning, setIsCloning] = useState<boolean>(false);
|
||||
const [isNewSectionOpen, setIsNewSectionOpen] = useState<boolean>(false);
|
||||
|
||||
const { contextHolder: deleteConfirmHolder, confirmDeleteDashboard } =
|
||||
useDeleteDashboardAction({
|
||||
dashboardId: dashboard.id,
|
||||
dashboardName: title,
|
||||
panelCount: Object.keys(dashboard.spec.panels).length,
|
||||
});
|
||||
const [isDeleteOpen, setIsDeleteOpen] = useState<boolean>(false);
|
||||
const deleteDashboardMutation = useDeleteDashboard(dashboard.id);
|
||||
|
||||
// Open the settings drawer when something in the tree requests it (e.g. the
|
||||
// variables bar's "Add variable" button).
|
||||
@@ -134,6 +132,19 @@ function DashboardActions({
|
||||
}
|
||||
}, [dashboard.id, title, safeNavigate, showErrorModal]);
|
||||
|
||||
const handleConfirmDelete = useCallback((): void => {
|
||||
deleteDashboardMutation.mutate(undefined, {
|
||||
onSuccess: () => {
|
||||
void logEvent(DashboardDetailEvents.Deleted, {
|
||||
dashboardId: dashboard.id,
|
||||
panelCount: Object.keys(dashboard.spec.panels).length,
|
||||
});
|
||||
setIsDeleteOpen(false);
|
||||
history.replace(ROUTES.ALL_DASHBOARD);
|
||||
},
|
||||
});
|
||||
}, [deleteDashboardMutation, dashboard.id, dashboard.spec.panels]);
|
||||
|
||||
const handleOpenSettings = useCallback((): void => {
|
||||
void logEvent(DashboardDetailEvents.SettingsOpened, {
|
||||
dashboardId: dashboard.id,
|
||||
@@ -237,7 +248,7 @@ function DashboardActions({
|
||||
icon: <Trash2 size={14} />,
|
||||
danger: true,
|
||||
disabled: isLocked,
|
||||
onClick: confirmDeleteDashboard,
|
||||
onClick: (): void => setIsDeleteOpen(true),
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -255,7 +266,6 @@ function DashboardActions({
|
||||
handleClone,
|
||||
onLockToggle,
|
||||
handleEnterFullScreen,
|
||||
confirmDeleteDashboard,
|
||||
]);
|
||||
|
||||
return (
|
||||
@@ -338,7 +348,14 @@ function DashboardActions({
|
||||
isOpen={isJsonEditorOpen}
|
||||
onClose={(): void => setIsJsonEditorOpen(false)}
|
||||
/>
|
||||
{deleteConfirmHolder}
|
||||
<ConfirmDeleteDialog
|
||||
open={isDeleteOpen}
|
||||
title={`Delete dashboard"?`}
|
||||
description={`Are you sure you want to delete this dashboard - "${title}"? This action cannot be undone.`}
|
||||
isLoading={deleteDashboardMutation.isLoading}
|
||||
onConfirm={handleConfirmDelete}
|
||||
onClose={(): void => setIsDeleteOpen(false)}
|
||||
/>
|
||||
<SectionTitleModal
|
||||
open={isNewSectionOpen}
|
||||
heading="New section"
|
||||
|
||||
@@ -1,89 +0,0 @@
|
||||
import { type ReactNode, useCallback } from 'react';
|
||||
import { useQueryClient } from 'react-query';
|
||||
import { toast } from '@signozhq/ui/sonner';
|
||||
import { Typography } from '@signozhq/ui/typography';
|
||||
import logEvent from 'api/common/logEvent';
|
||||
import {
|
||||
invalidateListDashboardsForUserV2,
|
||||
useDeleteDashboardV2,
|
||||
} from 'api/generated/services/dashboard';
|
||||
import { useDeleteConfirm } from 'components/DeleteConfirmModal/useDeleteConfirm';
|
||||
import ROUTES from 'constants/routes';
|
||||
import { useDashboardPreferencesStore } from 'hooks/dashboard/useDashboardPreference';
|
||||
import history from 'lib/history';
|
||||
import { DashboardDetailEvents } from 'pages/DashboardPageV2/constants/events';
|
||||
import { useErrorModal } from 'providers/ErrorModalProvider';
|
||||
import APIError from 'types/api/error';
|
||||
|
||||
import styles from './DashboardActions.module.scss';
|
||||
|
||||
interface UseDeleteDashboardActionArgs {
|
||||
dashboardId: string;
|
||||
dashboardName: string;
|
||||
panelCount: number;
|
||||
}
|
||||
|
||||
interface UseDeleteDashboardAction {
|
||||
/** Must be rendered in the calling component for the modal to appear. */
|
||||
contextHolder: ReactNode;
|
||||
confirmDeleteDashboard: () => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Deletes the open dashboard through the v2 endpoint behind the shared
|
||||
* destructive confirmation, then drops its local preferences, refreshes the
|
||||
* dashboard list cache and sends the user back to the list.
|
||||
*/
|
||||
export function useDeleteDashboardAction({
|
||||
dashboardId,
|
||||
dashboardName,
|
||||
panelCount,
|
||||
}: UseDeleteDashboardActionArgs): UseDeleteDashboardAction {
|
||||
const queryClient = useQueryClient();
|
||||
const { showErrorModal } = useErrorModal();
|
||||
const { contextHolder, confirmDelete } = useDeleteConfirm();
|
||||
const removePreferences = useDashboardPreferencesStore(
|
||||
(state) => state.removePreferences,
|
||||
);
|
||||
|
||||
const { mutate: deleteDashboard } = useDeleteDashboardV2({
|
||||
mutation: {
|
||||
onSuccess: async (): Promise<void> => {
|
||||
void logEvent(DashboardDetailEvents.Deleted, { dashboardId, panelCount });
|
||||
removePreferences(dashboardId);
|
||||
await invalidateListDashboardsForUserV2(queryClient);
|
||||
toast.success('Dashboard deleted successfully');
|
||||
history.replace(ROUTES.ALL_DASHBOARD);
|
||||
},
|
||||
onError: (error: unknown): void => {
|
||||
showErrorModal(error as APIError);
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const confirmDeleteDashboard = useCallback((): void => {
|
||||
confirmDelete({
|
||||
title: (
|
||||
<Typography.Title level={5}>
|
||||
Are you sure you want to delete the
|
||||
<Typography.Text className={styles.deleteName}>
|
||||
{' '}
|
||||
{dashboardName}{' '}
|
||||
</Typography.Text>
|
||||
dashboard?
|
||||
</Typography.Title>
|
||||
),
|
||||
content: 'This action cannot be undone.',
|
||||
// Keeps the Delete button loading until the mutation settles, then closes.
|
||||
onConfirm: () =>
|
||||
new Promise<void>((resolve) => {
|
||||
deleteDashboard(
|
||||
{ pathParams: { id: dashboardId } },
|
||||
{ onSettled: () => resolve() },
|
||||
);
|
||||
}),
|
||||
});
|
||||
}, [confirmDelete, dashboardName, deleteDashboard, dashboardId]);
|
||||
|
||||
return { contextHolder, confirmDeleteDashboard };
|
||||
}
|
||||
@@ -1,6 +1,5 @@
|
||||
import { useCallback } from 'react';
|
||||
import { SolidAlertTriangle, X } from '@signozhq/icons';
|
||||
import { Badge } from '@signozhq/ui/badge';
|
||||
import { Button } from '@signozhq/ui/button';
|
||||
import { DialogWrapper } from '@signozhq/ui/dialog';
|
||||
import { Divider } from '@signozhq/ui/divider';
|
||||
@@ -14,7 +13,6 @@ import DisabledControlTooltip from '../../components/DisabledControlTooltip/Disa
|
||||
import styles from './Header.module.scss';
|
||||
|
||||
interface HeaderProps {
|
||||
/** Unsaved edits exist — shows the "Unsaved Changes" badge and gates the discard confirmation on close (not the Save button). */
|
||||
isDirty: boolean;
|
||||
isSaving: boolean;
|
||||
showSwitchToView?: boolean;
|
||||
@@ -68,11 +66,6 @@ function Header({
|
||||
/>
|
||||
<Divider type="vertical" />
|
||||
<Typography.Text>Configure panel</Typography.Text>
|
||||
{isDirty && (
|
||||
<Badge color="warning" data-testid="panel-editor-v2-unsaved-badge">
|
||||
Unsaved Changes
|
||||
</Badge>
|
||||
)}
|
||||
</div>
|
||||
<div className={styles.actions}>
|
||||
<HeaderRightSection
|
||||
@@ -95,7 +88,7 @@ function Header({
|
||||
variant="solid"
|
||||
color="primary"
|
||||
data-testid="panel-editor-v2-save"
|
||||
disabled={readOnly || isSaving}
|
||||
disabled={readOnly || !isDirty || isSaving}
|
||||
loading={!readOnly && isSaving}
|
||||
onClick={readOnly ? undefined : onSave}
|
||||
>
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import type { ComponentProps } from 'react';
|
||||
import { MemoryRouter } from 'react-router-dom';
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import { TooltipProvider } from '@signozhq/ui/tooltip';
|
||||
@@ -24,9 +23,7 @@ jest.mock('api/common/logEvent', () => ({
|
||||
|
||||
const mockUseIsAIAssistantEnabled = useIsAIAssistantEnabled as jest.Mock;
|
||||
|
||||
function renderHeader(
|
||||
props: Partial<ComponentProps<typeof Header>> = {},
|
||||
): void {
|
||||
function renderHeader(): void {
|
||||
// AppLayout supplies the TooltipProvider in the app; the header is rendered bare here.
|
||||
render(
|
||||
<MemoryRouter>
|
||||
@@ -36,7 +33,6 @@ function renderHeader(
|
||||
isSaving={false}
|
||||
onSave={jest.fn()}
|
||||
onClose={jest.fn()}
|
||||
{...props}
|
||||
/>
|
||||
</TooltipProvider>
|
||||
</MemoryRouter>,
|
||||
@@ -70,34 +66,4 @@ describe('PanelEditor Header', () => {
|
||||
).not.toBeInTheDocument();
|
||||
expect(screen.getByTestId('panel-editor-v2-save')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('keeps Save enabled even when there are no unsaved edits', () => {
|
||||
mockUseIsAIAssistantEnabled.mockReturnValue(false);
|
||||
|
||||
renderHeader({ isDirty: false });
|
||||
|
||||
expect(screen.getByTestId('panel-editor-v2-save')).toBeEnabled();
|
||||
});
|
||||
|
||||
it('disables Save only while read-only or saving', () => {
|
||||
mockUseIsAIAssistantEnabled.mockReturnValue(false);
|
||||
|
||||
renderHeader({ isDirty: true, readOnly: true, readOnlyReason: 'Locked' });
|
||||
|
||||
expect(screen.getByTestId('panel-editor-v2-save')).toBeDisabled();
|
||||
});
|
||||
|
||||
it('shows the Unsaved Changes badge only when there are unsaved edits', () => {
|
||||
mockUseIsAIAssistantEnabled.mockReturnValue(false);
|
||||
|
||||
renderHeader({ isDirty: false });
|
||||
expect(
|
||||
screen.queryByTestId('panel-editor-v2-unsaved-badge'),
|
||||
).not.toBeInTheDocument();
|
||||
|
||||
renderHeader({ isDirty: true });
|
||||
expect(
|
||||
screen.getByTestId('panel-editor-v2-unsaved-badge'),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { render, screen, waitFor } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { toast } from '@signozhq/ui/sonner';
|
||||
import type { DashboardtypesPanelDTO } from 'api/generated/services/sigNoz.schemas';
|
||||
import { PANEL_TYPES } from 'constants/queryBuilder';
|
||||
import { getPanelDefinition } from 'pages/DashboardPageV2/DashboardContainer/Panels/registry';
|
||||
@@ -101,12 +100,6 @@ jest.mock('@signozhq/ui/resizable', () => ({
|
||||
jest.mock('@signozhq/ui/sonner', () => ({
|
||||
toast: { success: jest.fn(), error: jest.fn() },
|
||||
}));
|
||||
const mockShowErrorModal = jest.fn();
|
||||
jest.mock('providers/ErrorModalProvider', () => ({
|
||||
useErrorModal: (): { showErrorModal: jest.Mock } => ({
|
||||
showErrorModal: mockShowErrorModal,
|
||||
}),
|
||||
}));
|
||||
|
||||
// Children mocked to capture props (and expose a Save trigger / footer slot).
|
||||
const mockHeaderProps = jest.fn();
|
||||
@@ -268,13 +261,13 @@ describe('PanelEditorContainer composition', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps a query-less new panel clean but still serializes its seed query', () => {
|
||||
it('keeps a query-less new panel unsaveable but still serializes its seed query', () => {
|
||||
setup(makePanel('signoz/TimeSeriesPanel'), { isNew: true });
|
||||
|
||||
expect(mockUseQuerySync).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ alwaysSerializeQuery: true }),
|
||||
);
|
||||
// No query and no edits yet → not dirty, so closing won't prompt to discard.
|
||||
// No query and no edits yet → nothing to save, so Save stays disabled.
|
||||
expect(mockHeaderProps).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ isDirty: false }),
|
||||
);
|
||||
@@ -297,7 +290,7 @@ describe('PanelEditorContainer composition', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('marks a new panel that already has a query dirty (e.g. list auto-runs one)', () => {
|
||||
it('marks a new panel that already has a query saveable (e.g. list auto-runs one)', () => {
|
||||
const seededQuery = {
|
||||
spec: { plugin: { kind: 'signoz/BuilderQuery', spec: { signal: 'logs' } } },
|
||||
};
|
||||
@@ -334,24 +327,6 @@ describe('PanelEditorContainer composition', () => {
|
||||
expect(mockSave).toHaveBeenCalledWith(panel.spec);
|
||||
});
|
||||
|
||||
it('surfaces a save failure through the error modal', async () => {
|
||||
// The raw thrown value flows straight to the modal, which normalizes it to an
|
||||
// APIError itself (that normalization is covered by ErrorModalProvider's tests).
|
||||
const failure = {
|
||||
response: {
|
||||
status: 400,
|
||||
data: { error: { code: 'INVALID', message: 'Panel name already exists' } },
|
||||
},
|
||||
};
|
||||
mockSave.mockRejectedValueOnce(failure);
|
||||
setup(makePanel('signoz/TimeSeriesPanel'));
|
||||
|
||||
await userEvent.click(screen.getByTestId('editor-save'));
|
||||
|
||||
await waitFor(() => expect(mockShowErrorModal).toHaveBeenCalledWith(failure));
|
||||
expect(toast.error).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('marks the saved panel to be scrolled into view on the dashboard', async () => {
|
||||
setup(makePanel('signoz/TimeSeriesPanel'));
|
||||
|
||||
|
||||
@@ -117,49 +117,6 @@ describe('usePanelEditorQuerySync (real query builder)', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('a NEW panel stays query-dirty after Stage & Run commits the edited query into the draft', async () => {
|
||||
// Repro of the P0: new panel → edit → Run commits the query into the draft; the
|
||||
// dirty baseline must not drift onto it, or Save re-disables.
|
||||
const editedInUrl: Query = {
|
||||
...initialQueriesMap[DataSource.METRICS],
|
||||
id: 'edited-new-panel',
|
||||
builder: {
|
||||
...initialQueriesMap[DataSource.METRICS].builder,
|
||||
queryData: [
|
||||
{
|
||||
...initialQueriesMap[DataSource.METRICS].builder.queryData[0],
|
||||
legend: 'edited-legend',
|
||||
},
|
||||
],
|
||||
},
|
||||
};
|
||||
const committed = toPerses(editedInUrl, panelType);
|
||||
|
||||
const { result, rerender } = renderHook(
|
||||
({ draftQueries }: { draftQueries: DashboardtypesQueryDTO[] }) =>
|
||||
usePanelEditorQuerySync({
|
||||
draft: makePanel(draftQueries),
|
||||
panelType,
|
||||
setSpec: jest.fn(),
|
||||
refetch: jest.fn(),
|
||||
alwaysSerializeQuery: true,
|
||||
signal: DataSource.METRICS as unknown as TelemetrytypesSignalDTO,
|
||||
// savedQueries omitted — new panel.
|
||||
}),
|
||||
{
|
||||
wrapper: makeUrlWrapper(editedInUrl),
|
||||
initialProps: { draftQueries: [] as DashboardtypesQueryDTO[] },
|
||||
},
|
||||
);
|
||||
|
||||
// The edited query (from the URL) diverges from the seeded default → dirty.
|
||||
await waitFor(() => expect(result.current.isQueryDirty).toBe(true));
|
||||
|
||||
// Stage & Run commits the edited query into the draft → must stay dirty.
|
||||
rerender({ draftQueries: committed });
|
||||
expect(result.current.isQueryDirty).toBe(true);
|
||||
});
|
||||
|
||||
it('an untouched panel with a minimal/older stored query is NOT dirty (drift fix)', async () => {
|
||||
// An older saved query carries only a few fields; the builder re-emits many more
|
||||
// (source, stepInterval, filter, spaceAggregation, …). Comparing raw would read
|
||||
|
||||
@@ -79,11 +79,6 @@ export function usePanelEditorQuerySync({
|
||||
// in-editor edit in the URL survives a refresh / browser Back-Forward.
|
||||
useShareBuilderUrl({ defaultValue: seedQuery });
|
||||
|
||||
// Frozen seed for the new-panel dirty baseline: Stage & Run commits the live query
|
||||
// into `draft.spec.queries`, so re-reading `seedQuery` would drift the baseline onto
|
||||
// the run query and Save would re-disable right after a run.
|
||||
const initialSeedRef = useRef(seedQuery);
|
||||
|
||||
// Commit the live query into the draft (what the preview fetches).
|
||||
const commitQuery = useCallback(
|
||||
(query: Query): boolean => {
|
||||
@@ -157,19 +152,16 @@ export function usePanelEditorQuerySync({
|
||||
// the V5 envelope level. Anchoring to `savedQueries` (not the builder-seed) keeps a
|
||||
// handed-off / URL-restored edit reading as dirty; routing both sides through the same
|
||||
// `fromPerses → toPerses` round-trip stops builder-added defaults (absent from an older
|
||||
// stored query) reading an untouched panel as modified. New panel: fall back to the
|
||||
// frozen initial seed (see `initialSeedRef`).
|
||||
// stored query) reading an untouched panel as modified. New panel: fall back to seed.
|
||||
const baselineEnvelopes = useMemo(
|
||||
() =>
|
||||
toQueryEnvelopes(
|
||||
toPerses(
|
||||
savedQueries
|
||||
? fromPerses(savedQueries, panelType)
|
||||
: initialSeedRef.current,
|
||||
savedQueries ? fromPerses(savedQueries, panelType) : seedQuery,
|
||||
panelType,
|
||||
),
|
||||
),
|
||||
[savedQueries, panelType],
|
||||
[savedQueries, seedQuery, panelType],
|
||||
);
|
||||
const isQueryDirty = useMemo(
|
||||
() =>
|
||||
|
||||
@@ -19,7 +19,6 @@ import {
|
||||
SectionKind,
|
||||
} from 'pages/DashboardPageV2/DashboardContainer/Panels/types/sections';
|
||||
import { getBuilderQueries } from 'pages/DashboardPageV2/DashboardContainer/Panels/utils/getBuilderQueries';
|
||||
import { useErrorModal } from 'providers/ErrorModalProvider';
|
||||
|
||||
import { getExecStats } from '../queryV5/v5ResponseData';
|
||||
import { usePanelInteractions } from '../PanelsAndSectionsLayout/Panel/hooks/usePanelInteractions';
|
||||
@@ -160,10 +159,9 @@ function PanelEditorContainer({
|
||||
return section?.controls;
|
||||
}, [panelDefinition]);
|
||||
|
||||
// Unsaved-edits flag driving the discard confirmation on close (Save is always
|
||||
// enabled). Read the seed `panel`, not the live `draft` — the staged-query sync
|
||||
// commits the seed into the draft on open, which would falsely dirty an untouched
|
||||
// query-less new panel.
|
||||
// New panels are savable once seeded with a query (List auto-seeds one). Read the
|
||||
// seed `panel`, not the live `draft` — the staged-query sync commits the seed into
|
||||
// the draft on open, which would falsely dirty an untouched query-less new panel.
|
||||
const isDirty = useMemo(
|
||||
() => isSpecDirty || isQueryDirty || (isNew && panel.spec.queries.length > 0),
|
||||
[isSpecDirty, isQueryDirty, isNew, panel.spec.queries.length],
|
||||
@@ -228,7 +226,6 @@ function PanelEditorContainer({
|
||||
});
|
||||
|
||||
const setScrollTargetId = useScrollIntoViewStore((s) => s.setScrollTargetId);
|
||||
const { showErrorModal } = useErrorModal();
|
||||
|
||||
const onSave = useCallback(async (): Promise<void> => {
|
||||
if (!isEditable) {
|
||||
@@ -239,22 +236,12 @@ function PanelEditorContainer({
|
||||
const savedPanelId = await save(buildSaveSpec(draft.spec));
|
||||
// Reveal the saved panel once the dashboard re-renders.
|
||||
setScrollTargetId(savedPanelId);
|
||||
toast.success('Panel saved', {
|
||||
position: 'top-center',
|
||||
});
|
||||
toast.success('Panel saved');
|
||||
onSaved();
|
||||
} catch (err) {
|
||||
showErrorModal(err);
|
||||
} catch {
|
||||
toast.error('Failed to save panel');
|
||||
}
|
||||
}, [
|
||||
isEditable,
|
||||
save,
|
||||
buildSaveSpec,
|
||||
draft.spec,
|
||||
setScrollTargetId,
|
||||
onSaved,
|
||||
showErrorModal,
|
||||
]);
|
||||
}, [isEditable, save, buildSaveSpec, draft.spec, setScrollTargetId, onSaved]);
|
||||
|
||||
// Leaving an existing panel's editor (without saving) still returns to it, so
|
||||
// the dashboard lands on that panel rather than scrolled to the top. A new,
|
||||
|
||||
@@ -51,7 +51,7 @@ describe('panelStatusFromError', () => {
|
||||
|
||||
it('falls back to the error message when there is no structured body', () => {
|
||||
expect(panelStatusFromError(new Error('boom'))).toStrictEqual({
|
||||
code: 'UPSTREAM_UNAVAILABLE',
|
||||
code: 'unknown_error',
|
||||
message: 'boom',
|
||||
docsUrl: undefined,
|
||||
messages: [],
|
||||
|
||||
@@ -12,13 +12,13 @@ import {
|
||||
deleteDashboardV2,
|
||||
invalidateListDashboardsForUserV2,
|
||||
} from 'api/generated/services/dashboard';
|
||||
import { useDeleteConfirm } from 'components/DeleteConfirmModal/useDeleteConfirm';
|
||||
import { DashboardListEvents } from 'pages/DashboardsListPageV2/constants/events';
|
||||
import { useAppContext } from 'providers/App/App';
|
||||
import { useErrorModal } from 'providers/ErrorModalProvider';
|
||||
import APIError from 'types/api/error';
|
||||
import { USER_ROLES } from 'types/roles';
|
||||
|
||||
import { useDeleteConfirm } from '../DeleteConfirmModal/useDeleteConfirm';
|
||||
import styles from './ActionsPopover.module.scss';
|
||||
|
||||
interface Props {
|
||||
|
||||
@@ -6,11 +6,11 @@ import { Typography } from '@signozhq/ui/typography';
|
||||
import { Bookmark, PenLine, Plus, Search, Trash2 } from '@signozhq/icons';
|
||||
import cx from 'classnames';
|
||||
|
||||
import { useDeleteConfirm } from 'components/DeleteConfirmModal/useDeleteConfirm';
|
||||
import { DashboardListEvents } from 'pages/DashboardsListPageV2/constants/events';
|
||||
|
||||
import type { SavedView } from '../../types';
|
||||
import { type BuiltinView } from '../../utils/views';
|
||||
import { useDeleteConfirm } from '../DeleteConfirmModal/useDeleteConfirm';
|
||||
import ViewNamePopover from './ViewNamePopover';
|
||||
|
||||
import styles from './ViewsRail.module.scss';
|
||||
|
||||
@@ -42,12 +42,7 @@ export function toAPIError(
|
||||
try {
|
||||
ErrorResponseHandlerForGeneratedAPIs(error);
|
||||
} catch (apiError) {
|
||||
// UPSTREAM_UNAVAILABLE means the handler couldn't find a backend error code
|
||||
// (non-envelope body); prefer the caller's context-specific defaultMessage.
|
||||
if (
|
||||
apiError instanceof APIError &&
|
||||
apiError.getErrorCode() !== 'UPSTREAM_UNAVAILABLE'
|
||||
) {
|
||||
if (apiError instanceof APIError) {
|
||||
return apiError;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,14 +5,14 @@ import (
|
||||
"fmt"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
func (m *module) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
|
||||
stats := make(map[string]any)
|
||||
|
||||
metadataTable := fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName)
|
||||
metadataTable := fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName)
|
||||
var (
|
||||
systemMetricCount uint64
|
||||
k8sMetricCount uint64
|
||||
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -281,7 +281,7 @@ func (m *module) getPerGroupContainerStatusCounts(
|
||||
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
|
||||
|
||||
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
|
||||
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
|
||||
// Built once; identical across the two fps CTEs (buildFilterClause hits the
|
||||
// metadata store + parses the expression). AddWhereClause only reads it.
|
||||
@@ -310,7 +310,7 @@ func (m *module) getPerGroupContainerStatusCounts(
|
||||
)
|
||||
}
|
||||
stateFps.Select(stateFpsCols...)
|
||||
stateFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
stateFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
stateFps.Where(
|
||||
stateFps.E("metric_name", containerStatusStateMetricName),
|
||||
stateFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -342,7 +342,7 @@ func (m *module) getPerGroupContainerStatusCounts(
|
||||
containerState.Select(containerStateCols...)
|
||||
containerState.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN state_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
containerState.Where(
|
||||
containerState.E("samples.metric_name", containerStatusStateMetricName),
|
||||
@@ -361,7 +361,7 @@ func (m *module) getPerGroupContainerStatusCounts(
|
||||
fmt.Sprintf("JSONExtractString(labels, %s) AS container_name", reasonFps.Var(containerNameAttrKey)),
|
||||
fmt.Sprintf("JSONExtractString(labels, %s) AS reason", reasonFps.Var(containerStatusReasonAttrKey)),
|
||||
)
|
||||
reasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
reasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
reasonFps.Where(
|
||||
reasonFps.E("metric_name", containerStatusReasonMetricName),
|
||||
reasonFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -391,7 +391,7 @@ func (m *module) getPerGroupContainerStatusCounts(
|
||||
)
|
||||
reasonInner.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
reasonInner.Where(
|
||||
reasonInner.E("samples.metric_name", containerStatusReasonMetricName),
|
||||
@@ -546,7 +546,7 @@ func (m *module) getPerGroupContainerRestartCounts(
|
||||
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
|
||||
|
||||
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
|
||||
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
|
||||
var (
|
||||
filterClause *sqlbuilder.WhereClause
|
||||
@@ -572,7 +572,7 @@ func (m *module) getPerGroupContainerRestartCounts(
|
||||
)
|
||||
}
|
||||
restartFps.Select(restartFpsCols...)
|
||||
restartFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
restartFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
restartFps.Where(
|
||||
restartFps.E("metric_name", containerRestartsMetricName),
|
||||
restartFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -602,7 +602,7 @@ func (m *module) getPerGroupContainerRestartCounts(
|
||||
containerRestarts.Select(containerRestartsCols...)
|
||||
containerRestarts.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN restart_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
containerRestarts.Where(
|
||||
containerRestarts.E("samples.metric_name", containerRestartsMetricName),
|
||||
@@ -690,7 +690,7 @@ func (m *module) getPerGroupContainerReadyCounts(
|
||||
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
|
||||
|
||||
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
|
||||
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
|
||||
var (
|
||||
filterClause *sqlbuilder.WhereClause
|
||||
@@ -716,7 +716,7 @@ func (m *module) getPerGroupContainerReadyCounts(
|
||||
)
|
||||
}
|
||||
readyFps.Select(readyFpsCols...)
|
||||
readyFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
readyFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
readyFps.Where(
|
||||
readyFps.E("metric_name", containerReadyMetricName),
|
||||
readyFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -746,7 +746,7 @@ func (m *module) getPerGroupContainerReadyCounts(
|
||||
containerReady.Select(containerReadyCols...)
|
||||
containerReady.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN ready_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
containerReady.Where(
|
||||
containerReady.E("samples.metric_name", containerReadyMetricName),
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/types/featuretypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/metrictypes"
|
||||
@@ -344,11 +344,11 @@ func alignedMetricWindow(startMs, endMs int64) (
|
||||
flooredEndMs = flooredEndMs - (flooredEndMs % (adjustStep * 1000))
|
||||
}
|
||||
|
||||
tsAdjustedStartMs, _, distributedTSTable, localTSTable := metricstelemetryschema.WhichTSTableToUse(
|
||||
tsAdjustedStartMs, _, distributedTSTable, localTSTable := telemetrymetrics.WhichTSTableToUse(
|
||||
samplesAdjustedStartMs, flooredEndMs, false, nil,
|
||||
)
|
||||
|
||||
distributedSamplesTable, localSamplesTable := metricstelemetryschema.WhichSamplesTableToUse(
|
||||
distributedSamplesTable, localSamplesTable := telemetrymetrics.WhichSamplesTableToUse(
|
||||
samplesAdjustedStartMs, flooredEndMs,
|
||||
metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil,
|
||||
)
|
||||
@@ -362,7 +362,7 @@ func alignedMetricWindow(startMs, endMs int64) (
|
||||
func (m *module) buildSamplesTblFingerprintSubQuery(metricNames []string, samplesTable string, flooredStart, flooredEnd uint64) *sqlbuilder.SelectBuilder {
|
||||
fpSB := sqlbuilder.NewSelectBuilder()
|
||||
fpSB.Select("DISTINCT fingerprint")
|
||||
fpSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, samplesTable))
|
||||
fpSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, samplesTable))
|
||||
fpSB.Where(
|
||||
fpSB.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
fpSB.GE("unix_milli", flooredStart),
|
||||
@@ -376,7 +376,7 @@ func (m *module) buildSamplesTblFingerprintSubQuery(metricNames []string, sample
|
||||
func (m *module) buildReducedSamplesTblFingerprintSubQuery(metricNames []string, flooredStart, flooredEnd uint64) *sqlbuilder.SelectBuilder {
|
||||
lastSB := sqlbuilder.NewSelectBuilder()
|
||||
lastSB.Select("reduced_fingerprint")
|
||||
lastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
|
||||
lastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
|
||||
lastSB.Where(
|
||||
lastSB.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
lastSB.GE("unix_milli", flooredStart),
|
||||
@@ -385,7 +385,7 @@ func (m *module) buildReducedSamplesTblFingerprintSubQuery(metricNames []string,
|
||||
|
||||
sumSB := sqlbuilder.NewSelectBuilder()
|
||||
sumSB.Select("reduced_fingerprint")
|
||||
sumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
|
||||
sumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
|
||||
sumSB.Where(
|
||||
sumSB.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
sumSB.GE("unix_milli", flooredStart),
|
||||
@@ -478,7 +478,7 @@ func (m *module) getEarliestMetricTime(ctx context.Context, metricNames []string
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("min(first_reported_unix_milli) AS min_first_reported")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
@@ -504,7 +504,7 @@ func (m *module) getMetricsExistence(ctx context.Context, metricNames []string)
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("metric_name", "count(*) AS cnt")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(sb.In("metric_name", sqlbuilder.List(metricNames)))
|
||||
sb.GroupBy("metric_name")
|
||||
|
||||
@@ -549,7 +549,7 @@ func (m *module) getAttributesExistence(ctx context.Context, metricNames, attrNa
|
||||
}
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("attr_name", "count(*) AS cnt")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(
|
||||
sb.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
sb.In("attr_name", sqlbuilder.List(attrNames)),
|
||||
@@ -656,7 +656,7 @@ func (m *module) getMetadata(
|
||||
|
||||
rawSrc := sqlbuilder.NewSelectBuilder()
|
||||
rawSrc.Select("labels", "unix_milli")
|
||||
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTableName))
|
||||
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTableName))
|
||||
rawSrc.Where(
|
||||
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
rawSrc.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -669,7 +669,7 @@ func (m *module) getMetadata(
|
||||
|
||||
reducedSrc := sqlbuilder.NewSelectBuilder()
|
||||
reducedSrc.Select("labels", "unix_milli")
|
||||
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
reducedSrc.Where(
|
||||
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -683,7 +683,7 @@ func (m *module) getMetadata(
|
||||
// Inner query reads over the union of raw + reduced series.
|
||||
innerSB.From(innerSB.BuilderAs(sqlbuilder.UnionAll(rawSrc, reducedSrc), "series"))
|
||||
} else {
|
||||
innerSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTableName))
|
||||
innerSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTableName))
|
||||
innerSB.Where(
|
||||
innerSB.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
innerSB.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -870,7 +870,7 @@ func (m *module) getPerGroupDistinctCounts(
|
||||
|
||||
rawSrc := sqlbuilder.NewSelectBuilder()
|
||||
rawSrc.Select("labels")
|
||||
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTbl))
|
||||
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTbl))
|
||||
rawSrc.Where(
|
||||
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
rawSrc.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -883,7 +883,7 @@ func (m *module) getPerGroupDistinctCounts(
|
||||
|
||||
reducedSrc := sqlbuilder.NewSelectBuilder()
|
||||
reducedSrc.Select("labels")
|
||||
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
reducedSrc.Where(
|
||||
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -896,7 +896,7 @@ func (m *module) getPerGroupDistinctCounts(
|
||||
|
||||
sb.From(sb.BuilderAs(sqlbuilder.UnionAll(rawSrc, reducedSrc), "series"))
|
||||
} else {
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTbl))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTbl))
|
||||
sb.Where(
|
||||
sb.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
sb.GE("unix_milli", tsAdjustedStartMs),
|
||||
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/types/featuretypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
@@ -72,7 +72,7 @@ func (m *module) getPerGroupHostStatusCounts(
|
||||
|
||||
rawSrc := sqlbuilder.NewSelectBuilder()
|
||||
rawSrc.Select("labels")
|
||||
rawSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTableName))
|
||||
rawSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTableName))
|
||||
rawSrc.Where(
|
||||
rawSrc.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
rawSrc.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -85,7 +85,7 @@ func (m *module) getPerGroupHostStatusCounts(
|
||||
|
||||
reducedSrc := sqlbuilder.NewSelectBuilder()
|
||||
reducedSrc.Select("labels")
|
||||
reducedSrc.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedSrc.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
reducedSrc.Where(
|
||||
reducedSrc.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
reducedSrc.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -101,7 +101,7 @@ func (m *module) getPerGroupHostStatusCounts(
|
||||
|
||||
fpSB := m.buildSamplesTblFingerprintSubQuery(metricNames, localSamplesTable, samplesStartMs, flooredEndMs)
|
||||
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTimeSeriesTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTimeSeriesTableName))
|
||||
sb.Where(
|
||||
sb.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
sb.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -357,7 +357,7 @@ func (m *module) getActiveHostsQuery(metricNames []string, hostNameAttr string,
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Distinct()
|
||||
sb.Select("attr_string_value")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(
|
||||
sb.In("metric_name", sqlbuilder.List(metricNames)),
|
||||
sb.E("attr_name", hostNameAttr),
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/modules/inframonitoring"
|
||||
"github.com/SigNoz/signoz/pkg/querier"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
@@ -40,8 +40,8 @@ func NewModule(
|
||||
providerSettings factory.ProviderSettings,
|
||||
cfg inframonitoring.Config,
|
||||
) inframonitoring.Module {
|
||||
fieldMapper := metricstelemetryschema.NewFieldMapper()
|
||||
condBuilder := metricstelemetryschema.NewConditionBuilder(fieldMapper)
|
||||
fieldMapper := telemetrymetrics.NewFieldMapper()
|
||||
condBuilder := telemetrymetrics.NewConditionBuilder(fieldMapper)
|
||||
return &module{
|
||||
telemetryStore: telemetryStore,
|
||||
telemetryMetadataStore: telemetryMetadataStore,
|
||||
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -186,7 +186,7 @@ func (m *module) getPerGroupNodeConditionCounts(
|
||||
|
||||
// Step-floor bounds + resolve tables in one shot to match QB v5 querier.
|
||||
samplesStartMs, flooredEndMs, tsAdjustedStartMs, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
|
||||
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
|
||||
// ----- timeSeriesFPs -----
|
||||
timeSeriesFPs := sqlbuilder.NewSelectBuilder()
|
||||
@@ -200,7 +200,7 @@ func (m *module) getPerGroupNodeConditionCounts(
|
||||
)
|
||||
}
|
||||
timeSeriesFPs.Select(timeSeriesFPsSelectCols...)
|
||||
timeSeriesFPs.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
timeSeriesFPs.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
timeSeriesFPs.Where(
|
||||
timeSeriesFPs.E("metric_name", nodeConditionMetricName),
|
||||
timeSeriesFPs.GE("unix_milli", tsAdjustedStartMs),
|
||||
@@ -237,7 +237,7 @@ func (m *module) getPerGroupNodeConditionCounts(
|
||||
latestConditionPerNode.Select(latestConditionPerNodeSelectCols...)
|
||||
latestConditionPerNode.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN time_series_fps AS tsfp ON samples.fingerprint = tsfp.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
latestConditionPerNode.Where(
|
||||
latestConditionPerNode.E("samples.metric_name", nodeConditionMetricName),
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -284,7 +284,7 @@ func (m *module) getPerGroupPodStatusCounts(
|
||||
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
|
||||
|
||||
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
|
||||
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
|
||||
// Build the merged filter clause once; it's identical across the three fps
|
||||
// CTEs, and buildFilterClause hits the metadata store + parses the
|
||||
@@ -313,7 +313,7 @@ func (m *module) getPerGroupPodStatusCounts(
|
||||
)
|
||||
}
|
||||
phaseFps.Select(phaseFpsCols...)
|
||||
phaseFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
phaseFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
phaseFps.Where(
|
||||
phaseFps.E("metric_name", podPhaseMetricName),
|
||||
phaseFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -340,7 +340,7 @@ func (m *module) getPerGroupPodStatusCounts(
|
||||
phasePerPod.Select(phasePerPodCols...)
|
||||
phasePerPod.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN phase_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
phasePerPod.Where(
|
||||
phasePerPod.E("samples.metric_name", podPhaseMetricName),
|
||||
@@ -357,7 +357,7 @@ func (m *module) getPerGroupPodStatusCounts(
|
||||
"fingerprint",
|
||||
fmt.Sprintf("JSONExtractString(labels, %s) AS pod_uid", podReasonFps.Var(podUIDAttrKey)),
|
||||
)
|
||||
podReasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
podReasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
podReasonFps.Where(
|
||||
podReasonFps.E("metric_name", podStatusReasonMetricName),
|
||||
podReasonFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -377,7 +377,7 @@ func (m *module) getPerGroupPodStatusCounts(
|
||||
)
|
||||
podReasonPerPod.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN pod_reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
podReasonPerPod.Where(
|
||||
podReasonPerPod.E("samples.metric_name", podStatusReasonMetricName),
|
||||
@@ -396,7 +396,7 @@ func (m *module) getPerGroupPodStatusCounts(
|
||||
fmt.Sprintf("JSONExtractString(labels, %s) AS container_name", containerReasonFps.Var(containerNameAttrKey)),
|
||||
fmt.Sprintf("JSONExtractString(labels, %s) AS reason", containerReasonFps.Var(containerStatusReasonAttrKey)),
|
||||
)
|
||||
containerReasonFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
containerReasonFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
containerReasonFps.Where(
|
||||
containerReasonFps.E("metric_name", containerStatusReasonMetricName),
|
||||
containerReasonFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -432,7 +432,7 @@ func (m *module) getPerGroupPodStatusCounts(
|
||||
)
|
||||
containerInner.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN container_reason_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
containerInner.Where(
|
||||
containerInner.E("samples.metric_name", containerStatusReasonMetricName),
|
||||
@@ -611,7 +611,7 @@ func (m *module) getPerGroupPodRestartCounts(
|
||||
mergedFilterExpr := mergeFilterExpressions(userFilterExpr, buildPageGroupsFilterExpr(pageGroups))
|
||||
|
||||
samplesStartMs, flooredEndMs, tsAdjustedStart, _, localTimeSeriesTable, distributedSamplesTable, _ := alignedMetricWindow(start, end)
|
||||
valueCol := metricstelemetryschema.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
valueCol := telemetrymetrics.ValueColumnForSamplesTable(distributedSamplesTable)
|
||||
|
||||
var (
|
||||
filterClause *sqlbuilder.WhereClause
|
||||
@@ -637,7 +637,7 @@ func (m *module) getPerGroupPodRestartCounts(
|
||||
)
|
||||
}
|
||||
restartFps.Select(restartFpsCols...)
|
||||
restartFps.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTimeSeriesTable))
|
||||
restartFps.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
|
||||
restartFps.Where(
|
||||
restartFps.E("metric_name", containerRestartsMetricName),
|
||||
restartFps.GE("unix_milli", tsAdjustedStart),
|
||||
@@ -667,7 +667,7 @@ func (m *module) getPerGroupPodRestartCounts(
|
||||
containerRestarts.Select(containerRestartsCols...)
|
||||
containerRestarts.From(fmt.Sprintf(
|
||||
"%s.%s AS samples INNER JOIN restart_fps AS fps ON samples.fingerprint = fps.fingerprint",
|
||||
metricstelemetryschema.DBName, distributedSamplesTable,
|
||||
telemetrymetrics.DBName, distributedSamplesTable,
|
||||
))
|
||||
containerRestarts.Where(
|
||||
containerRestarts.E("samples.metric_name", containerRestartsMetricName),
|
||||
|
||||
@@ -19,8 +19,8 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/modules/dashboard"
|
||||
"github.com/SigNoz/signoz/pkg/modules/metricsexplorer"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymeter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
|
||||
@@ -49,8 +49,8 @@ type module struct {
|
||||
|
||||
// NewModule constructs the metrics module with the provided dependencies.
|
||||
func NewModule(ts telemetrystore.TelemetryStore, telemetryMetadataStore telemetrytypes.MetadataStore, cache cache.Cache, ruleStore ruletypes.RuleStore, dashboardModule dashboard.Module, fl flagger.Flagger, providerSettings factory.ProviderSettings, cfg metricsexplorer.Config) metricsexplorer.Module {
|
||||
fieldMapper := metricstelemetryschema.NewFieldMapper()
|
||||
condBuilder := metricstelemetryschema.NewConditionBuilder(fieldMapper)
|
||||
fieldMapper := telemetrymetrics.NewFieldMapper()
|
||||
condBuilder := telemetrymetrics.NewConditionBuilder(fieldMapper)
|
||||
return &module{
|
||||
telemetryStore: ts,
|
||||
fieldMapper: fieldMapper,
|
||||
@@ -89,7 +89,7 @@ func (m *module) listMeterMetrics(ctx context.Context, params *metricsexplorerty
|
||||
"argMax(temporality, unix_milli) AS temporality",
|
||||
"argMax(is_monotonic, unix_milli) AS is_monotonic",
|
||||
)
|
||||
sb.From(fmt.Sprintf("%s.%s", metertelemetryschema.DBName, metertelemetryschema.SamplesTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymeter.DBName, telemetrymeter.SamplesTableName))
|
||||
|
||||
if params.Start != nil && params.End != nil {
|
||||
sb.Where(sb.Between("unix_milli", *params.Start, *params.End))
|
||||
@@ -160,11 +160,11 @@ func (m *module) listMetrics(ctx context.Context, orgID valuer.UUID, params *met
|
||||
hasRange := params.Start != nil && params.End != nil
|
||||
if hasRange {
|
||||
var distributedTsTable string
|
||||
rangeStart, rangeEnd, distributedTsTable, _ = metricstelemetryschema.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), false, nil)
|
||||
rawSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
|
||||
rangeStart, rangeEnd, distributedTsTable, _ = telemetrymetrics.WhichTSTableToUse(uint64(*params.Start), uint64(*params.End), false, nil)
|
||||
rawSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
|
||||
rawSB.Where(rawSB.Between("unix_milli", rangeStart, rangeEnd))
|
||||
} else {
|
||||
rawSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
|
||||
rawSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
|
||||
}
|
||||
|
||||
applyListFilters(rawSB)
|
||||
@@ -174,7 +174,7 @@ func (m *module) listMetrics(ctx context.Context, orgID valuer.UUID, params *met
|
||||
if reductionEnabled {
|
||||
reducedSB := sqlbuilder.NewSelectBuilder()
|
||||
reducedSB.Select("DISTINCT metric_name")
|
||||
reducedSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
if hasRange {
|
||||
reducedSB.Where(reducedSB.Between("unix_milli", rangeStart, rangeEnd))
|
||||
}
|
||||
@@ -512,7 +512,7 @@ func (m *module) CheckMetricExists(ctx context.Context, orgID valuer.UUID, metri
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("count(*) > 0 as metricExists")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(sb.E("metric_name", metricName))
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
@@ -534,7 +534,7 @@ func (m *module) HasNonSigNozMetrics(ctx context.Context) (bool, error) {
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("count(*) > 0")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
|
||||
sb.Where("metric_name NOT LIKE 'signoz_%'")
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
@@ -570,10 +570,10 @@ func (m *module) InspectMetrics(
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tsStart, _, tsTable, _ := metricstelemetryschema.WhichTSTableToUse(start, end, false, nil)
|
||||
tsStart, _, tsTable, _ := telemetrymetrics.WhichTSTableToUse(start, end, false, nil)
|
||||
tsSb := sqlbuilder.NewSelectBuilder()
|
||||
tsSb.Select("fingerprint", "labels")
|
||||
tsSb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, tsTable))
|
||||
tsSb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, tsTable))
|
||||
tsSb.Where(
|
||||
tsSb.E("metric_name", req.MetricName),
|
||||
tsSb.GE("unix_milli", tsStart),
|
||||
@@ -625,7 +625,7 @@ func (m *module) InspectMetrics(
|
||||
|
||||
samplesSb := sqlbuilder.NewSelectBuilder()
|
||||
samplesSb.Select("fingerprint", "unix_milli", "value")
|
||||
samplesSb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName))
|
||||
samplesSb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4TableName))
|
||||
samplesSb.Where(
|
||||
samplesSb.In("fingerprint", fingerprints...),
|
||||
samplesSb.E("metric_name", req.MetricName),
|
||||
@@ -717,7 +717,7 @@ func (m *module) fetchUpdatedMetadata(ctx context.Context, orgID valuer.UUID, me
|
||||
"argMax(temporality, created_at) AS temporality",
|
||||
"argMax(is_monotonic, created_at) AS is_monotonic",
|
||||
)
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.UpdatedMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.UpdatedMetadataTableName))
|
||||
sb.Where(sb.In("metric_name", args...))
|
||||
sb.GroupBy("metric_name")
|
||||
|
||||
@@ -777,7 +777,7 @@ func (m *module) fetchTimeseriesMetadata(ctx context.Context, orgID valuer.UUID,
|
||||
"anyLast(temporality) AS temporality",
|
||||
"argMax(is_monotonic, unix_milli) AS is_monotonic",
|
||||
)
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4TableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4TableName))
|
||||
sb.Where(sb.In("metric_name", args...))
|
||||
sb.GroupBy("metric_name")
|
||||
|
||||
@@ -891,7 +891,7 @@ func (m *module) checkForLabelInMetric(ctx context.Context, metricName string, l
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("count(*) > 0 AS has_label")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(sb.E("metric_name", metricName))
|
||||
sb.Where(sb.E("attr_name", label))
|
||||
sb.Limit(1)
|
||||
@@ -914,7 +914,7 @@ func (m *module) insertMetricsMetadata(ctx context.Context, orgID valuer.UUID, r
|
||||
createdAt := time.Now().UnixMilli()
|
||||
|
||||
ib := sqlbuilder.NewInsertBuilder()
|
||||
ib.InsertInto(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.UpdatedMetadataTableName))
|
||||
ib.InsertInto(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.UpdatedMetadataTableName))
|
||||
ib.Cols("metric_name", "temporality", "is_monotonic", "type", "description", "unit", "created_at")
|
||||
ib.Values(
|
||||
req.MetricName,
|
||||
@@ -1016,9 +1016,9 @@ func (m *module) fetchMetricsStatsWithSamples(
|
||||
}
|
||||
}
|
||||
|
||||
start, end, distributedTsTable, localTsTable := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
|
||||
distributedSamplesTable, _ := metricstelemetryschema.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
|
||||
countExp := metricstelemetryschema.CountExpressionForSamplesTable(distributedSamplesTable)
|
||||
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
|
||||
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
|
||||
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
|
||||
|
||||
// Timeseries counts per metric
|
||||
tsSB := sqlbuilder.NewSelectBuilder()
|
||||
@@ -1026,7 +1026,7 @@ func (m *module) fetchMetricsStatsWithSamples(
|
||||
"metric_name",
|
||||
"uniq(fingerprint) AS timeseries",
|
||||
)
|
||||
tsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
|
||||
tsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
|
||||
tsSB.Where(tsSB.Between("unix_milli", start, end))
|
||||
tsSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
if filterWhereClause != nil {
|
||||
@@ -1040,7 +1040,7 @@ func (m *module) fetchMetricsStatsWithSamples(
|
||||
"metric_name",
|
||||
fmt.Sprintf("%s AS samples", countExp),
|
||||
)
|
||||
samplesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
|
||||
samplesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
|
||||
samplesSB.Where(samplesSB.Between("unix_milli", req.Start, req.End))
|
||||
samplesSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
|
||||
@@ -1053,7 +1053,7 @@ func (m *module) fetchMetricsStatsWithSamples(
|
||||
if filterWhereClause != nil {
|
||||
fingerprintSB := sqlbuilder.NewSelectBuilder()
|
||||
fingerprintSB.Select("fingerprint")
|
||||
fingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
|
||||
fingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
|
||||
fingerprintSB.Where(fingerprintSB.Between("unix_milli", start, end))
|
||||
fingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
fingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
|
||||
@@ -1064,7 +1064,7 @@ func (m *module) fetchMetricsStatsWithSamples(
|
||||
} else {
|
||||
metricNamesSB := sqlbuilder.NewSelectBuilder()
|
||||
metricNamesSB.Select("DISTINCT metric_name")
|
||||
metricNamesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
|
||||
metricNamesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
|
||||
metricNamesSB.Where(metricNamesSB.Between("unix_milli", start, end))
|
||||
metricNamesSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
|
||||
@@ -1090,18 +1090,18 @@ func (m *module) fetchMetricsStatsWithSamples(
|
||||
// the data lands either of the _reduced_last/_reduced_sum tables
|
||||
reducedLastSB := sqlbuilder.NewSelectBuilder()
|
||||
reducedLastSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
|
||||
reducedLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
|
||||
reducedLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
|
||||
reducedLastSB.Where(reducedLastSB.Between("unix_milli", req.Start, req.End))
|
||||
reducedLastSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
|
||||
reducedSumSB := sqlbuilder.NewSelectBuilder()
|
||||
reducedSumSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
|
||||
reducedSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
|
||||
reducedSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
|
||||
reducedSumSB.Where(reducedSumSB.Between("unix_milli", req.Start, req.End))
|
||||
reducedSumSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
|
||||
// separate query for reduced series counts
|
||||
reducedTsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedTsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
reducedTsSB.Where(reducedTsSB.Between("unix_milli", start, end))
|
||||
reducedTsSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
reducedTsSB.GroupBy("metric_name")
|
||||
@@ -1112,7 +1112,7 @@ func (m *module) fetchMetricsStatsWithSamples(
|
||||
// samples uses a separate cte with local table
|
||||
reducedFpSB := sqlbuilder.NewSelectBuilder()
|
||||
reducedFpSB.Select("fingerprint")
|
||||
reducedFpSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedLocalTableName))
|
||||
reducedFpSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedLocalTableName))
|
||||
reducedFpSB.Where(reducedFpSB.Between("unix_milli", start, end))
|
||||
reducedFpSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
reducedFpSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
|
||||
@@ -1220,11 +1220,11 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
|
||||
}
|
||||
}
|
||||
|
||||
start, end, distributedTsTable, _ := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
|
||||
start, end, distributedTsTable, _ := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
|
||||
|
||||
totalTSBuilder := sqlbuilder.NewSelectBuilder()
|
||||
totalTSBuilder.Select("uniq(fingerprint) AS total_time_series")
|
||||
totalTSBuilder.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
|
||||
totalTSBuilder.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
|
||||
totalTSBuilder.Where(totalTSBuilder.Between("unix_milli", start, end))
|
||||
|
||||
metricsSB := sqlbuilder.NewSelectBuilder()
|
||||
@@ -1232,7 +1232,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
|
||||
"metric_name",
|
||||
"uniq(fingerprint) AS total_value",
|
||||
)
|
||||
metricsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
|
||||
metricsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
|
||||
metricsSB.Where(metricsSB.Between("unix_milli", start, end))
|
||||
metricsSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
if filterWhereClause != nil {
|
||||
@@ -1244,7 +1244,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
|
||||
if reductionEnabled {
|
||||
reducedTotalTSBuilder := sqlbuilder.NewSelectBuilder()
|
||||
reducedTotalTSBuilder.Select("uniq(fingerprint) AS total_time_series")
|
||||
reducedTotalTSBuilder.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedTotalTSBuilder.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
reducedTotalTSBuilder.Where(reducedTotalTSBuilder.Between("unix_milli", start, end))
|
||||
|
||||
reducedMetricsSB := sqlbuilder.NewSelectBuilder()
|
||||
@@ -1252,7 +1252,7 @@ func (m *module) computeTimeseriesTreemap(ctx context.Context, orgID valuer.UUID
|
||||
"metric_name",
|
||||
"uniq(fingerprint) AS total_value",
|
||||
)
|
||||
reducedMetricsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedMetricsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
reducedMetricsSB.Where(reducedMetricsSB.Between("unix_milli", start, end))
|
||||
reducedMetricsSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
if filterWhereClause != nil {
|
||||
@@ -1338,15 +1338,15 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
}
|
||||
}
|
||||
|
||||
start, end, distributedTsTable, localTsTable := metricstelemetryschema.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
|
||||
distributedSamplesTable, _ := metricstelemetryschema.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
|
||||
countExp := metricstelemetryschema.CountExpressionForSamplesTable(distributedSamplesTable)
|
||||
start, end, distributedTsTable, localTsTable := telemetrymetrics.WhichTSTableToUse(uint64(req.Start), uint64(req.End), false, nil)
|
||||
distributedSamplesTable, _ := telemetrymetrics.WhichSamplesTableToUse(uint64(req.Start), uint64(req.End), metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, false, nil)
|
||||
countExp := telemetrymetrics.CountExpressionForSamplesTable(distributedSamplesTable)
|
||||
|
||||
candidateLimit := req.Limit + 50
|
||||
|
||||
metricCandidatesSB := sqlbuilder.NewSelectBuilder()
|
||||
metricCandidatesSB.Select("metric_name")
|
||||
metricCandidatesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedTsTable))
|
||||
metricCandidatesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedTsTable))
|
||||
metricCandidatesSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
metricCandidatesSB.Where(metricCandidatesSB.Between("unix_milli", start, end))
|
||||
if filterWhereClause != nil {
|
||||
@@ -1364,7 +1364,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
if reductionEnabled {
|
||||
reducedCandidatesSB = sqlbuilder.NewSelectBuilder()
|
||||
reducedCandidatesSB.Select("metric_name")
|
||||
reducedCandidatesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedTableName))
|
||||
reducedCandidatesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedTableName))
|
||||
reducedCandidatesSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
reducedCandidatesSB.Where(reducedCandidatesSB.Between("unix_milli", start, end))
|
||||
if filterWhereClause != nil {
|
||||
@@ -1379,7 +1379,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
|
||||
totalSamplesSB := sqlbuilder.NewSelectBuilder()
|
||||
totalSamplesSB.Select(fmt.Sprintf("%s AS total_samples", countExp))
|
||||
totalSamplesSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
|
||||
totalSamplesSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
|
||||
totalSamplesSB.Where(totalSamplesSB.Between("unix_milli", req.Start, req.End))
|
||||
|
||||
sampleCountsSB := sqlbuilder.NewSelectBuilder()
|
||||
@@ -1387,7 +1387,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
"metric_name",
|
||||
fmt.Sprintf("%s AS samples", countExp),
|
||||
)
|
||||
sampleCountsSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, distributedSamplesTable))
|
||||
sampleCountsSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, distributedSamplesTable))
|
||||
sampleCountsSB.Where(sampleCountsSB.Between("unix_milli", req.Start, req.End))
|
||||
sampleCountsSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __metric_candidates)")
|
||||
|
||||
@@ -1397,13 +1397,13 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
if reductionEnabled {
|
||||
reducedLastSB = sqlbuilder.NewSelectBuilder()
|
||||
reducedLastSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
|
||||
reducedLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
|
||||
reducedLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
|
||||
reducedLastSB.Where(reducedLastSB.Between("unix_milli", req.Start, req.End))
|
||||
reducedLastSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __reduced_metric_candidates)")
|
||||
|
||||
reducedSumSB = sqlbuilder.NewSelectBuilder()
|
||||
reducedSumSB.Select("metric_name", "uniq(reduced_fingerprint, unix_milli) AS cnt")
|
||||
reducedSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
|
||||
reducedSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
|
||||
reducedSumSB.Where(reducedSumSB.Between("unix_milli", req.Start, req.End))
|
||||
reducedSumSB.Where("metric_name GLOBAL IN (SELECT metric_name FROM __reduced_metric_candidates)")
|
||||
}
|
||||
@@ -1411,7 +1411,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
if filterWhereClause != nil {
|
||||
fingerprintSB := sqlbuilder.NewSelectBuilder()
|
||||
fingerprintSB.Select("fingerprint")
|
||||
fingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, localTsTable))
|
||||
fingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTsTable))
|
||||
fingerprintSB.Where(fingerprintSB.Between("unix_milli", start, end))
|
||||
fingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
fingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
|
||||
@@ -1423,7 +1423,7 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
if reductionEnabled {
|
||||
reducedFingerprintSB := sqlbuilder.NewSelectBuilder()
|
||||
reducedFingerprintSB.Select("fingerprint")
|
||||
reducedFingerprintSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4ReducedLocalTableName))
|
||||
reducedFingerprintSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4ReducedLocalTableName))
|
||||
reducedFingerprintSB.Where(reducedFingerprintSB.Between("unix_milli", start, end))
|
||||
reducedFingerprintSB.Where("NOT startsWith(metric_name, 'signoz')")
|
||||
reducedFingerprintSB.AddWhereClause(sqlbuilder.CopyWhereClause(filterWhereClause))
|
||||
@@ -1457,12 +1457,12 @@ func (m *module) computeSamplesTreemap(ctx context.Context, orgID valuer.UUID, r
|
||||
// Grand total includes reduced sample volume so percentages stay consistent.
|
||||
reducedTotalLastSB := sqlbuilder.NewSelectBuilder()
|
||||
reducedTotalLastSB.Select("uniq(reduced_fingerprint, unix_milli) AS cnt")
|
||||
reducedTotalLastSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedLastTableName))
|
||||
reducedTotalLastSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedLastTableName))
|
||||
reducedTotalLastSB.Where(reducedTotalLastSB.Between("unix_milli", req.Start, req.End))
|
||||
|
||||
reducedTotalSumSB := sqlbuilder.NewSelectBuilder()
|
||||
reducedTotalSumSB.Select("uniq(reduced_fingerprint, unix_milli) AS cnt")
|
||||
reducedTotalSumSB.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4ReducedSumTableName))
|
||||
reducedTotalSumSB.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4ReducedSumTableName))
|
||||
reducedTotalSumSB.Where(reducedTotalSumSB.Between("unix_milli", req.Start, req.End))
|
||||
|
||||
reducedTotalSamplesSB := sqlbuilder.NewSelectBuilder()
|
||||
@@ -1552,7 +1552,7 @@ func (m *module) getMetricDataPoints(ctx context.Context, metricName string) (ui
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("sum(count) AS data_points")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4Agg30mTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4Agg30mTableName))
|
||||
sb.Where(sb.E("metric_name", metricName))
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
@@ -1574,7 +1574,7 @@ func (m *module) getMetricLastReceived(ctx context.Context, metricName string) (
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("MAX(last_reported_unix_milli) AS last_received_time")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(sb.E("metric_name", metricName))
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
|
||||
@@ -1599,7 +1599,7 @@ func (m *module) getTotalTimeSeriesForMetricName(ctx context.Context, metricName
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("uniq(fingerprint) AS time_series_count")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV41weekTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV41weekTableName))
|
||||
sb.Where(sb.E("metric_name", metricName))
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
@@ -1623,7 +1623,7 @@ func (m *module) getActiveTimeSeriesForMetricName(ctx context.Context, metricNam
|
||||
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
sb.Select("uniq(fingerprint) AS active_time_series")
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.TimeseriesV4TableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.TimeseriesV4TableName))
|
||||
sb.Where(sb.E("metric_name", metricName))
|
||||
sb.Where(sb.GTE("unix_milli", milli))
|
||||
|
||||
@@ -1649,7 +1649,7 @@ func (m *module) fetchMetricAttributes(ctx context.Context, metricName string, s
|
||||
"groupUniqArray(1000)(attr_string_value) AS values",
|
||||
"uniq(attr_string_value) AS valueCount",
|
||||
)
|
||||
sb.From(fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.AttributesMetadataTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.AttributesMetadataTableName))
|
||||
sb.Where(sb.E("metric_name", metricName))
|
||||
sb.Where("NOT startsWith(attr_name, '__')")
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
schemamigrator "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/modules/promote"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
|
||||
@@ -52,7 +52,7 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
|
||||
|
||||
response := []promotetypes.PromotePath{}
|
||||
for _, path := range promotedPaths {
|
||||
fullPath := logstelemetryschema.BodyPromotedColumnPrefix + path
|
||||
fullPath := telemetrylogs.BodyPromotedColumnPrefix + path
|
||||
path = telemetrytypes.BodyJSONStringSearchPrefix + path
|
||||
item := promotetypes.PromotePath{
|
||||
Path: path,
|
||||
@@ -68,7 +68,7 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
|
||||
|
||||
// add the paths that are not promoted but have indexes
|
||||
for path, indexes := range aggr {
|
||||
path := strings.TrimPrefix(path, logstelemetryschema.BodyV2ColumnPrefix)
|
||||
path := strings.TrimPrefix(path, telemetrylogs.BodyV2ColumnPrefix)
|
||||
path = telemetrytypes.BodyJSONStringSearchPrefix + path
|
||||
response = append(response, promotetypes.PromotePath{
|
||||
Path: path,
|
||||
@@ -108,8 +108,8 @@ func (m *module) createIndexes(ctx context.Context, indexes []schemamigrator.Ind
|
||||
|
||||
for _, index := range indexes {
|
||||
alterStmt := schemamigrator.AlterTableAddIndex{
|
||||
Database: logstelemetryschema.DBName,
|
||||
Table: logstelemetryschema.LogsV2LocalTableName,
|
||||
Database: telemetrylogs.DBName,
|
||||
Table: telemetrylogs.LogsV2LocalTableName,
|
||||
Index: index,
|
||||
}
|
||||
op := alterStmt.OnCluster(m.telemetryStore.Cluster())
|
||||
@@ -153,10 +153,10 @@ func (m *module) PromoteAndIndexPaths(
|
||||
}
|
||||
}
|
||||
if len(it.Indexes) > 0 {
|
||||
parentColumn := logstelemetryschema.LogsV2BodyV2Column
|
||||
parentColumn := telemetrylogs.LogsV2BodyV2Column
|
||||
// if the path is already promoted or is being promoted, add it to the promoted column
|
||||
if _, promoted := existingPromotedPaths[it.Path]; promoted || it.Promote {
|
||||
parentColumn = logstelemetryschema.LogsV2BodyPromotedColumn
|
||||
parentColumn = telemetrylogs.LogsV2BodyPromotedColumn
|
||||
}
|
||||
|
||||
for _, index := range it.Indexes {
|
||||
|
||||
@@ -11,8 +11,8 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/modules/services"
|
||||
"github.com/SigNoz/signoz/pkg/querier"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
@@ -39,7 +39,7 @@ func (m *module) FetchTopLevelOperations(ctx context.Context, start time.Time, s
|
||||
ctx = m.withServicesContext(ctx, "FetchTopLevelOperations")
|
||||
|
||||
db := m.TelemetryStore.ClickhouseDB()
|
||||
query := fmt.Sprintf("SELECT name, serviceName, max(time) as ts FROM %s.%s WHERE time >= @start", tracestelemetryschema.DBName, tracestelemetryschema.TopLevelOperationsTableName)
|
||||
query := fmt.Sprintf("SELECT name, serviceName, max(time) as ts FROM %s.%s WHERE time >= @start", telemetrytraces.DBName, telemetrytraces.TopLevelOperationsTableName)
|
||||
args := []any{clickhouse.Named("start", start)}
|
||||
if len(services) > 0 {
|
||||
query += " AND serviceName IN @services"
|
||||
|
||||
@@ -22,13 +22,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/variables"
|
||||
)
|
||||
|
||||
type Handler interface {
|
||||
QueryRange(rw http.ResponseWriter, req *http.Request)
|
||||
QueryRangePreview(rw http.ResponseWriter, req *http.Request)
|
||||
QueryRawStream(rw http.ResponseWriter, req *http.Request)
|
||||
ReplaceVariables(rw http.ResponseWriter, req *http.Request)
|
||||
}
|
||||
|
||||
type handler struct {
|
||||
set factory.ProviderSettings
|
||||
analytics analytics.Analytics
|
||||
|
||||
@@ -15,14 +15,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
// BucketCache is the interface for bucket-based caching.
|
||||
type BucketCache interface {
|
||||
// cached portion + list of gaps to fetch
|
||||
GetMissRanges(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step) (cached *qbtypes.Result, missing []*qbtypes.TimeRange)
|
||||
// store fresh buckets for future hits
|
||||
Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step, fresh *qbtypes.Result)
|
||||
}
|
||||
|
||||
// bucketCache implements the BucketCache interface.
|
||||
type bucketCache struct {
|
||||
cache cache.Cache
|
||||
|
||||
@@ -11,8 +11,8 @@ import (
|
||||
|
||||
"github.com/ClickHouse/clickhouse-go/v2"
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
@@ -277,12 +277,12 @@ func (q *builderQuery[T]) narrowWindowByTraceID(ctx context.Context, fromMS, toM
|
||||
return fromMS, toMS, true, ""
|
||||
}
|
||||
|
||||
traceIDs, found := tracestelemetryschema.ExtractTraceIDsFromFilter(q.spec.Filter.Expression)
|
||||
traceIDs, found := telemetrytraces.ExtractTraceIDsFromFilter(q.spec.Filter.Expression)
|
||||
if !found || len(traceIDs) == 0 {
|
||||
return fromMS, toMS, true, ""
|
||||
}
|
||||
|
||||
finder := tracestelemetryschema.NewTraceTimeRangeFinder(q.telemetryStore)
|
||||
finder := telemetrytraces.NewTraceTimeRangeFinder(q.telemetryStore)
|
||||
traceStart, traceEnd, exists, err := finder.GetTraceTimeRangeMulti(ctx, traceIDs)
|
||||
if err != nil {
|
||||
return fromMS, toMS, true, ""
|
||||
|
||||
@@ -100,20 +100,9 @@ func (q *chSQLQuery) renderVars(query string, vars map[string]qbtypes.VariableIt
|
||||
return newQuery.String(), nil
|
||||
}
|
||||
|
||||
func (q *chSQLQuery) render(ctx context.Context) (string, error) {
|
||||
rendered, err := q.renderVars(q.query.Query, q.vars, q.fromMS, q.toMS)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
querybuilder.LogIfStatementIsNotValid(ctx, q.logger, rendered)
|
||||
|
||||
return rendered, nil
|
||||
}
|
||||
|
||||
// Statement renders the SQL without executing it, for the preview path.
|
||||
func (q *chSQLQuery) Statement(ctx context.Context) (*qbtypes.Statement, error) {
|
||||
rendered, err := q.render(ctx)
|
||||
func (q *chSQLQuery) Statement(_ context.Context) (*qbtypes.Statement, error) {
|
||||
rendered, err := q.renderVars(q.query.Query, q.vars, q.fromMS, q.toMS)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -135,7 +124,7 @@ func (q *chSQLQuery) Execute(ctx context.Context) (*qbtypes.Result, error) {
|
||||
elapsed += p.Elapsed
|
||||
}))
|
||||
|
||||
query, err := q.render(ctx)
|
||||
query, err := q.renderVars(q.query.Query, q.vars, q.fromMS, q.toMS)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -146,11 +135,11 @@ func (q *chSQLQuery) Execute(ctx context.Context) (*qbtypes.Result, error) {
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
// TODO: map the errors from ClickHouse to our error types
|
||||
payload, err := consume(rows, q.kind, nil, qbtypes.Step{}, q.query.Name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &qbtypes.Result{
|
||||
Type: q.kind,
|
||||
Value: payload,
|
||||
|
||||
@@ -6,18 +6,18 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
|
||||
stats := make(map[string]any)
|
||||
|
||||
tracesTable := fmt.Sprintf("%s.%s", tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName)
|
||||
logsTable := fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName)
|
||||
metricsTable := fmt.Sprintf("%s.%s", metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName)
|
||||
tracesTable := fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.SpanIndexV3TableName)
|
||||
logsTable := fmt.Sprintf("%s.%s", telemetrylogs.DBName, telemetrylogs.LogsV2TableName)
|
||||
metricsTable := fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4TableName)
|
||||
|
||||
var (
|
||||
traces uint64
|
||||
|
||||
@@ -9,6 +9,12 @@ import (
|
||||
|
||||
const DefaultMaxConcurrentQueries = 8
|
||||
|
||||
type SkipResourceFingerprint struct {
|
||||
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
|
||||
// If count of fingerprint is above threshold, skip the fingerprint subquery and filter on main table instead.
|
||||
Threshold uint64 `yaml:"threshold" mapstructure:"threshold"`
|
||||
}
|
||||
|
||||
// Config represents the configuration for the querier.
|
||||
type Config struct {
|
||||
// CacheTTL is the TTL for cached query results
|
||||
@@ -17,6 +23,8 @@ type Config struct {
|
||||
FluxInterval time.Duration `yaml:"flux_interval" mapstructure:"flux_interval"`
|
||||
// MaxConcurrentQueries is the maximum number of concurrent queries for missing ranges
|
||||
MaxConcurrentQueries int `yaml:"max_concurrent_queries" mapstructure:"max_concurrent_queries"`
|
||||
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
|
||||
SkipResourceFingerprint SkipResourceFingerprint `yaml:"skip_resource_fingerprint" mapstructure:"skip_resource_fingerprint"`
|
||||
// LogTraceIDWindowPadding is the padding added to narrowed down timerange from trace summary to logs with trace_id filter.
|
||||
LogTraceIDWindowPadding time.Duration `yaml:"log_trace_id_window_padding" mapstructure:"log_trace_id_window_padding"`
|
||||
}
|
||||
@@ -31,7 +39,11 @@ func newConfig() factory.Config {
|
||||
// Default values
|
||||
CacheTTL: 168 * time.Hour,
|
||||
FluxInterval: 5 * time.Minute,
|
||||
MaxConcurrentQueries: DefaultMaxConcurrentQueries,
|
||||
MaxConcurrentQueries: DefaultMaxConcurrentQueries,
|
||||
SkipResourceFingerprint: SkipResourceFingerprint{
|
||||
Enabled: false,
|
||||
Threshold: 100000,
|
||||
},
|
||||
LogTraceIDWindowPadding: 5 * time.Minute,
|
||||
}
|
||||
}
|
||||
@@ -47,6 +59,9 @@ func (c Config) Validate() error {
|
||||
if c.MaxConcurrentQueries <= 0 {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "max_concurrent_queries must be positive, got %v", c.MaxConcurrentQueries)
|
||||
}
|
||||
if c.SkipResourceFingerprint.Enabled && c.SkipResourceFingerprint.Threshold == 0 {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "skip_resource_fingerprint.threshold must be > 0 when enabled")
|
||||
}
|
||||
if c.LogTraceIDWindowPadding < 0 {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "log_trace_id_window_padding must not be negative, got %v", c.LogTraceIDWindowPadding)
|
||||
}
|
||||
|
||||
35
pkg/querier/interfaces.go
Normal file
35
pkg/querier/interfaces.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package querier
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
// Querier interface defines the contract for querying data.
|
||||
type Querier interface {
|
||||
QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
|
||||
QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
|
||||
statsreporter.StatsCollector
|
||||
// QueryRangePreview validates and renders the queries without executing them.
|
||||
QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
|
||||
}
|
||||
|
||||
// BucketCache is the interface for bucket-based caching.
|
||||
type BucketCache interface {
|
||||
// cached portion + list of gaps to fetch
|
||||
GetMissRanges(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step) (cached *qbtypes.Result, missing []*qbtypes.TimeRange)
|
||||
// store fresh buckets for future hits
|
||||
Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Query, step qbtypes.Step, fresh *qbtypes.Result)
|
||||
}
|
||||
|
||||
type Handler interface {
|
||||
QueryRange(rw http.ResponseWriter, req *http.Request)
|
||||
// QueryRangePreview is the dry-run endpoint: validate and render without executing.
|
||||
QueryRangePreview(rw http.ResponseWriter, req *http.Request)
|
||||
QueryRawStream(rw http.ResponseWriter, req *http.Request)
|
||||
ReplaceVariables(rw http.ResponseWriter, req *http.Request)
|
||||
}
|
||||
@@ -21,8 +21,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/prometheus"
|
||||
"github.com/SigNoz/signoz/pkg/query-service/utils"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
|
||||
@@ -37,15 +35,6 @@ var (
|
||||
intervalWarn = "Query %s is requesting aggregation interval %v seconds, which is smaller than the minimum allowed interval of %v seconds for selected time range. Using the minimum instead"
|
||||
)
|
||||
|
||||
// Querier interface defines the contract for querying data.
|
||||
type Querier interface {
|
||||
QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
|
||||
QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
|
||||
statsreporter.StatsCollector
|
||||
// QueryRangePreview validates and renders the queries without executing them.
|
||||
QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
|
||||
}
|
||||
|
||||
type querier struct {
|
||||
logger *slog.Logger
|
||||
fl flagger.Flagger
|
||||
@@ -71,7 +60,12 @@ func New(
|
||||
telemetryStore telemetrystore.TelemetryStore,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
promEngine prometheus.Prometheus,
|
||||
builders *statementbuilder.Builders,
|
||||
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
|
||||
logStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
|
||||
auditStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation],
|
||||
metricStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
|
||||
meterStmtBuilder qbtypes.StatementBuilder[qbtypes.MetricAggregation],
|
||||
traceOperatorStmtBuilder qbtypes.TraceOperatorStatementBuilder,
|
||||
bucketCache BucketCache,
|
||||
flagger flagger.Flagger,
|
||||
logTraceIDWindowPadding time.Duration,
|
||||
@@ -81,21 +75,18 @@ func New(
|
||||
if maxConcurrentQueries <= 0 {
|
||||
maxConcurrentQueries = DefaultMaxConcurrentQueries
|
||||
}
|
||||
if builders == nil {
|
||||
builders = &statementbuilder.Builders{}
|
||||
}
|
||||
return &querier{
|
||||
logger: querierSettings.Logger(),
|
||||
fl: flagger,
|
||||
telemetryStore: telemetryStore,
|
||||
metadataStore: metadataStore,
|
||||
promEngine: promEngine,
|
||||
traceStmtBuilder: builders.Trace,
|
||||
logStmtBuilder: builders.Log,
|
||||
auditStmtBuilder: builders.Audit,
|
||||
metricStmtBuilder: builders.Metric,
|
||||
meterStmtBuilder: builders.Meter,
|
||||
traceOperatorStmtBuilder: builders.TraceOperator,
|
||||
traceStmtBuilder: traceStmtBuilder,
|
||||
logStmtBuilder: logStmtBuilder,
|
||||
auditStmtBuilder: auditStmtBuilder,
|
||||
metricStmtBuilder: metricStmtBuilder,
|
||||
meterStmtBuilder: meterStmtBuilder,
|
||||
traceOperatorStmtBuilder: traceOperatorStmtBuilder,
|
||||
bucketCache: bucketCache,
|
||||
liveDataRefresh: 5 * time.Second,
|
||||
builderConfig: builderConfig{
|
||||
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
|
||||
"github.com/SigNoz/signoz/pkg/types/metrictypes"
|
||||
@@ -49,7 +48,12 @@ func TestQueryRange_MetricTypeMissing(t *testing.T) {
|
||||
nil, // telemetryStore
|
||||
metadataStore,
|
||||
nil, // prometheus
|
||||
nil, // builders
|
||||
nil, // traceStmtBuilder
|
||||
nil, // logStmtBuilder
|
||||
nil, // auditStmtBuilder
|
||||
nil, // metricStmtBuilder
|
||||
nil, // meterStmtBuilder
|
||||
nil, // traceOperatorStmtBuilder
|
||||
nil, // bucketCache
|
||||
flaggertest.New(t), // flagger
|
||||
0, // logTraceIDWindowPadding
|
||||
@@ -115,12 +119,17 @@ func TestQueryRange_MetricTypeFromStore(t *testing.T) {
|
||||
providerSettings,
|
||||
telemetryStore,
|
||||
metadataStore,
|
||||
nil, // prometheus
|
||||
&statementbuilder.Builders{Metric: &mockMetricStmtBuilder{}},
|
||||
nil, // bucketCache
|
||||
flaggertest.New(t), // flagger
|
||||
0, // logTraceIDWindowPadding
|
||||
0, // maxConcurrentQueries
|
||||
nil, // prometheus
|
||||
nil, // traceStmtBuilder
|
||||
nil, // logStmtBuilder
|
||||
nil, // auditStmtBuilder
|
||||
&mockMetricStmtBuilder{}, // metricStmtBuilder
|
||||
nil, // meterStmtBuilder
|
||||
nil, // traceOperatorStmtBuilder
|
||||
nil, // bucketCache
|
||||
flaggertest.New(t), // flagger
|
||||
0, // logTraceIDWindowPadding
|
||||
0, // maxConcurrentQueries
|
||||
)
|
||||
|
||||
req := &qbtypes.QueryRangeRequest{
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
package queriertest
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/querier"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
// MockQuerier implements querier.Querier for testing. Each behavior can be
|
||||
// overridden via a function field; unset fields return empty, non-error values.
|
||||
type MockQuerier struct {
|
||||
QueryRangeFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
|
||||
QueryRawStreamFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
|
||||
QueryRangePreviewFunc func(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error)
|
||||
CollectFunc func(ctx context.Context, orgID valuer.UUID) (map[string]any, error)
|
||||
}
|
||||
|
||||
var _ querier.Querier = (*MockQuerier)(nil)
|
||||
|
||||
// NewMockQuerier creates a new MockQuerier.
|
||||
func NewMockQuerier() *MockQuerier {
|
||||
return &MockQuerier{}
|
||||
}
|
||||
|
||||
func (m *MockQuerier) QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error) {
|
||||
if m.QueryRangeFunc != nil {
|
||||
return m.QueryRangeFunc(ctx, orgID, req)
|
||||
}
|
||||
return &qbtypes.QueryRangeResponse{}, nil
|
||||
}
|
||||
|
||||
func (m *MockQuerier) QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream) {
|
||||
if m.QueryRawStreamFunc != nil {
|
||||
m.QueryRawStreamFunc(ctx, orgID, req, client)
|
||||
}
|
||||
}
|
||||
|
||||
func (m *MockQuerier) QueryRangePreview(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, opts qbtypes.QueryRangePreviewOptions) (*qbtypes.QueryRangePreviewResponse, error) {
|
||||
if m.QueryRangePreviewFunc != nil {
|
||||
return m.QueryRangePreviewFunc(ctx, orgID, req, opts)
|
||||
}
|
||||
return &qbtypes.QueryRangePreviewResponse{}, nil
|
||||
}
|
||||
|
||||
func (m *MockQuerier) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
|
||||
if m.CollectFunc != nil {
|
||||
return m.CollectFunc(ctx, orgID)
|
||||
}
|
||||
return map[string]any{}, nil
|
||||
}
|
||||
@@ -3,44 +3,192 @@ package signozquerier
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/cache"
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/prometheus"
|
||||
"github.com/SigNoz/signoz/pkg/querier"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryaudit"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymeter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
)
|
||||
|
||||
// NewFactory creates a new factory for the signoz querier provider. The query
|
||||
// stack (metadata store, statement builders, bucket cache) is assembled once in
|
||||
// signoz.go and injected here.
|
||||
// NewFactory creates a new factory for the signoz querier provider.
|
||||
func NewFactory(
|
||||
telemetryStore telemetrystore.TelemetryStore,
|
||||
prometheus prometheus.Prometheus,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
builders *statementbuilder.Builders,
|
||||
bucketCache querier.BucketCache,
|
||||
cache cache.Cache,
|
||||
flagger flagger.Flagger,
|
||||
) factory.ProviderFactory[querier.Querier, querier.Config] {
|
||||
return factory.NewProviderFactory(
|
||||
factory.MustNewName("signoz"),
|
||||
func(
|
||||
_ context.Context,
|
||||
ctx context.Context,
|
||||
settings factory.ProviderSettings,
|
||||
cfg querier.Config,
|
||||
) (querier.Querier, error) {
|
||||
return querier.New(
|
||||
settings,
|
||||
telemetryStore,
|
||||
metadataStore,
|
||||
prometheus,
|
||||
builders,
|
||||
bucketCache,
|
||||
flagger,
|
||||
cfg.LogTraceIDWindowPadding,
|
||||
cfg.MaxConcurrentQueries,
|
||||
), nil
|
||||
return newProvider(ctx, settings, cfg, telemetryStore, prometheus, cache, flagger)
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
func newProvider(
|
||||
_ context.Context,
|
||||
settings factory.ProviderSettings,
|
||||
cfg querier.Config,
|
||||
telemetryStore telemetrystore.TelemetryStore,
|
||||
prometheus prometheus.Prometheus,
|
||||
cache cache.Cache,
|
||||
flagger flagger.Flagger,
|
||||
) (querier.Querier, error) {
|
||||
|
||||
// Create telemetry metadata store
|
||||
telemetryMetadataStore := telemetrymetadata.NewTelemetryMetaStore(
|
||||
settings,
|
||||
telemetryStore,
|
||||
telemetrytraces.DBName,
|
||||
telemetrytraces.TagAttributesV2TableName,
|
||||
telemetrytraces.SpanAttributesKeysTblName,
|
||||
telemetrytraces.SpanIndexV3TableName,
|
||||
telemetrymetrics.DBName,
|
||||
telemetrymetrics.AttributesMetadataTableName,
|
||||
telemetrymeter.DBName,
|
||||
telemetrymeter.SamplesAgg1dTableName,
|
||||
telemetrylogs.DBName,
|
||||
telemetrylogs.LogsV2TableName,
|
||||
telemetrylogs.TagAttributesV2TableName,
|
||||
telemetrylogs.LogAttributeKeysTblName,
|
||||
telemetrylogs.LogResourceKeysTblName,
|
||||
telemetryaudit.DBName,
|
||||
telemetryaudit.AuditLogsTableName,
|
||||
telemetryaudit.TagAttributesTableName,
|
||||
telemetryaudit.LogAttributeKeysTblName,
|
||||
telemetryaudit.LogResourceKeysTblName,
|
||||
telemetrymetadata.DBName,
|
||||
telemetrymetadata.AttributesMetadataTableName,
|
||||
telemetrymetadata.ColumnEvolutionMetadataTableName,
|
||||
flagger,
|
||||
)
|
||||
|
||||
// Create trace statement builder
|
||||
traceFieldMapper := telemetrytraces.NewFieldMapper()
|
||||
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
|
||||
|
||||
traceAggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, traceFieldMapper, traceConditionBuilder, flagger)
|
||||
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
|
||||
settings,
|
||||
telemetryMetadataStore,
|
||||
traceFieldMapper,
|
||||
traceConditionBuilder,
|
||||
traceAggExprRewriter,
|
||||
telemetryStore,
|
||||
flagger,
|
||||
cfg.SkipResourceFingerprint.Enabled,
|
||||
cfg.SkipResourceFingerprint.Threshold,
|
||||
)
|
||||
|
||||
// Create trace operator statement builder
|
||||
traceOperatorStmtBuilder := telemetrytraces.NewTraceOperatorStatementBuilder(
|
||||
settings,
|
||||
telemetryMetadataStore,
|
||||
traceFieldMapper,
|
||||
traceConditionBuilder,
|
||||
traceStmtBuilder,
|
||||
traceAggExprRewriter,
|
||||
flagger,
|
||||
)
|
||||
|
||||
// Create log statement builder
|
||||
logFieldMapper := telemetrylogs.NewFieldMapper(flagger)
|
||||
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, flagger)
|
||||
logAggExprRewriter := querybuilder.NewAggExprRewriter(
|
||||
settings,
|
||||
telemetrylogs.DefaultFullTextColumn,
|
||||
logFieldMapper,
|
||||
logConditionBuilder,
|
||||
flagger,
|
||||
)
|
||||
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
|
||||
settings,
|
||||
telemetryMetadataStore,
|
||||
logFieldMapper,
|
||||
logConditionBuilder,
|
||||
logAggExprRewriter,
|
||||
telemetrylogs.DefaultFullTextColumn,
|
||||
flagger,
|
||||
telemetryStore,
|
||||
cfg.SkipResourceFingerprint.Enabled,
|
||||
cfg.SkipResourceFingerprint.Threshold,
|
||||
)
|
||||
|
||||
// Create audit statement builder
|
||||
auditFieldMapper := telemetryaudit.NewFieldMapper()
|
||||
auditConditionBuilder := telemetryaudit.NewConditionBuilder(auditFieldMapper)
|
||||
auditAggExprRewriter := querybuilder.NewAggExprRewriter(
|
||||
settings,
|
||||
telemetryaudit.DefaultFullTextColumn,
|
||||
auditFieldMapper,
|
||||
auditConditionBuilder,
|
||||
flagger,
|
||||
)
|
||||
auditStmtBuilder := telemetryaudit.NewAuditQueryStatementBuilder(
|
||||
settings,
|
||||
telemetryMetadataStore,
|
||||
auditFieldMapper,
|
||||
auditConditionBuilder,
|
||||
auditAggExprRewriter,
|
||||
telemetryaudit.DefaultFullTextColumn,
|
||||
flagger,
|
||||
)
|
||||
|
||||
// Create metric statement builder
|
||||
metricFieldMapper := telemetrymetrics.NewFieldMapper()
|
||||
metricConditionBuilder := telemetrymetrics.NewConditionBuilder(metricFieldMapper)
|
||||
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(
|
||||
settings,
|
||||
telemetryMetadataStore,
|
||||
metricFieldMapper,
|
||||
metricConditionBuilder,
|
||||
flagger,
|
||||
)
|
||||
|
||||
// Create meter statement builder
|
||||
meterStmtBuilder := telemetrymeter.NewMeterQueryStatementBuilder(
|
||||
settings,
|
||||
telemetryMetadataStore,
|
||||
metricFieldMapper,
|
||||
metricConditionBuilder,
|
||||
metricStmtBuilder,
|
||||
)
|
||||
|
||||
// Create bucket cache
|
||||
bucketCache := querier.NewBucketCache(
|
||||
settings,
|
||||
cache,
|
||||
cfg.CacheTTL,
|
||||
cfg.FluxInterval,
|
||||
)
|
||||
|
||||
// Create and return the querier
|
||||
return querier.New(
|
||||
settings,
|
||||
telemetryStore,
|
||||
telemetryMetadataStore,
|
||||
prometheus,
|
||||
traceStmtBuilder,
|
||||
logStmtBuilder,
|
||||
auditStmtBuilder,
|
||||
metricStmtBuilder,
|
||||
meterStmtBuilder,
|
||||
traceOperatorStmtBuilder,
|
||||
bucketCache,
|
||||
flagger,
|
||||
cfg.LogTraceIDWindowPadding,
|
||||
cfg.MaxConcurrentQueries,
|
||||
), nil
|
||||
}
|
||||
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/modules/thirdpartyapi"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/queryparser"
|
||||
|
||||
"log/slog"
|
||||
@@ -1093,8 +1092,6 @@ func (aH *APIHandler) queryDashboardVarsV2(w http.ResponseWriter, r *http.Reques
|
||||
return
|
||||
}
|
||||
|
||||
querybuilder.LogIfStatementIsNotValid(r.Context(), aH.logger, query)
|
||||
|
||||
dashboardVars, err := aH.reader.QueryDashboardVars(r.Context(), query)
|
||||
if err != nil {
|
||||
RespondError(w, &model.ApiError{Typ: model.ErrorBadData, Err: err}, nil)
|
||||
|
||||
@@ -18,8 +18,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/querier/signozquerier"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstoretest"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -107,11 +105,7 @@ func NewTestManager(t *testing.T, testOpts *TestManagerOptions) *Manager {
|
||||
}
|
||||
|
||||
// Create querier with test values
|
||||
metadataStore := telemetrymetadata.NewTelemetryMetaStore(providerSettings, telemetryStore, flagger)
|
||||
builders, err := statementbuilder.NewFactory(telemetryStore, metadataStore, flagger).New(context.Background(), providerSettings, statementbuilder.Config{})
|
||||
require.NoError(t, err)
|
||||
bucketCache := querier.NewBucketCache(providerSettings, cache, 0, 0)
|
||||
providerFactory := signozquerier.NewFactory(telemetryStore, prometheus, metadataStore, builders, bucketCache, flagger)
|
||||
providerFactory := signozquerier.NewFactory(telemetryStore, prometheus, cache, flagger)
|
||||
mockQuerier, err := providerFactory.New(context.Background(), providerSettings, querier.Config{})
|
||||
require.NoError(t, err)
|
||||
|
||||
|
||||
@@ -8,11 +8,11 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/querier"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -24,7 +24,7 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
|
||||
providerSettings := instrumentationtest.New().ToProviderSettings()
|
||||
metadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
|
||||
fl, err := flagger.New(
|
||||
flagger, err := flagger.New(
|
||||
context.Background(),
|
||||
instrumentationtest.New().ToProviderSettings(),
|
||||
flagger.Config{},
|
||||
@@ -32,16 +32,29 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
|
||||
)
|
||||
require.NoError(t, err)
|
||||
|
||||
metricStmtBuilder := metricsstatementbuilder.New(providerSettings, metadataStore, fl)
|
||||
metricFieldMapper := telemetrymetrics.NewFieldMapper()
|
||||
metricConditionBuilder := telemetrymetrics.NewConditionBuilder(metricFieldMapper)
|
||||
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(
|
||||
providerSettings,
|
||||
metadataStore,
|
||||
metricFieldMapper,
|
||||
metricConditionBuilder,
|
||||
flagger,
|
||||
)
|
||||
|
||||
return querier.New(
|
||||
providerSettings,
|
||||
telemetryStore,
|
||||
metadataStore,
|
||||
nil, // prometheus
|
||||
&statementbuilder.Builders{Metric: metricStmtBuilder},
|
||||
nil, // traceStmtBuilder
|
||||
nil, // logStmtBuilder
|
||||
nil, // auditStmtBuilder
|
||||
metricStmtBuilder,
|
||||
nil, // meterStmtBuilder
|
||||
nil, // traceOperatorStmtBuilder
|
||||
nil, // bucketCache
|
||||
fl,
|
||||
flagger,
|
||||
0,
|
||||
0, // maxConcurrentQueries (0 means default)
|
||||
), metadataStore
|
||||
@@ -60,15 +73,40 @@ func prepareQuerierForLogs(t *testing.T, telemetryStore telemetrystore.Telemetry
|
||||
metadataStore.KeysMap = keysMap
|
||||
|
||||
fl := flaggertest.New(t)
|
||||
logStmtBuilder := logsstatementbuilder.New(providerSettings, metadataStore, nil, fl, false, 100000)
|
||||
logFieldMapper := telemetrylogs.NewFieldMapper(fl)
|
||||
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, fl)
|
||||
logAggExprRewriter := querybuilder.NewAggExprRewriter(
|
||||
providerSettings,
|
||||
telemetrylogs.DefaultFullTextColumn,
|
||||
logFieldMapper,
|
||||
logConditionBuilder,
|
||||
fl,
|
||||
)
|
||||
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
|
||||
providerSettings,
|
||||
metadataStore,
|
||||
logFieldMapper,
|
||||
logConditionBuilder,
|
||||
logAggExprRewriter,
|
||||
telemetrylogs.DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
100000,
|
||||
)
|
||||
|
||||
return querier.New(
|
||||
providerSettings,
|
||||
telemetryStore,
|
||||
metadataStore,
|
||||
nil, // prometheus
|
||||
&statementbuilder.Builders{Log: logStmtBuilder},
|
||||
nil, // bucketCache
|
||||
nil, // prometheus
|
||||
nil, // traceStmtBuilder
|
||||
logStmtBuilder, // logStmtBuilder
|
||||
nil, // auditStmtBuilder
|
||||
nil, // metricStmtBuilder
|
||||
nil, // meterStmtBuilder
|
||||
nil, // traceOperatorStmtBuilder
|
||||
nil, // bucketCache
|
||||
fl,
|
||||
5*time.Minute, // logTraceIDWindowPadding
|
||||
0, // maxConcurrentQueries (0 means default)
|
||||
@@ -88,16 +126,36 @@ func prepareQuerierForTraces(t *testing.T, telemetryStore telemetrystore.Telemet
|
||||
}
|
||||
metadataStore.KeysMap = keysMap
|
||||
|
||||
// Create trace statement builder
|
||||
traceFieldMapper := telemetrytraces.NewFieldMapper()
|
||||
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
|
||||
|
||||
fl := flaggertest.New(t)
|
||||
traceStmtBuilder, _ := tracesstatementbuilder.New(providerSettings, metadataStore, telemetryStore, fl, false, 100000)
|
||||
traceAggExprRewriter := querybuilder.NewAggExprRewriter(providerSettings, nil, traceFieldMapper, traceConditionBuilder, fl)
|
||||
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
|
||||
providerSettings,
|
||||
metadataStore,
|
||||
traceFieldMapper,
|
||||
traceConditionBuilder,
|
||||
traceAggExprRewriter,
|
||||
telemetryStore,
|
||||
fl,
|
||||
false,
|
||||
100000,
|
||||
)
|
||||
|
||||
return querier.New(
|
||||
providerSettings,
|
||||
telemetryStore,
|
||||
metadataStore,
|
||||
nil, // prometheus
|
||||
&statementbuilder.Builders{Trace: traceStmtBuilder},
|
||||
nil, // bucketCache
|
||||
nil, // prometheus
|
||||
traceStmtBuilder, // traceStmtBuilder
|
||||
nil, // logStmtBuilder
|
||||
nil, // auditStmtBuilder
|
||||
nil, // metricStmtBuilder
|
||||
nil, // meterStmtBuilder
|
||||
nil, // traceOperatorStmtBuilder
|
||||
nil, // bucketCache
|
||||
fl,
|
||||
0,
|
||||
0, // maxConcurrentQueries (0 means default)
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
package querybuilder
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"strings"
|
||||
|
||||
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
)
|
||||
|
||||
// internalDatabases hold server metadata, credentials and grants rather than telemetry.
|
||||
var internalDatabases = map[string]struct{}{
|
||||
"system": {},
|
||||
"information_schema": {},
|
||||
}
|
||||
|
||||
// The parser's grammar has gaps against SQL that ClickHouse itself accepts. See TestErrIfStatementIsNotValid_ShouldPassButFails.
|
||||
func ErrIfStatementIsNotValid(query string) (err error) {
|
||||
defer func() {
|
||||
// The parser has a history of panicking on malformed input rather than returning an error.
|
||||
if recovered := recover(); recovered != nil {
|
||||
err = errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid ClickHouse SQL (recovered): %v", recovered)
|
||||
}
|
||||
}()
|
||||
|
||||
stmts, parseErr := chparser.NewParser(query).ParseStmts()
|
||||
if parseErr != nil {
|
||||
// Wrapped rather than formatted in, so that callers can recover the parser's *ParseError and read the position off it.
|
||||
return errors.WrapInvalidInputf(parseErr, errors.CodeInvalidInput, "invalid ClickHouse SQL: %s", parseErr.Error())
|
||||
}
|
||||
|
||||
if len(stmts) != 1 {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse SQL must contain exactly one statement, found %d statements", len(stmts))
|
||||
}
|
||||
|
||||
selectQuery, ok := stmts[0].(*chparser.SelectQuery)
|
||||
if !ok {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "only SELECT statements are allowed in ClickHouse SQL queries")
|
||||
}
|
||||
|
||||
visitor := &chparser.DefaultASTVisitor{Visit: func(node chparser.Expr) error {
|
||||
switch expr := node.(type) {
|
||||
case *chparser.TableFunctionExpr:
|
||||
// Source table functions remain usable in ClickHouse read-only mode.
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse table functions are not allowed in SQL queries: %s", chparser.Format(expr.Name))
|
||||
|
||||
case *chparser.TableIdentifier:
|
||||
// Reading these is unaffected by ClickHouse read-only mode.
|
||||
if expr.Database == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
if _, ok := internalDatabases[strings.ToLower(expr.Database.Name)]; ok {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse %s database is not allowed in SQL queries", expr.Database.Name)
|
||||
}
|
||||
|
||||
case *chparser.SettingExpr:
|
||||
// A query-level setting takes precedence over the context setting.
|
||||
if strings.EqualFold(expr.Name.Name, "readonly") {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse readonly setting cannot be overridden")
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}}
|
||||
|
||||
return selectQuery.Accept(visitor)
|
||||
}
|
||||
|
||||
// TODO(@therealpandey): remove this and move to ErrIfStatementIsNotValid.
|
||||
func LogIfStatementIsNotValid(ctx context.Context, logger *slog.Logger, query string) {
|
||||
if err := ErrIfStatementIsNotValid(query); err != nil {
|
||||
logger.WarnContext(ctx, "clickhouse sql is not valid", errors.Attr(err), slog.String("query", query))
|
||||
}
|
||||
}
|
||||
@@ -1,154 +0,0 @@
|
||||
package querybuilder
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
|
||||
testCases := []struct {
|
||||
name string
|
||||
query string
|
||||
}{
|
||||
// Shapes a telemetry read is allowed to take.
|
||||
{"Select", "SELECT region AS r, zone FROM metrics WHERE metric_name = 'cpu' GROUP BY region, zone"},
|
||||
{"TrailingSemicolon", "SELECT count() FROM signoz_logs.distributed_logs_v2;"},
|
||||
{"CommonTableExpression", "WITH t AS (SELECT fingerprint FROM signoz_metrics.time_series_v4) SELECT * FROM t"},
|
||||
{"Join", "SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.b"},
|
||||
{"GlobalIn", "SELECT a FROM t WHERE a GLOBAL IN (SELECT b FROM t2)"},
|
||||
{"Union", "SELECT * FROM t UNION ALL SELECT * FROM t2"},
|
||||
{"Intersect", "SELECT * FROM t INTERSECT SELECT * FROM t2"},
|
||||
{"WindowFunction", "SELECT sum(v) OVER (PARTITION BY a ORDER BY t) FROM t"},
|
||||
{"UnrelatedSetting", "SELECT * FROM t SETTINGS max_threads = 4"},
|
||||
{"TerminatedBlockComment", "SELECT /* keep me */ count() FROM t"},
|
||||
{"BlockCommentMarkerInsideStringLiteral", "SELECT count() FROM t WHERE body = '/* not a comment'"},
|
||||
// The parser used to loop forever on this; it now reads the comment to the end of
|
||||
// the input, so this doubles as a canary for that regression.
|
||||
{"TrailingUnterminatedBlockComment", "SELECT count() FROM t /* unterminated"},
|
||||
// The rule keys on the database, not on the table name.
|
||||
{"TableNamedSystemInTelemetryDatabase", "SELECT * FROM signoz_logs.system"},
|
||||
{"SignedLiteralAfterClosingParen", "SELECT (toUnixTimestamp(now()) - 3600)*1000000000"},
|
||||
// order by interval
|
||||
{"OrderByInterval", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval ORDER BY interval"},
|
||||
{"OrderByIntervalAndDirection", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS `interval` ORDER BY `interval` ASC"},
|
||||
{"TrimFunction", "SELECT trimBoth('/api/endpoint/', '/');"},
|
||||
}
|
||||
|
||||
for _, testCase := range testCases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
err := ErrIfStatementIsNotValid(testCase.query)
|
||||
assert.NoError(t, err)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
|
||||
testCases := []struct {
|
||||
name string
|
||||
query string
|
||||
}{
|
||||
// Not a single statement, or not a statement at all.
|
||||
{"Empty", ""},
|
||||
{"UnterminatedBlockCommentOnly", "/* x"},
|
||||
{"Unparseable", "SELECT FROM WHERE"},
|
||||
{"MultipleStatements", "SELECT 1; DROP TABLE signoz_logs.logs_v2"},
|
||||
// Parses, but is not a SELECT.
|
||||
{"Drop", "DROP TABLE signoz_logs.logs_v2"},
|
||||
{"Insert", "INSERT INTO signoz_logs.logs_v2 SELECT * FROM signoz_logs.logs_v2"},
|
||||
{"AlterDelete", "ALTER TABLE signoz_logs.logs_v2 DELETE WHERE 1 = 1"},
|
||||
{"CreateTable", "CREATE TABLE evil (a Int) ENGINE = Memory"},
|
||||
{"Grant", "GRANT ALL ON *.* TO admin"},
|
||||
{"Set", "SET readonly = 0"},
|
||||
// These the parser rejects outright rather than classifying.
|
||||
{"ShowGrants", "SHOW GRANTS"},
|
||||
{"IntoOutfile", "SELECT * FROM t INTO OUTFILE '/tmp/x.csv'"},
|
||||
// Table functions, which read through something other than a telemetry table.
|
||||
{"UrlTableFunction", "SELECT * FROM url('http://attacker.example/x', CSV, 'a String')"},
|
||||
{"FileTableFunction", "SELECT * FROM file('/etc/passwd', CSV, 'a String')"},
|
||||
{"ExecutableTableFunction", "SELECT * FROM executable('script.sh', CSV, 'a String')"},
|
||||
{"TableFunctionInJoin", "SELECT * FROM t1 JOIN url('http://x', CSV, 'a String') u ON 1 = 1"},
|
||||
{"TableFunctionInCommonTableExpression", "WITH c AS (SELECT * FROM url('http://x', CSV, 'a String')) SELECT * FROM c"},
|
||||
{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM file('/etc/passwd', CSV, 'a String'))"},
|
||||
{"TableFunctionInUnion", "SELECT * FROM t UNION ALL SELECT * FROM url('http://x', CSV, 'a String')"},
|
||||
// Internal databases, which hold grants and server metadata rather than telemetry.
|
||||
{"SystemUsers", "SELECT * FROM system.users"},
|
||||
{"SystemUppercase", "SELECT * FROM SYSTEM.USERS"},
|
||||
{"SystemQuoted", "SELECT count() FROM `system`.`tables`"},
|
||||
{"SystemInSubquery", "SELECT * FROM (SELECT name FROM system.parts)"},
|
||||
{"SystemInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l JOIN system.users AS u ON 1 = 1"},
|
||||
{"SystemInIntersect", "SELECT * FROM t INTERSECT SELECT * FROM system.users"},
|
||||
{"InformationSchema", "SELECT * FROM information_schema.tables"},
|
||||
// A query-level setting takes precedence over the one the caller applies.
|
||||
{"ReadonlySettingOverride", "SELECT * FROM t SETTINGS readonly = 0"},
|
||||
{"ReadonlySettingOverrideAmongOthers", "SELECT * FROM t SETTINGS max_threads = 4, readonly = 0"},
|
||||
}
|
||||
|
||||
for _, testCase := range testCases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
err := ErrIfStatementIsNotValid(testCase.query)
|
||||
assert.Error(t, err)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Queries the parser cannot read. ClickHouse runs all of them.
|
||||
func TestErrIfStatementIsNotValid_ShouldPassButFails(t *testing.T) {
|
||||
testCases := []struct {
|
||||
name string
|
||||
query string
|
||||
// The construct the parser stops after, which is the one it cannot read.
|
||||
expectedStopsAfter string
|
||||
// The same construct written so the parser accepts it.
|
||||
fix string
|
||||
}{
|
||||
{
|
||||
name: "IntervalAliasInOrderBy",
|
||||
query: "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, resource_string_service$$name AS `service.name`, attributes_string['http.route'] AS `http.route`, quantile(0.95)(duration_nano) / 1000000000 AS value FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_string_service$$name = 'svc-a' AND resources_string['deployment.environment'] = 'dev' AND attributes_string['http.route'] = '/v1' AND http_method = 'POST' AND timestamp BETWEEN toDateTime(1784601720) AND toDateTime(1784602620) AND ts_bucket_start BETWEEN 1784601720 - 1800 AND 1784602620 GROUP BY `service.name`, `http.route`, interval ORDER BY interval ASC",
|
||||
expectedStopsAfter: "ORDER BY interval ASC",
|
||||
fix: "SELECT count() AS interval FROM t ORDER BY `interval` ASC",
|
||||
},
|
||||
{
|
||||
name: "IntervalAliasInOrderByDesc",
|
||||
query: "SELECT count() AS value, toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, serviceName, resourceTagsMap['deployment.environment'] AS environment, exceptionStacktrace FROM signoz_traces.distributed_signoz_error_index_v2 WHERE exceptionType != 'OSError' AND resourceTagsMap['deployment.environment'] = 'staging' AND timestamp BETWEEN toDateTime(1785186300) AND toDateTime(1785186600) GROUP BY serviceName, interval, environment, exceptionStacktrace ORDER BY interval DESC",
|
||||
expectedStopsAfter: "ORDER BY interval DESC",
|
||||
fix: "SELECT count() AS interval FROM t ORDER BY `interval` DESC",
|
||||
},
|
||||
{
|
||||
name: "StandardTrimSyntax",
|
||||
query: "SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 5 MINUTE) AS interval, resources_string['host.name'] as host_name, toFloat64(countIf( lower(trim(BOTH ' ' FROM replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ))) IN ('unhealthy','starting','failing') )) as value FROM signoz_logs.distributed_logs_v2 WHERE timestamp BETWEEN 1784602320000000000 AND 1784602620000000000 AND ts_bucket_start BETWEEN 1784602320 - 300 AND 1784602620 AND JSONExtractString(body, 'Type') = 'container' AND JSONExtractString(body, 'Actor', 'Attributes', 'name') IS NOT NULL AND resources_string['host.name'] IS NOT NULL AND resources_string['host.name'] = 'aihub-nightly' GROUP BY interval, host_name ORDER BY interval, host_name",
|
||||
expectedStopsAfter: "trim(BOTH '",
|
||||
fix: "SELECT trimBoth(replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ), ' ')",
|
||||
},
|
||||
{
|
||||
name: "SignedLiteralAfterClosingParen",
|
||||
query: "SELECT now() AS ts, toFloat64(count()) AS value FROM ( SELECT attributes_string['TableName'] AS T, attributes_string['MissingId'] AS M, max(fromUnixTimestamp64Nano(timestamp)) AS last_seen, dateDiff('minute', min(fromUnixTimestamp64Nano(timestamp)), max(fromUnixTimestamp64Nano(timestamp))) AS age_min FROM signoz_logs.distributed_logs_v2 WHERE body='missing_map_record' AND timestamp >= (toUnixTimestamp(now())-3600)*1000000000 GROUP BY T, M ) WHERE age_min >= 20 AND last_seen >= now() - toIntervalMinute(8)",
|
||||
expectedStopsAfter: "(toUnixTimestamp(now())",
|
||||
fix: "SELECT (toUnixTimestamp(now()) - 3600) * 1000000000",
|
||||
},
|
||||
{
|
||||
name: "SignedLiteralAfterClosingParenMinimal",
|
||||
query: "SELECT (1)-1",
|
||||
expectedStopsAfter: "(1)",
|
||||
fix: "SELECT (1) - 1",
|
||||
},
|
||||
}
|
||||
|
||||
for _, testCase := range testCases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
err := ErrIfStatementIsNotValid(testCase.query)
|
||||
|
||||
var parseErr *chparser.ParseError
|
||||
require.ErrorAs(t, err, &parseErr, "expected a parser failure rather than a rule violation")
|
||||
|
||||
// The parser reports the offset it stopped at, which sits just past the construct
|
||||
// it choked on, so the text leading up to it is what needs looking at.
|
||||
consumed := testCase.query[:parseErr.Pos]
|
||||
assert.Equal(t, testCase.expectedStopsAfter, consumed[max(0, len(consumed)-len(testCase.expectedStopsAfter)):])
|
||||
|
||||
assert.NoError(t, ErrIfStatementIsNotValid(testCase.fix))
|
||||
})
|
||||
}
|
||||
}
|
||||
1
pkg/querybuilder/init.go
Normal file
1
pkg/querybuilder/init.go
Normal file
@@ -0,0 +1 @@
|
||||
package querybuilder
|
||||
@@ -39,7 +39,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/sqlmigrator"
|
||||
"github.com/SigNoz/signoz/pkg/sqlschema"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/tokenizer"
|
||||
@@ -98,9 +97,6 @@ type Config struct {
|
||||
// Querier config
|
||||
Querier querier.Config `mapstructure:"querier"`
|
||||
|
||||
// StatementBuilder config
|
||||
StatementBuilder statementbuilder.Config `mapstructure:"statementbuilder"`
|
||||
|
||||
// Ruler config
|
||||
Ruler ruler.Config `mapstructure:"ruler"`
|
||||
|
||||
@@ -170,7 +166,6 @@ func NewConfig(ctx context.Context, logger *slog.Logger, resolverConfig config.R
|
||||
prometheus.NewConfigFactory(),
|
||||
alertmanager.NewConfigFactory(),
|
||||
querier.NewConfigFactory(),
|
||||
statementbuilder.NewConfigFactory(),
|
||||
ruler.NewConfigFactory(),
|
||||
emailing.NewConfigFactory(),
|
||||
sharder.NewConfigFactory(),
|
||||
|
||||
@@ -56,7 +56,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore/sqlitesqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstorehook"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter"
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter/analyticsstatsreporter"
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter/noopstatsreporter"
|
||||
@@ -70,7 +69,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/types/alertmanagertypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/featuretypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/version"
|
||||
"github.com/SigNoz/signoz/pkg/web"
|
||||
"github.com/SigNoz/signoz/pkg/web/noopweb"
|
||||
@@ -287,9 +285,9 @@ func NewStatsReporterProviderFactories(aggregator statsreporter.Aggregator, orgG
|
||||
)
|
||||
}
|
||||
|
||||
func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, prometheus prometheus.Prometheus, metadataStore telemetrytypes.MetadataStore, builders *statementbuilder.Builders, bucketCache querier.BucketCache, flagger flagger.Flagger) factory.NamedMap[factory.ProviderFactory[querier.Querier, querier.Config]] {
|
||||
func NewQuerierProviderFactories(telemetryStore telemetrystore.TelemetryStore, prometheus prometheus.Prometheus, cache cache.Cache, flagger flagger.Flagger) factory.NamedMap[factory.ProviderFactory[querier.Querier, querier.Config]] {
|
||||
return factory.MustNewNamedMap(
|
||||
signozquerier.NewFactory(telemetryStore, prometheus, metadataStore, builders, bucketCache, flagger),
|
||||
signozquerier.NewFactory(telemetryStore, prometheus, cache, flagger),
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
@@ -48,10 +48,14 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/sqlmigrator"
|
||||
"github.com/SigNoz/signoz/pkg/sqlschema"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryaudit"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetadata"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymeter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
pkgtokenizer "github.com/SigNoz/signoz/pkg/tokenizer"
|
||||
"github.com/SigNoz/signoz/pkg/types/authtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
@@ -92,29 +96,6 @@ type SigNoz struct {
|
||||
MeterReporter meterreporter.Reporter
|
||||
}
|
||||
|
||||
// newQueryStack assembles the query stack once: the shared telemetry metadata
|
||||
// store, the per-signal statement builders, and the bucket cache. The metadata
|
||||
// store is returned so callers can reuse the single instance downstream.
|
||||
func newQueryStack(
|
||||
ctx context.Context,
|
||||
settings factory.ProviderSettings,
|
||||
config Config,
|
||||
telemetryStore telemetrystore.TelemetryStore,
|
||||
cache cache.Cache,
|
||||
fl flagger.Flagger,
|
||||
) (telemetrytypes.MetadataStore, *statementbuilder.Builders, querier.BucketCache, error) {
|
||||
metadataStore := telemetrymetadata.NewTelemetryMetaStore(settings, telemetryStore, fl)
|
||||
|
||||
builders, err := statementbuilder.NewFactory(telemetryStore, metadataStore, fl).New(ctx, settings, config.StatementBuilder)
|
||||
if err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
|
||||
bucketCache := querier.NewBucketCache(settings, cache, config.Querier.CacheTTL, config.Querier.FluxInterval)
|
||||
|
||||
return metadataStore, builders, bucketCache, nil
|
||||
}
|
||||
|
||||
func New(
|
||||
ctx context.Context,
|
||||
config Config,
|
||||
@@ -273,19 +254,12 @@ func New(
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Assemble the query stack (metadata store, statement builders, bucket cache) once,
|
||||
// and reuse the single metadata store everywhere downstream.
|
||||
telemetryMetadataStore, builders, bucketCache, err := newQueryStack(ctx, providerSettings, config, telemetrystore, cache, flagger)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Initialize querier from the available querier provider factories
|
||||
querier, err := factory.NewProviderFromNamedMap(
|
||||
ctx,
|
||||
providerSettings,
|
||||
config.Querier,
|
||||
NewQuerierProviderFactories(telemetrystore, prometheus, telemetryMetadataStore, builders, bucketCache, flagger),
|
||||
NewQuerierProviderFactories(telemetrystore, prometheus, cache, flagger),
|
||||
config.Querier.Provider(),
|
||||
)
|
||||
if err != nil {
|
||||
@@ -453,6 +427,35 @@ func New(
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Initialize telemetry metadata store
|
||||
// TODO: consolidate other telemetrymetadata.NewTelemetryMetaStore initializations to reuse this instance instead.
|
||||
telemetryMetadataStore := telemetrymetadata.NewTelemetryMetaStore(
|
||||
providerSettings,
|
||||
telemetrystore,
|
||||
telemetrytraces.DBName,
|
||||
telemetrytraces.TagAttributesV2TableName,
|
||||
telemetrytraces.SpanAttributesKeysTblName,
|
||||
telemetrytraces.SpanIndexV3TableName,
|
||||
telemetrymetrics.DBName,
|
||||
telemetrymetrics.AttributesMetadataTableName,
|
||||
telemetrymeter.DBName,
|
||||
telemetrymeter.SamplesAgg1dTableName,
|
||||
telemetrylogs.DBName,
|
||||
telemetrylogs.LogsV2TableName,
|
||||
telemetrylogs.TagAttributesV2TableName,
|
||||
telemetrylogs.LogAttributeKeysTblName,
|
||||
telemetrylogs.LogResourceKeysTblName,
|
||||
telemetryaudit.DBName,
|
||||
telemetryaudit.AuditLogsTableName,
|
||||
telemetryaudit.TagAttributesTableName,
|
||||
telemetryaudit.LogAttributeKeysTblName,
|
||||
telemetryaudit.LogResourceKeysTblName,
|
||||
telemetrymetadata.DBName,
|
||||
telemetrymetadata.AttributesMetadataTableName,
|
||||
telemetrymetadata.ColumnEvolutionMetadataTableName,
|
||||
flagger,
|
||||
)
|
||||
|
||||
global, err := factory.NewProviderFromNamedMap(
|
||||
ctx,
|
||||
providerSettings,
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
package auditstatementbuilder
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
)
|
||||
|
||||
// New builds the audit statement builder, internalizing the FieldMapper,
|
||||
// ConditionBuilder, and AggExprRewriter.
|
||||
func New(
|
||||
settings factory.ProviderSettings,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
fl flagger.Flagger,
|
||||
) qbtypes.StatementBuilder[qbtypes.LogAggregation] {
|
||||
fm := audittelemetryschema.NewFieldMapper()
|
||||
cb := audittelemetryschema.NewConditionBuilder(fm)
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, audittelemetryschema.DefaultFullTextColumn, fm, cb, fl)
|
||||
|
||||
return NewAuditQueryStatementBuilder(
|
||||
settings, metadataStore, fm, cb, aggExprRewriter, audittelemetryschema.DefaultFullTextColumn, fl,
|
||||
)
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
package statementbuilder
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
)
|
||||
|
||||
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
|
||||
type SkipResourceFingerprint struct {
|
||||
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
|
||||
// If count of fingerprint is above threshold, skip the fingerprint subquery and filter on main table instead.
|
||||
Threshold uint64 `yaml:"threshold" mapstructure:"threshold"`
|
||||
}
|
||||
|
||||
// Config represents the configuration for the statement builders.
|
||||
type Config struct {
|
||||
// SkipResourceFingerprint configures when the resource fingerprint subquery is skipped in favor of main-table filtering.
|
||||
SkipResourceFingerprint SkipResourceFingerprint `yaml:"skip_resource_fingerprint" mapstructure:"skip_resource_fingerprint"`
|
||||
}
|
||||
|
||||
// NewConfigFactory creates a new config factory for the statement builders.
|
||||
func NewConfigFactory() factory.ConfigFactory {
|
||||
return factory.NewConfigFactory(factory.MustNewName("statementbuilder"), newConfig)
|
||||
}
|
||||
|
||||
func newConfig() factory.Config {
|
||||
return Config{
|
||||
SkipResourceFingerprint: SkipResourceFingerprint{
|
||||
Enabled: false,
|
||||
Threshold: 100000,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Validate validates the configuration.
|
||||
func (c Config) Validate() error {
|
||||
if c.SkipResourceFingerprint.Enabled && c.SkipResourceFingerprint.Threshold == 0 {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput, "skip_resource_fingerprint.threshold must be > 0 when enabled")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package logsstatementbuilder
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
)
|
||||
|
||||
// New builds the logs statement builder, internalizing the FieldMapper,
|
||||
// ConditionBuilder, and AggExprRewriter.
|
||||
func New(
|
||||
settings factory.ProviderSettings,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
telemetryStore telemetrystore.TelemetryStore,
|
||||
fl flagger.Flagger,
|
||||
skipResourceFingerprintEnabled bool,
|
||||
skipResourceFingerprintThreshold uint64,
|
||||
) qbtypes.StatementBuilder[qbtypes.LogAggregation] {
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, logstelemetryschema.DefaultFullTextColumn, fm, cb, fl)
|
||||
|
||||
return NewLogQueryStatementBuilder(
|
||||
settings, metadataStore, fm, cb, aggExprRewriter, logstelemetryschema.DefaultFullTextColumn,
|
||||
fl, telemetryStore, skipResourceFingerprintEnabled, skipResourceFingerprintThreshold,
|
||||
)
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
package meterstatementbuilder
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
)
|
||||
|
||||
// New builds the meter statement builder. It reuses the metrics FieldMapper and
|
||||
// ConditionBuilder, and delegates the final SELECT to the provided metrics statement builder.
|
||||
func New(
|
||||
settings factory.ProviderSettings,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
metricsStatementBuilder *metricsstatementbuilder.StatementBuilder,
|
||||
) qbtypes.StatementBuilder[qbtypes.MetricAggregation] {
|
||||
fm := metricstelemetryschema.NewFieldMapper()
|
||||
cb := metricstelemetryschema.NewConditionBuilder(fm)
|
||||
return NewMeterQueryStatementBuilder(settings, metadataStore, fm, cb, metricsStatementBuilder)
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
package metricsstatementbuilder
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
)
|
||||
|
||||
// New builds the metrics statement builder, internalizing the FieldMapper and
|
||||
// ConditionBuilder. It returns the concrete type so the meter builder can reuse it.
|
||||
func New(
|
||||
settings factory.ProviderSettings,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
fl flagger.Flagger,
|
||||
) *StatementBuilder {
|
||||
fm := metricstelemetryschema.NewFieldMapper()
|
||||
cb := metricstelemetryschema.NewConditionBuilder(fm)
|
||||
return NewMetricQueryStatementBuilder(settings, metadataStore, fm, cb, fl)
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
package statementbuilder
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/auditstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/logsstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/meterstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/tracesstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
)
|
||||
|
||||
// Builders bundles the per-signal statement builders that make up the query stack.
|
||||
type Builders struct {
|
||||
Trace qbtypes.StatementBuilder[qbtypes.TraceAggregation]
|
||||
TraceOperator qbtypes.TraceOperatorStatementBuilder
|
||||
Log qbtypes.StatementBuilder[qbtypes.LogAggregation]
|
||||
Audit qbtypes.StatementBuilder[qbtypes.LogAggregation]
|
||||
Metric qbtypes.StatementBuilder[qbtypes.MetricAggregation]
|
||||
Meter qbtypes.StatementBuilder[qbtypes.MetricAggregation]
|
||||
}
|
||||
|
||||
// NewFactory returns a provider factory that builds the per-signal statement builders once.
|
||||
func NewFactory(
|
||||
telemetryStore telemetrystore.TelemetryStore,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
fl flagger.Flagger,
|
||||
) factory.ProviderFactory[*Builders, Config] {
|
||||
return factory.NewProviderFactory(
|
||||
factory.MustNewName("signoz"),
|
||||
func(_ context.Context, settings factory.ProviderSettings, cfg Config) (*Builders, error) {
|
||||
traceStmtBuilder, traceOperatorStmtBuilder := tracesstatementbuilder.New(
|
||||
settings, metadataStore, telemetryStore, fl,
|
||||
cfg.SkipResourceFingerprint.Enabled, cfg.SkipResourceFingerprint.Threshold,
|
||||
)
|
||||
logStmtBuilder := logsstatementbuilder.New(
|
||||
settings, metadataStore, telemetryStore, fl,
|
||||
cfg.SkipResourceFingerprint.Enabled, cfg.SkipResourceFingerprint.Threshold,
|
||||
)
|
||||
auditStmtBuilder := auditstatementbuilder.New(settings, metadataStore, fl)
|
||||
metricStmtBuilder := metricsstatementbuilder.New(settings, metadataStore, fl)
|
||||
meterStmtBuilder := meterstatementbuilder.New(settings, metadataStore, metricStmtBuilder)
|
||||
|
||||
return &Builders{
|
||||
Trace: traceStmtBuilder,
|
||||
TraceOperator: traceOperatorStmtBuilder,
|
||||
Log: logStmtBuilder,
|
||||
Audit: auditStmtBuilder,
|
||||
Metric: metricStmtBuilder,
|
||||
Meter: meterStmtBuilder,
|
||||
}, nil
|
||||
},
|
||||
)
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
package tracesstatementbuilder
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
)
|
||||
|
||||
// New builds the trace and trace-operator statement builders, internalizing the
|
||||
// FieldMapper, ConditionBuilder, and AggExprRewriter.
|
||||
func New(
|
||||
settings factory.ProviderSettings,
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
telemetryStore telemetrystore.TelemetryStore,
|
||||
fl flagger.Flagger,
|
||||
skipResourceFingerprintEnabled bool,
|
||||
skipResourceFingerprintThreshold uint64,
|
||||
) (qbtypes.StatementBuilder[qbtypes.TraceAggregation], qbtypes.TraceOperatorStatementBuilder) {
|
||||
fm := tracestelemetryschema.NewFieldMapper()
|
||||
cb := tracestelemetryschema.NewConditionBuilder(fm)
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, fm, cb, fl)
|
||||
|
||||
traceStmtBuilder := NewTraceQueryStatementBuilder(
|
||||
settings, metadataStore, fm, cb, aggExprRewriter, telemetryStore, fl,
|
||||
skipResourceFingerprintEnabled, skipResourceFingerprintThreshold,
|
||||
)
|
||||
traceOperatorStmtBuilder := NewTraceOperatorStatementBuilder(
|
||||
settings, metadataStore, fm, cb, traceStmtBuilder, aggExprRewriter, fl,
|
||||
)
|
||||
return traceStmtBuilder, traceOperatorStmtBuilder
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
package audittelemetryschema
|
||||
package telemetryaudit
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -1,4 +1,4 @@
|
||||
package audittelemetryschema
|
||||
package telemetryaudit
|
||||
|
||||
import (
|
||||
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
|
||||
@@ -127,14 +127,3 @@ var auditLogColumns = map[string]*schema.Column{
|
||||
"scope_version": {Name: "scope_version", Type: schema.ColumnTypeString},
|
||||
"scope_string": {Name: "scope_string", Type: schema.MapColumnType{KeyType: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString}, ValueType: schema.ColumnTypeString}},
|
||||
}
|
||||
|
||||
const (
|
||||
DBName = "signoz_audit"
|
||||
AuditLogsTableName = "distributed_logs"
|
||||
AuditLogsLocalTableName = "logs"
|
||||
TagAttributesTableName = "distributed_tag_attributes"
|
||||
TagAttributesLocalTableName = "tag_attributes"
|
||||
LogAttributeKeysTblName = "distributed_logs_attribute_keys"
|
||||
LogResourceKeysTblName = "distributed_logs_resource_keys"
|
||||
LogsResourceTableName = "distributed_logs_resource"
|
||||
)
|
||||
@@ -1,4 +1,4 @@
|
||||
package audittelemetryschema
|
||||
package telemetryaudit
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -1,4 +1,4 @@
|
||||
package auditstatementbuilder
|
||||
package telemetryaudit
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -10,8 +10,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -39,12 +38,12 @@ func NewAuditQueryStatementBuilder(
|
||||
fullTextColumn *telemetrytypes.TelemetryFieldKey,
|
||||
flagger flagger.Flagger,
|
||||
) *auditQueryStatementBuilder {
|
||||
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema")
|
||||
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryaudit")
|
||||
|
||||
resourceFilterStmtBuilder := resourcefilter.New[qbtypes.LogAggregation](
|
||||
resourceFilterStmtBuilder := telemetryresourcefilter.New[qbtypes.LogAggregation](
|
||||
settings,
|
||||
audittelemetryschema.DBName,
|
||||
audittelemetryschema.LogsResourceTableName,
|
||||
DBName,
|
||||
LogsResourceTableName,
|
||||
telemetrytypes.SignalLogs,
|
||||
telemetrytypes.SourceAudit,
|
||||
metadataStore,
|
||||
@@ -192,8 +191,8 @@ func (b *auditQueryStatementBuilder) adjustKeys(ctx context.Context, keys map[st
|
||||
}
|
||||
|
||||
func (b *auditQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
|
||||
if _, ok := audittelemetryschema.IntrinsicFields[key.Name]; ok {
|
||||
intrinsicField := audittelemetryschema.IntrinsicFields[key.Name]
|
||||
if _, ok := IntrinsicFields[key.Name]; ok {
|
||||
intrinsicField := IntrinsicFields[key.Name]
|
||||
return querybuilder.AdjustKey(key, keys, &intrinsicField)
|
||||
}
|
||||
return querybuilder.AdjustKey(key, keys, nil)
|
||||
@@ -220,26 +219,26 @@ func (b *auditQueryStatementBuilder) buildListQuery(
|
||||
cteArgs = append(cteArgs, args)
|
||||
}
|
||||
|
||||
sb.Select(audittelemetryschema.TimestampColumn)
|
||||
sb.SelectMore(audittelemetryschema.IDColumn)
|
||||
sb.Select(TimestampColumn)
|
||||
sb.SelectMore(IDColumn)
|
||||
if len(query.SelectFields) == 0 {
|
||||
sb.SelectMore(audittelemetryschema.TraceIDColumn)
|
||||
sb.SelectMore(audittelemetryschema.SpanIDColumn)
|
||||
sb.SelectMore(audittelemetryschema.TraceFlagsColumn)
|
||||
sb.SelectMore(audittelemetryschema.SeverityTextColumn)
|
||||
sb.SelectMore(audittelemetryschema.SeverityNumberColumn)
|
||||
sb.SelectMore(audittelemetryschema.ScopeNameColumn)
|
||||
sb.SelectMore(audittelemetryschema.ScopeVersionColumn)
|
||||
sb.SelectMore(audittelemetryschema.BodyColumn)
|
||||
sb.SelectMore(audittelemetryschema.EventNameColumn)
|
||||
sb.SelectMore(audittelemetryschema.AttributesStringColumn)
|
||||
sb.SelectMore(audittelemetryschema.AttributesNumberColumn)
|
||||
sb.SelectMore(audittelemetryschema.AttributesBoolColumn)
|
||||
sb.SelectMore(audittelemetryschema.ResourceColumn)
|
||||
sb.SelectMore(audittelemetryschema.ScopeStringColumn)
|
||||
sb.SelectMore(TraceIDColumn)
|
||||
sb.SelectMore(SpanIDColumn)
|
||||
sb.SelectMore(TraceFlagsColumn)
|
||||
sb.SelectMore(SeverityTextColumn)
|
||||
sb.SelectMore(SeverityNumberColumn)
|
||||
sb.SelectMore(ScopeNameColumn)
|
||||
sb.SelectMore(ScopeVersionColumn)
|
||||
sb.SelectMore(BodyColumn)
|
||||
sb.SelectMore(EventNameColumn)
|
||||
sb.SelectMore(AttributesStringColumn)
|
||||
sb.SelectMore(AttributesNumberColumn)
|
||||
sb.SelectMore(AttributesBoolColumn)
|
||||
sb.SelectMore(ResourceColumn)
|
||||
sb.SelectMore(ScopeStringColumn)
|
||||
} else {
|
||||
for index := range query.SelectFields {
|
||||
if query.SelectFields[index].Name == audittelemetryschema.TimestampColumn || query.SelectFields[index].Name == audittelemetryschema.IDColumn {
|
||||
if query.SelectFields[index].Name == TimestampColumn || query.SelectFields[index].Name == IDColumn {
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -251,7 +250,7 @@ func (b *auditQueryStatementBuilder) buildListQuery(
|
||||
}
|
||||
}
|
||||
|
||||
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
|
||||
|
||||
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
|
||||
if err != nil {
|
||||
@@ -341,7 +340,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
|
||||
sb.SelectMore(fmt.Sprintf("%s AS __result_%d", rewritten, i))
|
||||
}
|
||||
|
||||
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
|
||||
|
||||
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
|
||||
if err != nil {
|
||||
@@ -479,7 +478,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
|
||||
}
|
||||
}
|
||||
|
||||
sb.From(fmt.Sprintf("%s.%s", audittelemetryschema.DBName, audittelemetryschema.AuditLogsTableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", DBName, AuditLogsTableName))
|
||||
|
||||
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables)
|
||||
if err != nil {
|
||||
@@ -1,4 +1,4 @@
|
||||
package auditstatementbuilder
|
||||
package telemetryaudit
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -8,7 +8,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
|
||||
@@ -55,8 +54,8 @@ func newTestAuditStatementBuilder(t *testing.T) *auditQueryStatementBuilder {
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
mockMetadataStore.KeysMap = auditFieldKeyMap()
|
||||
|
||||
fm := audittelemetryschema.NewFieldMapper()
|
||||
cb := audittelemetryschema.NewConditionBuilder(fm)
|
||||
fm := NewFieldMapper()
|
||||
cb := NewConditionBuilder(fm)
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
return NewAuditQueryStatementBuilder(
|
||||
@@ -65,7 +64,7 @@ func newTestAuditStatementBuilder(t *testing.T) *auditQueryStatementBuilder {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
audittelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
)
|
||||
}
|
||||
12
pkg/telemetryaudit/tables.go
Normal file
12
pkg/telemetryaudit/tables.go
Normal file
@@ -0,0 +1,12 @@
|
||||
package telemetryaudit
|
||||
|
||||
const (
|
||||
DBName = "signoz_audit"
|
||||
AuditLogsTableName = "distributed_logs"
|
||||
AuditLogsLocalTableName = "logs"
|
||||
TagAttributesTableName = "distributed_tag_attributes"
|
||||
TagAttributesLocalTableName = "tag_attributes"
|
||||
LogAttributeKeysTblName = "distributed_logs_attribute_keys"
|
||||
LogResourceKeysTblName = "distributed_logs_resource_keys"
|
||||
LogsResourceTableName = "distributed_logs_resource"
|
||||
)
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -548,7 +548,7 @@ func TestConditionForSynthesizedPrefixedKeys(t *testing.T) {
|
||||
t.Run("bare intrinsic column resolves to the column, not synthesized attributes", func(t *testing.T) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
key := telemetrytypes.TelemetryFieldKey{Name: "severity_text"}
|
||||
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "ERROR", sb)
|
||||
conds, _, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{},qbtypes.FilterOperatorEqual, "ERROR", sb)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, conds, 1)
|
||||
sb.Where(conds...)
|
||||
@@ -560,7 +560,7 @@ func TestConditionForSynthesizedPrefixedKeys(t *testing.T) {
|
||||
t.Run("attribute context", func(t *testing.T) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
key := telemetrytypes.TelemetryFieldKey{Name: "custom.key", FieldContext: telemetrytypes.FieldContextAttribute}
|
||||
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "v", sb)
|
||||
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{},qbtypes.FilterOperatorEqual, "v", sb)
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, warnings)
|
||||
require.Len(t, conds, 2)
|
||||
@@ -571,7 +571,7 @@ func TestConditionForSynthesizedPrefixedKeys(t *testing.T) {
|
||||
t.Run("resource context", func(t *testing.T) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
key := telemetrytypes.TelemetryFieldKey{Name: "custom.key", FieldContext: telemetrytypes.FieldContextResource}
|
||||
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "v", sb)
|
||||
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{},qbtypes.FilterOperatorEqual, "v", sb)
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, warnings)
|
||||
require.Len(t, conds, 2)
|
||||
@@ -582,7 +582,7 @@ func TestConditionForSynthesizedPrefixedKeys(t *testing.T) {
|
||||
t.Run("log context folds to attributes then body", func(t *testing.T) {
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
key := telemetrytypes.TelemetryFieldKey{Name: "custom.key", FieldContext: telemetrytypes.FieldContextLog}
|
||||
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "v", sb)
|
||||
conds, warnings, err := cb.ConditionFor(ctx, valuer.UUID{}, 0, 0, &key, nil, qbtypes.ConditionBuilderOptions{},qbtypes.FilterOperatorEqual, "v", sb)
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, warnings)
|
||||
require.Len(t, conds, 4)
|
||||
@@ -1,6 +1,8 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/SigNoz/signoz-otel-collector/constants"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
@@ -39,8 +41,9 @@ const (
|
||||
|
||||
// messageSubColumn is the ClickHouse sub-column that body searches map to
|
||||
// when use_json_body feature flag is true.
|
||||
messageSubField = "message"
|
||||
messageSubColumn = "body_v2.message"
|
||||
messageSubField = "message"
|
||||
messageSubColumn = "body_v2.message"
|
||||
bodySearchDefaultWarning = "body searches default to `body.message:string`. Use `body.<key>` to search a different field inside body"
|
||||
|
||||
// bodyMessageField is the field name addressing the message sub-field of the
|
||||
// body when use_json_body is enabled (i.e. `body` + `.` + `message`). hasToken
|
||||
@@ -134,16 +137,10 @@ var (
|
||||
}
|
||||
)
|
||||
|
||||
const (
|
||||
DBName = "signoz_logs"
|
||||
LogsV2TableName = "distributed_logs_v2"
|
||||
LogsV2LocalTableName = "logs_v2"
|
||||
TagAttributesV2TableName = "distributed_tag_attributes_v2"
|
||||
TagAttributesV2LocalTableName = "tag_attributes_v2"
|
||||
LogAttributeKeysTblName = "distributed_logs_attribute_keys"
|
||||
LogResourceKeysTblName = "distributed_logs_resource_keys"
|
||||
LogsResourceV2TableName = "distributed_logs_v2_resource"
|
||||
PathTypesTableName = "distributed_json_path_types"
|
||||
PromotedPathsTableName = "distributed_json_promoted_paths"
|
||||
SkipIndexTableName = "system.data_skipping_indices"
|
||||
)
|
||||
func bodyAliasExpression(bodyJSONEnabled bool) string {
|
||||
if !bodyJSONEnabled {
|
||||
return LogsV2BodyColumn
|
||||
}
|
||||
|
||||
return fmt.Sprintf("%s as body", LogsV2BodyV2Column)
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -19,7 +19,7 @@ func TestLikeAndILikeWithoutWildcards_Warns(t *testing.T) {
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
keys := BuildCompleteFieldKeyMap(releaseTime)
|
||||
keys := buildCompleteFieldKeyMap(releaseTime)
|
||||
|
||||
opts := querybuilder.FilterExprVisitorOpts{
|
||||
Context: ctx,
|
||||
@@ -57,7 +57,7 @@ func TestLikeAndILikeWithWildcards_NoWarn(t *testing.T) {
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
keys := BuildCompleteFieldKeyMap(releaseTime)
|
||||
keys := buildCompleteFieldKeyMap(releaseTime)
|
||||
|
||||
opts := querybuilder.FilterExprVisitorOpts{
|
||||
Context: context.Background(),
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -21,7 +21,7 @@ func TestFilterExprLogsBodyJSON(t *testing.T) {
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
// Define a comprehensive set of field keys to support all test cases
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
keys := BuildCompleteFieldKeyMap(releaseTime)
|
||||
keys := buildCompleteFieldKeyMap(releaseTime)
|
||||
|
||||
opts := querybuilder.FilterExprVisitorOpts{
|
||||
Context: context.Background(),
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -41,7 +41,7 @@ func TestFilterExprLogs(t *testing.T) {
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
// Define a comprehensive set of field keys to support all test cases
|
||||
keys := BuildCompleteFieldKeyMap(releaseTime)
|
||||
keys := buildCompleteFieldKeyMap(releaseTime)
|
||||
|
||||
opts := querybuilder.FilterExprVisitorOpts{
|
||||
Context: ctx,
|
||||
@@ -2476,7 +2476,7 @@ func TestFilterExprLogsConflictNegation(t *testing.T) {
|
||||
|
||||
// Define a comprehensive set of field keys to support all test cases
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
keys := BuildCompleteFieldKeyMap(releaseTime)
|
||||
keys := buildCompleteFieldKeyMap(releaseTime)
|
||||
|
||||
keys["body"] = []*telemetrytypes.TelemetryFieldKey{
|
||||
{
|
||||
@@ -1,4 +1,4 @@
|
||||
package logsstatementbuilder
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -8,7 +8,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
|
||||
@@ -44,14 +43,14 @@ func TestStatementBuilderGroupByUnknownKey(t *testing.T) {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
fl := flaggertest.WithUseJSONBody(t, c.useJSONBody)
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
|
||||
fm := NewFieldMapper(fl)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
statementBuilder := NewLogQueryStatementBuilder(
|
||||
instrumentationtest.New().ToProviderSettings(),
|
||||
mockMetadataStore, fm, cb, aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn, fl, nil, false, 100000,
|
||||
DefaultFullTextColumn, fl, nil, false, 100000,
|
||||
)
|
||||
|
||||
query := qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"strings"
|
||||
@@ -1,4 +1,4 @@
|
||||
package logsstatementbuilder
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
|
||||
@@ -1226,7 +1225,7 @@ func TestJSONStmtBuilder_OrderBy(t *testing.T) {
|
||||
func TestResourceAggrAndGroupBy_WithJSONEnabled(t *testing.T) {
|
||||
statementBuilder, metadataStore := buildJSONTestStatementBuilder(t, false)
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
keysMap := logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
|
||||
keysMap := buildCompleteFieldKeyMap(releaseTime)
|
||||
for _, keys := range keysMap {
|
||||
for _, key := range keys {
|
||||
metadataStore.SetKey(key)
|
||||
@@ -1288,7 +1287,7 @@ func TestResourceAggrAndGroupBy_WithJSONEnabled(t *testing.T) {
|
||||
|
||||
func buildTestTelemetryMetadataStore(t *testing.T, addIndexes bool) *telemetrytypestest.MockMetadataStore {
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
mockMetadataStore.SetStaticFields(logstelemetryschema.IntrinsicFields)
|
||||
mockMetadataStore.SetStaticFields(IntrinsicFields)
|
||||
types, _ := telemetrytypes.TestJSONTypeSet()
|
||||
for path, fieldDataTypes := range types {
|
||||
for _, fdt := range fieldDataTypes {
|
||||
@@ -1308,15 +1307,15 @@ func buildTestTelemetryMetadataStore(t *testing.T, addIndexes bool) *telemetryty
|
||||
Name: path,
|
||||
FieldContext: telemetrytypes.FieldContextBody,
|
||||
FieldDataType: fdt,
|
||||
BaseColumn: logstelemetryschema.LogsV2BodyV2Column,
|
||||
IndexExpression: schemamigrator.JSONSubColumnIndexExpr(logstelemetryschema.LogsV2BodyV2Column, path, jsonType.StringValue()),
|
||||
BaseColumn: LogsV2BodyV2Column,
|
||||
IndexExpression: schemamigrator.JSONSubColumnIndexExpr(LogsV2BodyV2Column, path, jsonType.StringValue()),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
err := key.SetJSONAccessPlan(telemetrytypes.JSONColumnMetadata{
|
||||
BaseColumn: logstelemetryschema.LogsV2BodyV2Column,
|
||||
PromotedColumn: logstelemetryschema.LogsV2BodyPromotedColumn,
|
||||
BaseColumn: LogsV2BodyV2Column,
|
||||
PromotedColumn: LogsV2BodyPromotedColumn,
|
||||
}, types)
|
||||
require.NoError(t, err)
|
||||
|
||||
@@ -1332,8 +1331,8 @@ func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) (*logQueryStat
|
||||
|
||||
mockMetadataStore := buildTestTelemetryMetadataStore(t, addIndexes)
|
||||
fl := flaggertest.WithUseJSONBody(t, true)
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
@@ -1343,7 +1342,7 @@ func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) (*logQueryStat
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -1,4 +1,4 @@
|
||||
package logsstatementbuilder
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -10,8 +10,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/types/featuretypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
@@ -20,22 +19,12 @@ import (
|
||||
"github.com/huandu/go-sqlbuilder"
|
||||
)
|
||||
|
||||
const bodySearchDefaultWarning = "body searches default to `body.message:string`. Use `body.<key>` to search a different field inside body"
|
||||
|
||||
func bodyAliasExpression(bodyJSONEnabled bool) string {
|
||||
if !bodyJSONEnabled {
|
||||
return logstelemetryschema.LogsV2BodyColumn
|
||||
}
|
||||
|
||||
return fmt.Sprintf("%s as body", logstelemetryschema.LogsV2BodyV2Column)
|
||||
}
|
||||
|
||||
type logQueryStatementBuilder struct {
|
||||
logger *slog.Logger
|
||||
metadataStore telemetrytypes.MetadataStore
|
||||
fm qbtypes.FieldMapper
|
||||
cb qbtypes.ConditionBuilder
|
||||
resourceFilterResolver *resourcefilter.ResourceFingerprintResolver[qbtypes.LogAggregation]
|
||||
resourceFilterResolver *telemetryresourcefilter.ResourceFingerprintResolver[qbtypes.LogAggregation]
|
||||
aggExprRewriter qbtypes.AggExprRewriter
|
||||
fl flagger.Flagger
|
||||
skipResourceFingerprintEnabled bool
|
||||
@@ -57,12 +46,12 @@ func NewLogQueryStatementBuilder(
|
||||
skipResourceFingerprintEnable bool,
|
||||
skipResourceFingerprintThreshold uint64,
|
||||
) *logQueryStatementBuilder {
|
||||
logsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema")
|
||||
logsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrylogs")
|
||||
|
||||
resourceFilterResolver := resourcefilter.NewResolver[qbtypes.LogAggregation](
|
||||
resourceFilterResolver := telemetryresourcefilter.NewResolver[qbtypes.LogAggregation](
|
||||
settings,
|
||||
logstelemetryschema.DBName,
|
||||
logstelemetryschema.LogsResourceV2TableName,
|
||||
DBName,
|
||||
LogsResourceV2TableName,
|
||||
telemetrytypes.SignalLogs,
|
||||
telemetrytypes.SourceUnspecified,
|
||||
metadataStore,
|
||||
@@ -185,7 +174,7 @@ func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation], bo
|
||||
// TODO(Piyush): Setup better for coming FTS support.
|
||||
if bodyJSONEnabled {
|
||||
for _, sel := range keySelectors {
|
||||
if sel.Name == logstelemetryschema.LogsV2BodyColumn {
|
||||
if sel.Name == LogsV2BodyColumn {
|
||||
warnings = append(warnings, bodySearchDefaultWarning)
|
||||
break
|
||||
}
|
||||
@@ -261,8 +250,8 @@ func (b *logQueryStatementBuilder) adjustKeys(ctx context.Context, keys map[stri
|
||||
func (b *logQueryStatementBuilder) adjustKey(key *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) []string {
|
||||
// First check if it matches with any intrinsic fields
|
||||
var intrinsicOrCalculatedField telemetrytypes.TelemetryFieldKey
|
||||
if _, ok := logstelemetryschema.IntrinsicFields[key.Name]; ok {
|
||||
intrinsicOrCalculatedField = logstelemetryschema.IntrinsicFields[key.Name]
|
||||
if _, ok := IntrinsicFields[key.Name]; ok {
|
||||
intrinsicOrCalculatedField = IntrinsicFields[key.Name]
|
||||
return querybuilder.AdjustKey(key, keys, &intrinsicOrCalculatedField)
|
||||
}
|
||||
|
||||
@@ -295,28 +284,28 @@ func (b *logQueryStatementBuilder) buildListQuery(
|
||||
}
|
||||
|
||||
// Select timestamp and id by default
|
||||
sb.Select(logstelemetryschema.LogsV2TimestampColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2IDColumn)
|
||||
sb.Select(LogsV2TimestampColumn)
|
||||
sb.SelectMore(LogsV2IDColumn)
|
||||
if len(query.SelectFields) == 0 {
|
||||
// Select all default columns
|
||||
sb.SelectMore(logstelemetryschema.LogsV2TraceIDColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2SpanIDColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2TraceFlagsColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2SeverityTextColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2SeverityNumberColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2ScopeNameColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2ScopeVersionColumn)
|
||||
sb.SelectMore(LogsV2TraceIDColumn)
|
||||
sb.SelectMore(LogsV2SpanIDColumn)
|
||||
sb.SelectMore(LogsV2TraceFlagsColumn)
|
||||
sb.SelectMore(LogsV2SeverityTextColumn)
|
||||
sb.SelectMore(LogsV2SeverityNumberColumn)
|
||||
sb.SelectMore(LogsV2ScopeNameColumn)
|
||||
sb.SelectMore(LogsV2ScopeVersionColumn)
|
||||
sb.SelectMore(bodyAliasExpression(b.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID))))
|
||||
sb.SelectMore(logstelemetryschema.LogsV2AttributesStringColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2AttributesNumberColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2AttributesBoolColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2ResourcesStringColumn)
|
||||
sb.SelectMore(logstelemetryschema.LogsV2ScopeStringColumn)
|
||||
sb.SelectMore(LogsV2AttributesStringColumn)
|
||||
sb.SelectMore(LogsV2AttributesNumberColumn)
|
||||
sb.SelectMore(LogsV2AttributesBoolColumn)
|
||||
sb.SelectMore(LogsV2ResourcesStringColumn)
|
||||
sb.SelectMore(LogsV2ScopeStringColumn)
|
||||
|
||||
} else {
|
||||
// Select specified columns
|
||||
for index := range query.SelectFields {
|
||||
if query.SelectFields[index].Name == logstelemetryschema.LogsV2TimestampColumn || query.SelectFields[index].Name == logstelemetryschema.LogsV2IDColumn {
|
||||
if query.SelectFields[index].Name == LogsV2TimestampColumn || query.SelectFields[index].Name == LogsV2IDColumn {
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -329,7 +318,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
|
||||
}
|
||||
}
|
||||
|
||||
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
|
||||
// Add filter conditions
|
||||
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
|
||||
|
||||
@@ -435,7 +424,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
|
||||
}
|
||||
|
||||
// Add FROM clause
|
||||
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
|
||||
|
||||
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
|
||||
|
||||
@@ -603,7 +592,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
|
||||
}
|
||||
}
|
||||
|
||||
sb.From(fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName))
|
||||
sb.From(fmt.Sprintf("%s.%s", DBName, LogsV2TableName))
|
||||
|
||||
// Add filter conditions
|
||||
preparedWhereClause, err := b.addFilterCondition(ctx, orgID, sb, start, end, query, keys, variables, skipResourceFilter)
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"testing"
|
||||
@@ -1,4 +1,4 @@
|
||||
package logsstatementbuilder
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
@@ -213,12 +212,12 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
|
||||
fl := flaggertest.New(t)
|
||||
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
keysMap := logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
|
||||
keysMap := buildCompleteFieldKeyMap(releaseTime)
|
||||
|
||||
mockMetadataStore.KeysMap = keysMap
|
||||
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
@@ -228,7 +227,7 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -356,12 +355,12 @@ func TestStatementBuilderListQuery(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fl := flaggertest.New(t)
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
|
||||
// Create a test release time
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
@@ -371,7 +370,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -508,11 +507,11 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fl := flaggertest.New(t)
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
// Create a test release time
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
@@ -522,7 +521,7 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -586,11 +585,11 @@ func TestStatementBuilderTimeSeriesBodyGroupBy(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fl := flaggertest.New(t)
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
// Create a test release time
|
||||
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
|
||||
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
@@ -600,7 +599,7 @@ func TestStatementBuilderTimeSeriesBodyGroupBy(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -685,9 +684,9 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
fl := flaggertest.New(t)
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMapCollision()
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
mockMetadataStore.KeysMap = buildCompleteFieldKeyMapCollision()
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
@@ -697,7 +696,7 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -741,8 +740,8 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: logstelemetryschema.IntrinsicFields["severity_text"],
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: IntrinsicFields["severity_text"],
|
||||
},
|
||||
{
|
||||
name: "intrinsic field with other key match - no override",
|
||||
@@ -778,7 +777,7 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextBody,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: telemetrytypes.TelemetryFieldKey{
|
||||
Name: "severity_number",
|
||||
FieldContext: telemetrytypes.FieldContextBody,
|
||||
@@ -792,8 +791,8 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["service.name"][0],
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["service.name"][0],
|
||||
},
|
||||
{
|
||||
name: "single matching key with incorrect context specified - no override",
|
||||
@@ -802,7 +801,7 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: telemetrytypes.TelemetryFieldKey{
|
||||
Name: "service.name",
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
@@ -816,8 +815,8 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["service.name"][0],
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["service.name"][0],
|
||||
},
|
||||
{
|
||||
name: "multiple matching keys - all materialized",
|
||||
@@ -826,7 +825,7 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: telemetrytypes.TelemetryFieldKey{
|
||||
Name: "multi.mat.key",
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
@@ -840,7 +839,7 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: telemetrytypes.TelemetryFieldKey{
|
||||
Name: "mixed.materialization.key",
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
@@ -854,8 +853,8 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["mixed.materialization.key"][0],
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["mixed.materialization.key"][0],
|
||||
},
|
||||
{
|
||||
name: "no matching keys - unknown field",
|
||||
@@ -864,7 +863,7 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: telemetrytypes.TelemetryFieldKey{
|
||||
Name: "unknown.field",
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
@@ -879,7 +878,7 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: telemetrytypes.TelemetryFieldKey{
|
||||
Name: "unknown.field",
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
@@ -894,8 +893,8 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["mat.key"][0],
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["mat.key"][0],
|
||||
},
|
||||
{
|
||||
name: "non-materialized field",
|
||||
@@ -904,16 +903,16 @@ func TestAdjustKey(t *testing.T) {
|
||||
FieldContext: telemetrytypes.FieldContextUnspecified,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeUnspecified,
|
||||
},
|
||||
keysMap: logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *logstelemetryschema.BuildCompleteFieldKeyMap(releaseTime)["user.id"][0],
|
||||
keysMap: buildCompleteFieldKeyMap(releaseTime),
|
||||
expectedKey: *buildCompleteFieldKeyMap(releaseTime)["user.id"][0],
|
||||
},
|
||||
}
|
||||
|
||||
fl := flaggertest.New(t)
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMapCollision()
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
mockMetadataStore.KeysMap = buildCompleteFieldKeyMapCollision()
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
|
||||
|
||||
@@ -923,7 +922,7 @@ func TestAdjustKey(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -1055,11 +1054,11 @@ func TestStmtBuilderBodyField(t *testing.T) {
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
fl := flaggertest.WithUseJSONBody(t, c.enableUseJSONBody)
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
// build the key map
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
for _, field := range logstelemetryschema.IntrinsicFields {
|
||||
for _, field := range IntrinsicFields {
|
||||
f := field
|
||||
mockMetadataStore.KeysMap[field.Name] = append(mockMetadataStore.KeysMap[field.Name], &f)
|
||||
}
|
||||
@@ -1070,7 +1069,7 @@ func TestStmtBuilderBodyField(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -1156,11 +1155,11 @@ func TestStmtBuilderBodyFullTextSearch(t *testing.T) {
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
fl := flaggertest.WithUseJSONBody(t, c.enableUseJSONBody)
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
// build the key map
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
for _, field := range logstelemetryschema.IntrinsicFields {
|
||||
for _, field := range IntrinsicFields {
|
||||
f := field
|
||||
mockMetadataStore.KeysMap[field.Name] = append(mockMetadataStore.KeysMap[field.Name], &f)
|
||||
}
|
||||
@@ -1171,7 +1170,7 @@ func TestStmtBuilderBodyFullTextSearch(t *testing.T) {
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
nil,
|
||||
false,
|
||||
@@ -1274,14 +1273,14 @@ func newSkipResourceFingerprintLogsBuilder(
|
||||
t.Helper()
|
||||
|
||||
fl := flaggertest.New(t)
|
||||
fm := logstelemetryschema.NewFieldMapper(fl)
|
||||
cb := logstelemetryschema.NewConditionBuilder(fm, fl)
|
||||
fm := NewFieldMapper(fl)
|
||||
cb := NewConditionBuilder(fm, fl)
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
mockMetadataStore.KeysMap = logstelemetryschema.BuildCompleteFieldKeyMap(time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC))
|
||||
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC))
|
||||
|
||||
aggExprRewriter := querybuilder.NewAggExprRewriter(
|
||||
instrumentationtest.New().ToProviderSettings(),
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fm,
|
||||
cb,
|
||||
fl,
|
||||
@@ -1293,7 +1292,7 @@ func newSkipResourceFingerprintLogsBuilder(
|
||||
fm,
|
||||
cb,
|
||||
aggExprRewriter,
|
||||
logstelemetryschema.DefaultFullTextColumn,
|
||||
DefaultFullTextColumn,
|
||||
fl,
|
||||
telemetryStore,
|
||||
skipEnable,
|
||||
15
pkg/telemetrylogs/tables.go
Normal file
15
pkg/telemetrylogs/tables.go
Normal file
@@ -0,0 +1,15 @@
|
||||
package telemetrylogs
|
||||
|
||||
const (
|
||||
DBName = "signoz_logs"
|
||||
LogsV2TableName = "distributed_logs_v2"
|
||||
LogsV2LocalTableName = "logs_v2"
|
||||
TagAttributesV2TableName = "distributed_tag_attributes_v2"
|
||||
TagAttributesV2LocalTableName = "tag_attributes_v2"
|
||||
LogAttributeKeysTblName = "distributed_logs_attribute_keys"
|
||||
LogResourceKeysTblName = "distributed_logs_resource_keys"
|
||||
LogsResourceV2TableName = "distributed_logs_v2_resource"
|
||||
PathTypesTableName = "distributed_json_path_types"
|
||||
PromotedPathsTableName = "distributed_json_promoted_paths"
|
||||
SkipIndexTableName = "system.data_skipping_indices"
|
||||
)
|
||||
@@ -1,4 +1,4 @@
|
||||
package logstelemetryschema
|
||||
package telemetrylogs
|
||||
|
||||
import (
|
||||
"strings"
|
||||
@@ -19,7 +19,7 @@ func limitString(s string, maxLen int) string {
|
||||
}
|
||||
|
||||
// Function to build a complete field key map for testing all scenarios.
|
||||
func BuildCompleteFieldKeyMap(releaseTime time.Time) map[string][]*telemetrytypes.TelemetryFieldKey {
|
||||
func buildCompleteFieldKeyMap(releaseTime time.Time) map[string][]*telemetrytypes.TelemetryFieldKey {
|
||||
keysMap := map[string][]*telemetrytypes.TelemetryFieldKey{
|
||||
"service.name": {
|
||||
{
|
||||
@@ -953,7 +953,7 @@ func BuildCompleteFieldKeyMap(releaseTime time.Time) map[string][]*telemetrytype
|
||||
return keysMap
|
||||
}
|
||||
|
||||
func BuildCompleteFieldKeyMapCollision() map[string][]*telemetrytypes.TelemetryFieldKey {
|
||||
func buildCompleteFieldKeyMapCollision() map[string][]*telemetrytypes.TelemetryFieldKey {
|
||||
keysMap := map[string][]*telemetrytypes.TelemetryFieldKey{
|
||||
"service.name": {
|
||||
{
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
"github.com/SigNoz/signoz-otel-collector/constants"
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
@@ -139,8 +139,8 @@ func buildListLogsJSONIndexesQuery(cluster string, filters ...string) (string, [
|
||||
"name", "type_full", "expr", "granularity",
|
||||
).From(fmt.Sprintf("clusterAllReplicas('%s', %s)", cluster, SkipIndexTableName))
|
||||
|
||||
sb.Where(sb.Equal("database", logstelemetryschema.DBName))
|
||||
sb.Where(sb.Equal("table", logstelemetryschema.LogsV2LocalTableName))
|
||||
sb.Where(sb.Equal("database", telemetrylogs.DBName))
|
||||
sb.Where(sb.Equal("table", telemetrylogs.LogsV2LocalTableName))
|
||||
sb.Where(sb.Or(
|
||||
sb.ILike("expr", fmt.Sprintf("%%%s%%", querybuilder.FormatValueForContains(constants.BodyV2ColumnPrefix))),
|
||||
sb.ILike("expr", fmt.Sprintf("%%%s%%", querybuilder.FormatValueForContains(constants.BodyPromotedColumnPrefix))),
|
||||
@@ -189,12 +189,12 @@ func (t *telemetryMetaStore) ListLogsJSONIndexes(ctx context.Context, filters ..
|
||||
baseColumn := ""
|
||||
fieldName := ""
|
||||
switch {
|
||||
case strings.HasPrefix(columnExpr, logstelemetryschema.BodyV2ColumnPrefix):
|
||||
baseColumn = logstelemetryschema.BodyV2ColumnPrefix
|
||||
fieldName = strings.TrimPrefix(columnExpr, logstelemetryschema.BodyV2ColumnPrefix)
|
||||
case strings.HasPrefix(columnExpr, logstelemetryschema.BodyPromotedColumnPrefix):
|
||||
baseColumn = logstelemetryschema.BodyPromotedColumnPrefix
|
||||
fieldName = strings.TrimPrefix(columnExpr, logstelemetryschema.BodyPromotedColumnPrefix)
|
||||
case strings.HasPrefix(columnExpr, telemetrylogs.BodyV2ColumnPrefix):
|
||||
baseColumn = telemetrylogs.BodyV2ColumnPrefix
|
||||
fieldName = strings.TrimPrefix(columnExpr, telemetrylogs.BodyV2ColumnPrefix)
|
||||
case strings.HasPrefix(columnExpr, telemetrylogs.BodyPromotedColumnPrefix):
|
||||
baseColumn = telemetrylogs.BodyPromotedColumnPrefix
|
||||
fieldName = strings.TrimPrefix(columnExpr, telemetrylogs.BodyPromotedColumnPrefix)
|
||||
}
|
||||
fieldName = strings.ReplaceAll(fieldName, "`", "")
|
||||
|
||||
@@ -228,12 +228,12 @@ func (t *telemetryMetaStore) ListJSONValues(ctx context.Context, path string, li
|
||||
}
|
||||
|
||||
if promoted {
|
||||
path = logstelemetryschema.BodyPromotedColumnPrefix + path
|
||||
path = telemetrylogs.BodyPromotedColumnPrefix + path
|
||||
} else {
|
||||
path = logstelemetryschema.BodyV2ColumnPrefix + path
|
||||
path = telemetrylogs.BodyV2ColumnPrefix + path
|
||||
}
|
||||
|
||||
from := fmt.Sprintf("%s.%s", logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName)
|
||||
from := fmt.Sprintf("%s.%s", telemetrylogs.DBName, telemetrylogs.LogsV2TableName)
|
||||
colExpr := func(typ telemetrytypes.JSONDataType) string {
|
||||
return fmt.Sprintf("dynamicElement(%s, '%s')", path, typ.StringValue())
|
||||
}
|
||||
@@ -382,7 +382,7 @@ func (t *telemetryMetaStore) IsPathPromoted(ctx context.Context, path string) (b
|
||||
split := strings.Split(path, telemetrytypes.ArraySep)
|
||||
pathSegment := split[0]
|
||||
query := fmt.Sprintf("SELECT 1 FROM %s.%s WHERE signal = ? AND column_name = ? AND field_context = ? AND field_name = ? LIMIT 1", DBName, PromotedPathsTableName)
|
||||
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, telemetrytypes.SignalLogs, logstelemetryschema.LogsV2BodyPromotedColumn, telemetrytypes.FieldContextBody, pathSegment)
|
||||
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, telemetrytypes.SignalLogs, telemetrylogs.LogsV2BodyPromotedColumn, telemetrytypes.FieldContextBody, pathSegment)
|
||||
if err != nil {
|
||||
return false, errors.WrapInternalf(err, CodeFailCheckPathPromoted, "failed to check if path %s is promoted", path)
|
||||
}
|
||||
@@ -397,7 +397,7 @@ func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, paths ...stri
|
||||
sb := sqlbuilder.Select("field_name").From(fmt.Sprintf("%s.%s", DBName, PromotedPathsTableName))
|
||||
conditions := []string{
|
||||
sb.Equal("signal", telemetrytypes.SignalLogs),
|
||||
sb.Equal("column_name", logstelemetryschema.LogsV2BodyPromotedColumn),
|
||||
sb.Equal("column_name", telemetrylogs.LogsV2BodyPromotedColumn),
|
||||
sb.Equal("field_context", telemetrytypes.FieldContextBody),
|
||||
sb.NotEqual("field_name", "__all__"),
|
||||
}
|
||||
@@ -433,8 +433,8 @@ func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, paths ...stri
|
||||
// TODO(Piyush): Remove this function.
|
||||
func CleanPathPrefixes(path string) string {
|
||||
path = strings.TrimPrefix(path, telemetrytypes.BodyJSONStringSearchPrefix)
|
||||
path = strings.TrimPrefix(path, logstelemetryschema.BodyV2ColumnPrefix)
|
||||
path = strings.TrimPrefix(path, logstelemetryschema.BodyPromotedColumnPrefix)
|
||||
path = strings.TrimPrefix(path, telemetrylogs.BodyV2ColumnPrefix)
|
||||
path = strings.TrimPrefix(path, telemetrylogs.BodyPromotedColumnPrefix)
|
||||
return path
|
||||
}
|
||||
|
||||
@@ -454,7 +454,7 @@ func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string)
|
||||
if trimmed == "" {
|
||||
continue
|
||||
}
|
||||
if err := batch.Append(telemetrytypes.SignalLogs, logstelemetryschema.LogsV2BodyPromotedColumn, "JSON()", telemetrytypes.FieldContextBody, trimmed, 0, releaseTime); err != nil {
|
||||
if err := batch.Append(telemetrytypes.SignalLogs, telemetrylogs.LogsV2BodyPromotedColumn, "JSON()", telemetrytypes.FieldContextBody, trimmed, 0, releaseTime); err != nil {
|
||||
_ = batch.Abort()
|
||||
return errors.WrapInternalf(err, CodeFailedToAppendPath, "failed to append path")
|
||||
}
|
||||
|
||||
@@ -6,7 +6,7 @@ import (
|
||||
|
||||
"github.com/SigNoz/signoz-otel-collector/constants"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
@@ -25,8 +25,8 @@ func TestBuildListLogsJSONIndexesQuery(t *testing.T) {
|
||||
expectedSQL: "SELECT name, type_full, expr, granularity FROM clusterAllReplicas('test-cluster', system.data_skipping_indices) " +
|
||||
"WHERE database = ? AND table = ? AND (LOWER(expr) LIKE LOWER(?) OR LOWER(expr) LIKE LOWER(?))",
|
||||
expectedArgs: []any{
|
||||
logstelemetryschema.DBName,
|
||||
logstelemetryschema.LogsV2LocalTableName,
|
||||
telemetrylogs.DBName,
|
||||
telemetrylogs.LogsV2LocalTableName,
|
||||
fmt.Sprintf("%%%s%%", querybuilder.FormatValueForContains(constants.BodyV2ColumnPrefix)),
|
||||
fmt.Sprintf("%%%s%%", querybuilder.FormatValueForContains(constants.BodyPromotedColumnPrefix)),
|
||||
},
|
||||
@@ -38,8 +38,8 @@ func TestBuildListLogsJSONIndexesQuery(t *testing.T) {
|
||||
expectedSQL: "SELECT name, type_full, expr, granularity FROM clusterAllReplicas('test-cluster', system.data_skipping_indices) " +
|
||||
"WHERE database = ? AND table = ? AND (LOWER(expr) LIKE LOWER(?) OR LOWER(expr) LIKE LOWER(?)) AND (LOWER(replaceAll(expr, '`', '')) LIKE LOWER(?) OR LOWER(replaceAll(expr, '`', '')) LIKE LOWER(?))",
|
||||
expectedArgs: []any{
|
||||
logstelemetryschema.DBName,
|
||||
logstelemetryschema.LogsV2LocalTableName,
|
||||
telemetrylogs.DBName,
|
||||
telemetrylogs.LogsV2LocalTableName,
|
||||
fmt.Sprintf("%%%s%%", querybuilder.FormatValueForContains(constants.BodyV2ColumnPrefix)),
|
||||
fmt.Sprintf("%%%s%%", querybuilder.FormatValueForContains(constants.BodyPromotedColumnPrefix)),
|
||||
fmt.Sprintf("%%%s%%", querybuilder.FormatValueForContains("foo")),
|
||||
|
||||
@@ -14,12 +14,11 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryaudit"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/featuretypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
|
||||
@@ -78,6 +77,27 @@ func escapeForLike(s string) string {
|
||||
func NewTelemetryMetaStore(
|
||||
settings factory.ProviderSettings,
|
||||
telemetrystore telemetrystore.TelemetryStore,
|
||||
tracesDBName string,
|
||||
tracesFieldsTblName string,
|
||||
spanAttributesKeysTblName string,
|
||||
indexV3TblName string,
|
||||
metricsDBName string,
|
||||
metricsFieldsTblName string,
|
||||
meterDBName string,
|
||||
meterFieldsTblName string,
|
||||
logsDBName string,
|
||||
logsV2TblName string,
|
||||
logsFieldsTblName string,
|
||||
logAttributeKeysTblName string,
|
||||
logResourceKeysTblName string,
|
||||
auditDBName string,
|
||||
auditLogsTblName string,
|
||||
auditFieldsTblName string,
|
||||
auditAttributeKeysTblName string,
|
||||
auditResourceKeysTblName string,
|
||||
relatedMetadataDBName string,
|
||||
relatedMetadataTblName string,
|
||||
columnEvolutionMetadataTblName string,
|
||||
fl flagger.Flagger,
|
||||
) telemetrytypes.MetadataStore {
|
||||
metadataSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrymetadata")
|
||||
@@ -88,32 +108,32 @@ func NewTelemetryMetaStore(
|
||||
t := &telemetryMetaStore{
|
||||
logger: metadataSettings.Logger(),
|
||||
telemetrystore: telemetrystore,
|
||||
tracesDBName: tracestelemetryschema.DBName,
|
||||
tracesFieldsTblName: tracestelemetryschema.TagAttributesV2TableName,
|
||||
spanAttributesKeysTblName: tracestelemetryschema.SpanAttributesKeysTblName,
|
||||
indexV3TblName: tracestelemetryschema.SpanIndexV3TableName,
|
||||
metricsDBName: metricstelemetryschema.DBName,
|
||||
metricsFieldsTblName: metricstelemetryschema.AttributesMetadataTableName,
|
||||
meterDBName: metertelemetryschema.DBName,
|
||||
meterFieldsTblName: metertelemetryschema.SamplesAgg1dTableName,
|
||||
logsDBName: logstelemetryschema.DBName,
|
||||
logsV2TblName: logstelemetryschema.LogsV2TableName,
|
||||
logsFieldsTblName: logstelemetryschema.TagAttributesV2TableName,
|
||||
logAttributeKeysTblName: logstelemetryschema.LogAttributeKeysTblName,
|
||||
logResourceKeysTblName: logstelemetryschema.LogResourceKeysTblName,
|
||||
auditDBName: audittelemetryschema.DBName,
|
||||
auditLogsTblName: audittelemetryschema.AuditLogsTableName,
|
||||
auditFieldsTblName: audittelemetryschema.TagAttributesTableName,
|
||||
auditAttributeKeysTblName: audittelemetryschema.LogAttributeKeysTblName,
|
||||
auditResourceKeysTblName: audittelemetryschema.LogResourceKeysTblName,
|
||||
relatedMetadataDBName: DBName,
|
||||
relatedMetadataTblName: AttributesMetadataTableName,
|
||||
columnEvolutionMetadataTblName: ColumnEvolutionMetadataTableName,
|
||||
tracesDBName: tracesDBName,
|
||||
tracesFieldsTblName: tracesFieldsTblName,
|
||||
spanAttributesKeysTblName: spanAttributesKeysTblName,
|
||||
indexV3TblName: indexV3TblName,
|
||||
metricsDBName: metricsDBName,
|
||||
metricsFieldsTblName: metricsFieldsTblName,
|
||||
meterDBName: meterDBName,
|
||||
meterFieldsTblName: meterFieldsTblName,
|
||||
logsDBName: logsDBName,
|
||||
logsV2TblName: logsV2TblName,
|
||||
logsFieldsTblName: logsFieldsTblName,
|
||||
logAttributeKeysTblName: logAttributeKeysTblName,
|
||||
logResourceKeysTblName: logResourceKeysTblName,
|
||||
auditDBName: auditDBName,
|
||||
auditLogsTblName: auditLogsTblName,
|
||||
auditFieldsTblName: auditFieldsTblName,
|
||||
auditAttributeKeysTblName: auditAttributeKeysTblName,
|
||||
auditResourceKeysTblName: auditResourceKeysTblName,
|
||||
relatedMetadataDBName: relatedMetadataDBName,
|
||||
relatedMetadataTblName: relatedMetadataTblName,
|
||||
columnEvolutionMetadataTblName: columnEvolutionMetadataTblName,
|
||||
jsonColumnMetadata: map[telemetrytypes.Signal]map[telemetrytypes.FieldContext]telemetrytypes.JSONColumnMetadata{
|
||||
telemetrytypes.SignalLogs: {
|
||||
telemetrytypes.FieldContextBody: telemetrytypes.JSONColumnMetadata{
|
||||
BaseColumn: logstelemetryschema.LogsV2BodyV2Column,
|
||||
PromotedColumn: logstelemetryschema.LogsV2BodyPromotedColumn,
|
||||
BaseColumn: telemetrylogs.LogsV2BodyV2Column,
|
||||
PromotedColumn: telemetrylogs.LogsV2BodyPromotedColumn,
|
||||
},
|
||||
},
|
||||
},
|
||||
@@ -289,8 +309,8 @@ func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelector
|
||||
complete := rowCount <= limit
|
||||
|
||||
staticKeys := []string{"isRoot", "isEntryPoint"}
|
||||
staticKeys = append(staticKeys, maps.Keys(tracestelemetryschema.IntrinsicFields)...)
|
||||
staticKeys = append(staticKeys, maps.Keys(tracestelemetryschema.CalculatedFields)...)
|
||||
staticKeys = append(staticKeys, maps.Keys(telemetrytraces.IntrinsicFields)...)
|
||||
staticKeys = append(staticKeys, maps.Keys(telemetrytraces.CalculatedFields)...)
|
||||
|
||||
// Add matching intrinsic and matching calculated fields
|
||||
// These don't count towards the limit
|
||||
@@ -304,14 +324,14 @@ func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelector
|
||||
}
|
||||
|
||||
if found {
|
||||
if field, exists := tracestelemetryschema.IntrinsicFields[key]; exists {
|
||||
if field, exists := telemetrytraces.IntrinsicFields[key]; exists {
|
||||
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
|
||||
keys = append(keys, &field)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if field, exists := tracestelemetryschema.CalculatedFields[key]; exists {
|
||||
if field, exists := telemetrytraces.CalculatedFields[key]; exists {
|
||||
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
|
||||
keys = append(keys, &field)
|
||||
}
|
||||
@@ -636,7 +656,7 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, orgID valuer.UUID,
|
||||
complete := rowCount <= limit
|
||||
|
||||
staticKeys := []string{}
|
||||
staticKeys = append(staticKeys, maps.Keys(logstelemetryschema.IntrinsicFields)...)
|
||||
staticKeys = append(staticKeys, maps.Keys(telemetrylogs.IntrinsicFields)...)
|
||||
|
||||
// Add matching intrinsic and matching calculated fields
|
||||
// These don't count towards the limit
|
||||
@@ -650,7 +670,7 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, orgID valuer.UUID,
|
||||
}
|
||||
|
||||
if found {
|
||||
if field, exists := logstelemetryschema.IntrinsicFields[key]; exists {
|
||||
if field, exists := telemetrylogs.IntrinsicFields[key]; exists {
|
||||
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
|
||||
keys = append(keys, &field)
|
||||
}
|
||||
@@ -878,7 +898,7 @@ func (t *telemetryMetaStore) getAuditKeys(ctx context.Context, fieldKeySelectors
|
||||
complete := rowCount <= limit
|
||||
|
||||
// Add intrinsic audit fields (same as logs intrinsics: body, severity_text, etc.)
|
||||
staticKeys := maps.Keys(audittelemetryschema.IntrinsicFields)
|
||||
staticKeys := maps.Keys(telemetryaudit.IntrinsicFields)
|
||||
for _, key := range staticKeys {
|
||||
found := false
|
||||
for _, v := range searchTexts {
|
||||
@@ -889,7 +909,7 @@ func (t *telemetryMetaStore) getAuditKeys(ctx context.Context, fieldKeySelectors
|
||||
}
|
||||
|
||||
if found {
|
||||
if field, exists := audittelemetryschema.IntrinsicFields[key]; exists {
|
||||
if field, exists := telemetryaudit.IntrinsicFields[key]; exists {
|
||||
if _, added := mapOfKeys[field.Name+";"+field.FieldContext.StringValue()+";"+field.FieldDataType.StringValue()]; !added {
|
||||
keys = append(keys, &field)
|
||||
}
|
||||
@@ -1156,7 +1176,7 @@ func enrichWithIntrinsicMetricKeys(keys map[string][]*telemetrytypes.TelemetryFi
|
||||
continue
|
||||
}
|
||||
|
||||
for name, key := range metricstelemetryschema.IntrinsicMetricFieldDefinitions {
|
||||
for name, key := range telemetrymetrics.IntrinsicMetricFieldDefinitions {
|
||||
if !selectorMatchesIntrinsicField(selector, key) {
|
||||
continue
|
||||
}
|
||||
@@ -1872,7 +1892,7 @@ func (t *telemetryMetaStore) getIntrinsicMetricFieldValues(ctx context.Context,
|
||||
instrumentationtypes.CodeNamespace: "metadata",
|
||||
instrumentationtypes.CodeFunctionName: "getIntrinsicMetricFieldValues",
|
||||
})
|
||||
key, ok := metricstelemetryschema.IntrinsicMetricFieldDefinitions[fieldValueSelector.Name]
|
||||
key, ok := telemetrymetrics.IntrinsicMetricFieldDefinitions[fieldValueSelector.Name]
|
||||
if !ok {
|
||||
return &telemetrytypes.TelemetryFieldValues{}, nil
|
||||
}
|
||||
@@ -1902,18 +1922,18 @@ func (t *telemetryMetaStore) getIntrinsicMetricFieldValues(ctx context.Context,
|
||||
tableName string
|
||||
extraConds []string
|
||||
}{
|
||||
{metricstelemetryschema.TimeseriesV41weekTableName, nil},
|
||||
{telemetrymetrics.TimeseriesV41weekTableName, nil},
|
||||
}
|
||||
if reductionEnabled {
|
||||
sources = append(sources,
|
||||
struct {
|
||||
tableName string
|
||||
extraConds []string
|
||||
}{metricstelemetryschema.TimeseriesV4BufferTableName, []string{"is_reduced = false"}},
|
||||
}{telemetrymetrics.TimeseriesV4BufferTableName, []string{"is_reduced = false"}},
|
||||
struct {
|
||||
tableName string
|
||||
extraConds []string
|
||||
}{metricstelemetryschema.TimeseriesV4ReducedTableName, nil},
|
||||
}{telemetrymetrics.TimeseriesV4ReducedTableName, nil},
|
||||
)
|
||||
}
|
||||
|
||||
@@ -2242,7 +2262,7 @@ func (t *telemetryMetaStore) fetchMetricsTemporalityAndType(ctx context.Context,
|
||||
instrumentationtypes.CodeNamespace: "metadata",
|
||||
instrumentationtypes.CodeFunctionName: "fetchMetricsTemporalityAndType",
|
||||
})
|
||||
adjustedStartTs, adjustedEndTs, tsTableName, _ := metricstelemetryschema.WhichTSTableToUse(queryTimeRangeStartTs, queryTimeRangeEndTs, false, nil)
|
||||
adjustedStartTs, adjustedEndTs, tsTableName, _ := telemetrymetrics.WhichTSTableToUse(queryTimeRangeStartTs, queryTimeRangeEndTs, false, nil)
|
||||
|
||||
temporalities, types, err := t.fetchTemporalityTypeForTable(ctx, tsTableName, adjustedStartTs, adjustedEndTs, metricNames)
|
||||
if err != nil {
|
||||
@@ -2251,7 +2271,7 @@ func (t *telemetryMetaStore) fetchMetricsTemporalityAndType(ctx context.Context,
|
||||
|
||||
reduced := make(map[string]bool)
|
||||
if reductionEnabled {
|
||||
bufferTemporalities, bufferTypes, err := t.fetchTemporalityTypeForTable(ctx, metricstelemetryschema.TimeseriesV4BufferTableName, adjustedStartTs, adjustedEndTs, metricNames, "is_reduced = false")
|
||||
bufferTemporalities, bufferTypes, err := t.fetchTemporalityTypeForTable(ctx, telemetrymetrics.TimeseriesV4BufferTableName, adjustedStartTs, adjustedEndTs, metricNames, "is_reduced = false")
|
||||
if err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
@@ -2264,7 +2284,7 @@ func (t *telemetryMetaStore) fetchMetricsTemporalityAndType(ctx context.Context,
|
||||
}
|
||||
}
|
||||
|
||||
reducedTemporalities, reducedTypes, err := t.fetchTemporalityTypeForTable(ctx, metricstelemetryschema.TimeseriesV4ReducedTableName, adjustedStartTs, adjustedEndTs, metricNames)
|
||||
reducedTemporalities, reducedTypes, err := t.fetchTemporalityTypeForTable(ctx, telemetrymetrics.TimeseriesV4ReducedTableName, adjustedStartTs, adjustedEndTs, metricNames)
|
||||
if err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
@@ -2596,13 +2616,13 @@ func (t *telemetryMetaStore) GetFirstSeenFromMetricMetadata(ctx context.Context,
|
||||
}
|
||||
|
||||
func (t *telemetryMetaStore) FetchLastSeenInfoMulti(ctx context.Context, orgID valuer.UUID, metricNames ...string) (map[string]int64, error) {
|
||||
lastSeenInfo, err := t.fetchLastSeenInfoForTable(ctx, metricstelemetryschema.TimeseriesV4TableName, metricNames)
|
||||
lastSeenInfo, err := t.fetchLastSeenInfoForTable(ctx, telemetrymetrics.TimeseriesV4TableName, metricNames)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if t.fl.BooleanOrEmpty(ctx, flagger.FeatureEnableMetricsReduction, featuretypes.NewFlaggerEvaluationContext(orgID)) {
|
||||
reducedLastSeen, err := t.fetchLastSeenInfoForTable(ctx, metricstelemetryschema.TimeseriesV4ReducedTableName, metricNames)
|
||||
reducedLastSeen, err := t.fetchLastSeenInfoForTable(ctx, telemetrymetrics.TimeseriesV4ReducedTableName, metricNames)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -9,8 +9,13 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryaudit"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymeter"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrytraces"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -37,6 +42,27 @@ func TestGetFirstSeenFromMetricMetadata(t *testing.T) {
|
||||
metadata := NewTelemetryMetaStore(
|
||||
instrumentationtest.New().ToProviderSettings(),
|
||||
mockTelemetryStore,
|
||||
telemetrytraces.DBName,
|
||||
telemetrytraces.TagAttributesV2TableName,
|
||||
telemetrytraces.SpanAttributesKeysTblName,
|
||||
telemetrytraces.SpanIndexV3TableName,
|
||||
telemetrymetrics.DBName,
|
||||
telemetrymetrics.AttributesMetadataTableName,
|
||||
telemetrymeter.DBName,
|
||||
telemetrymeter.SamplesAgg1dTableName,
|
||||
telemetrylogs.DBName,
|
||||
telemetrylogs.LogsV2TableName,
|
||||
telemetrylogs.TagAttributesV2TableName,
|
||||
telemetrylogs.LogAttributeKeysTblName,
|
||||
telemetrylogs.LogResourceKeysTblName,
|
||||
telemetryaudit.DBName,
|
||||
telemetryaudit.AuditLogsTableName,
|
||||
telemetryaudit.TagAttributesTableName,
|
||||
telemetryaudit.LogAttributeKeysTblName,
|
||||
telemetryaudit.LogResourceKeysTblName,
|
||||
DBName,
|
||||
AttributesMetadataTableName,
|
||||
ColumnEvolutionMetadataTableName,
|
||||
flaggertest.New(t),
|
||||
)
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
package meterstatementbuilder
|
||||
package telemetrymeter
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -8,8 +8,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/types/metrictypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
@@ -22,7 +21,7 @@ type meterQueryStatementBuilder struct {
|
||||
metadataStore telemetrytypes.MetadataStore
|
||||
fm qbtypes.FieldMapper
|
||||
cb qbtypes.ConditionBuilder
|
||||
metricsStatementBuilder *metricsstatementbuilder.StatementBuilder
|
||||
metricsStatementBuilder *telemetrymetrics.MetricQueryStatementBuilder
|
||||
}
|
||||
|
||||
var _ qbtypes.StatementBuilder[qbtypes.MetricAggregation] = (*meterQueryStatementBuilder)(nil)
|
||||
@@ -32,9 +31,9 @@ func NewMeterQueryStatementBuilder(
|
||||
metadataStore telemetrytypes.MetadataStore,
|
||||
fieldMapper qbtypes.FieldMapper,
|
||||
conditionBuilder qbtypes.ConditionBuilder,
|
||||
metricsStatementBuilder *metricsstatementbuilder.StatementBuilder,
|
||||
metricsStatementBuilder *telemetrymetrics.MetricQueryStatementBuilder,
|
||||
) *meterQueryStatementBuilder {
|
||||
metricsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema")
|
||||
metricsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrymeter")
|
||||
|
||||
return &meterQueryStatementBuilder{
|
||||
logger: metricsSettings.Logger(),
|
||||
@@ -54,7 +53,7 @@ func (b *meterQueryStatementBuilder) Build(
|
||||
query qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation],
|
||||
variables map[string]qbtypes.VariableItem,
|
||||
) (*qbtypes.Statement, error) {
|
||||
keySelectors := metricsstatementbuilder.GetKeySelectors(query)
|
||||
keySelectors := telemetrymetrics.GetKeySelectors(query)
|
||||
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, keySelectors)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -134,8 +133,8 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
|
||||
sb.SelectMore(col)
|
||||
}
|
||||
|
||||
tbl := metertelemetryschema.WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
aggCol, err := metertelemetryschema.AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
tbl := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
@@ -144,7 +143,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDeltaFastPath(
|
||||
}
|
||||
|
||||
sb.SelectMore(fmt.Sprintf("%s AS value", aggCol))
|
||||
sb.From(fmt.Sprintf("%s.%s AS points", metertelemetryschema.DBName, tbl))
|
||||
sb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
|
||||
sb.Where(
|
||||
sb.In("metric_name", query.Aggregations[0].MetricName),
|
||||
sb.GTE("unix_milli", start),
|
||||
@@ -223,8 +222,8 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
|
||||
sb.SelectMore(col)
|
||||
}
|
||||
|
||||
tbl := metertelemetryschema.WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
aggCol, err := metertelemetryschema.AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality,
|
||||
tbl := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality,
|
||||
query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
@@ -235,7 +234,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggDelta(
|
||||
|
||||
sb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
|
||||
|
||||
sb.From(fmt.Sprintf("%s.%s AS points", metertelemetryschema.DBName, tbl))
|
||||
sb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
|
||||
sb.Where(
|
||||
sb.In("metric_name", query.Aggregations[0].MetricName),
|
||||
sb.GTE("unix_milli", start),
|
||||
@@ -301,14 +300,14 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
|
||||
baseSb.SelectMore(col)
|
||||
}
|
||||
|
||||
tbl := metertelemetryschema.WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
aggCol, err := metertelemetryschema.AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
tbl := WhichSamplesTableToUse(start, end, query.Aggregations[0].Type, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
aggCol, err := AggregationColumnForSamplesTable(start, end, query.Aggregations[0].Type, query.Aggregations[0].Temporality, query.Aggregations[0].TimeAggregation, query.Aggregations[0].TableHints)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
baseSb.SelectMore(fmt.Sprintf("%s AS per_series_value", aggCol))
|
||||
|
||||
baseSb.From(fmt.Sprintf("%s.%s AS points", metertelemetryschema.DBName, tbl))
|
||||
baseSb.From(fmt.Sprintf("%s.%s AS points", DBName, tbl))
|
||||
baseSb.Where(
|
||||
baseSb.In("metric_name", query.Aggregations[0].MetricName),
|
||||
baseSb.GTE("unix_milli", start),
|
||||
@@ -351,7 +350,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
|
||||
for _, g := range query.GroupBy {
|
||||
wrapped.SelectMore(fmt.Sprintf("`%s`", g.Name))
|
||||
}
|
||||
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", metricsstatementbuilder.RateTmpl))
|
||||
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", telemetrymetrics.RateTmpl))
|
||||
wrapped.From(fmt.Sprintf("(%s) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)", innerQuery))
|
||||
q, args := wrapped.BuildWithFlavor(sqlbuilder.ClickHouse, innerArgs...)
|
||||
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", q), args, nil
|
||||
@@ -362,7 +361,7 @@ func (b *meterQueryStatementBuilder) buildTemporalAggCumulativeOrUnspecified(
|
||||
for _, g := range query.GroupBy {
|
||||
wrapped.SelectMore(fmt.Sprintf("`%s`", g.Name))
|
||||
}
|
||||
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", metricsstatementbuilder.IncreaseTmpl))
|
||||
wrapped.SelectMore(fmt.Sprintf("%s AS per_series_value", telemetrymetrics.IncreaseTmpl))
|
||||
wrapped.From(fmt.Sprintf("(%s) WINDOW rate_window AS (PARTITION BY fingerprint ORDER BY fingerprint, ts)", innerQuery))
|
||||
q, args := wrapped.BuildWithFlavor(sqlbuilder.ClickHouse, innerArgs...)
|
||||
return fmt.Sprintf("__temporal_aggregation_cte AS (%s)", q), args, nil
|
||||
@@ -1,4 +1,4 @@
|
||||
package meterstatementbuilder
|
||||
package telemetrymeter
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -7,8 +7,7 @@ import (
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
|
||||
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
|
||||
"github.com/SigNoz/signoz/pkg/statementbuilder/metricsstatementbuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metricstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
|
||||
"github.com/SigNoz/signoz/pkg/types/metrictypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
@@ -159,8 +158,8 @@ func TestStatementBuilder(t *testing.T) {
|
||||
},
|
||||
}
|
||||
|
||||
fm := metricstelemetryschema.NewFieldMapper()
|
||||
cb := metricstelemetryschema.NewConditionBuilder(fm)
|
||||
fm := telemetrymetrics.NewFieldMapper()
|
||||
cb := telemetrymetrics.NewConditionBuilder(fm)
|
||||
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
|
||||
keys, err := telemetrytypestest.LoadFieldKeysFromJSON("testdata/keys_map.json")
|
||||
if err != nil {
|
||||
@@ -170,7 +169,7 @@ func TestStatementBuilder(t *testing.T) {
|
||||
|
||||
flagger := flaggertest.New(t)
|
||||
|
||||
metricStmtBuilder := metricsstatementbuilder.NewMetricQueryStatementBuilder(instrumentationtest.New().ToProviderSettings(), mockMetadataStore, fm, cb, flagger)
|
||||
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(instrumentationtest.New().ToProviderSettings(), mockMetadataStore, fm, cb, flagger)
|
||||
|
||||
statementBuilder := NewMeterQueryStatementBuilder(
|
||||
instrumentationtest.New().ToProviderSettings(),
|
||||
@@ -1,4 +1,4 @@
|
||||
package metertelemetryschema
|
||||
package telemetrymeter
|
||||
|
||||
import (
|
||||
"time"
|
||||
@@ -1,4 +1,4 @@
|
||||
package metricstelemetryschema
|
||||
package telemetrymetrics
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -1,4 +1,4 @@
|
||||
package metricstelemetryschema
|
||||
package telemetrymetrics
|
||||
|
||||
import (
|
||||
"context"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user