mirror of
https://github.com/SigNoz/signoz.git
synced 2026-08-09 22:50:38 +01:00
Compare commits
14 Commits
issue_5601
...
proto/logi
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
17b73fa357 | ||
|
|
0c3223b23a | ||
|
|
a81c7d3f97 | ||
|
|
b2ff5ef99c | ||
|
|
58c21637a1 | ||
|
|
80fd5cc38a | ||
|
|
fa05a73aef | ||
|
|
38cc4d2bea | ||
|
|
e0b278e8e2 | ||
|
|
a711cda7ba | ||
|
|
e08ef01170 | ||
|
|
00b7ecbd71 | ||
|
|
7243560d8d | ||
|
|
53ab4546bc |
11
.claude/rules/comments.md
Normal file
11
.claude/rules/comments.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# Comments
|
||||
|
||||
Applies to everything in the repo — code, config, workflows.
|
||||
|
||||
- **No unnecessary comments.** Do not comment where the code is self-explanatory; never restate what the code already says.
|
||||
- **Document only** non-obvious behavior, constraints, formats, and edge cases.
|
||||
- **Rationale goes in prose, not source.** Why a version is pinned, why a job exists, how a subsystem fits together — that belongs in the README or the PR.
|
||||
- **Never remove pre-existing comments** when editing code. The bar above applies to comments you write, not comments already there.
|
||||
- **Never talk to the reviewer.** No comments about where a change came from, what was changed, or why the change is correct — that belongs in the PR description and is noise the moment it merges.
|
||||
|
||||
Language rules build on this one: [`go-comments`](go-comments.md), [`py-comments`](py-comments.md).
|
||||
12
.claude/rules/go-comments.md
Normal file
12
.claude/rules/go-comments.md
Normal file
@@ -0,0 +1,12 @@
|
||||
---
|
||||
paths:
|
||||
- "**/*.go"
|
||||
---
|
||||
|
||||
# Go comments
|
||||
|
||||
The bar is the [`comments`](comments.md) rule: nothing where the code is self-explanatory.
|
||||
|
||||
- **Names carry the meaning.** Make function, type, and variable names self-explanatory so the comment is unnecessary in the first place. If a comment is needed to explain what a function does, fix the name, not the comment.
|
||||
- **Godoc**: Skip comments that merely restate the identifier. Document only non-obvious behavior, constraints, formats, and edge cases.
|
||||
- **Generated code**: If the comment is emitted by an external codegen tool, leave it as-is — do not add or trim comments in generated files.
|
||||
7
.claude/rules/pull-requests.md
Normal file
7
.claude/rules/pull-requests.md
Normal file
@@ -0,0 +1,7 @@
|
||||
# Pull requests
|
||||
|
||||
- **Follow the template** (`.github/pull_request_template.md`): fill in its headings (Description / Issues closed by this PR / Screenshots / Additional Information). Don't add sections the template doesn't have.
|
||||
- **Keep only the headings that apply.** Delete every heading that has nothing under it, along with its `<!--...-->` placeholder comment. The body must never contain an empty heading — if only Description applies, the body has exactly that one heading.
|
||||
- **Keep the description concise and human-readable.** A few plain bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate.
|
||||
- **Reference issues with `Closes #issue-number`** under "Issues closed by this PR" so they auto-close on merge. This goes in the PR description only — never in commit messages.
|
||||
- **AI assistance in commits may optionally be disclosed with an `Assisted-by:` trailer** naming the model (e.g. `Assisted-by: Claude Opus 4.5`) — do NOT use a `Co-authored-by:` trailer for this.
|
||||
13
.claude/rules/py-comments.md
Normal file
13
.claude/rules/py-comments.md
Normal file
@@ -0,0 +1,13 @@
|
||||
---
|
||||
paths:
|
||||
- "**/*.py"
|
||||
---
|
||||
|
||||
# Python comments
|
||||
|
||||
The bar is the [`comments`](comments.md) rule: nothing where the code is self-explanatory.
|
||||
|
||||
- **Names carry the meaning.** Make function and variable names self-explanatory so the comment or docstring is unnecessary in the first place. If a docstring is needed to explain what a function does, fix the name, not the docstring.
|
||||
- **No file-level docstring.** The filename says what the module is for — `tool_bin.py` gets the tool binary. A module docstring restating that is noise, and a paragraph of design prose at the top of a file goes stale where nobody is looking. A constraint belongs next to the code it constrains, not in a preamble.
|
||||
- **Docstrings**: only when they say something the name and signature don't — drop them otherwise. Keep them short. A contract that genuinely needs a few lines (interacting flags, retry semantics, an edge case) is fine; a narrative is not.
|
||||
- **No song and dance.** Comment the constraint or the edge case. Not the narrative, not the rationale, not what the next line does.
|
||||
19
.claude/rules/pytest.md
Normal file
19
.claude/rules/pytest.md
Normal file
@@ -0,0 +1,19 @@
|
||||
---
|
||||
paths:
|
||||
- "tests/**/*.py"
|
||||
---
|
||||
|
||||
# pytest conventions
|
||||
|
||||
For the Python integration suite under `tests/`. Setup, running, and suite layout live in [`docs/contributing/tests/integration.md`](../../docs/contributing/tests/integration.md).
|
||||
|
||||
- **No `_`-prefixed helper functions in test modules — this is the rule that matters most.** A reader must be able to see what a test does in its body alone, without chasing private helpers that scatter the meaning across the file. Inline the logic: an expression, a comprehension, a few repeated lines are all fine — repetition across tests is cheaper than indirection. When several tests genuinely share non-trivial setup or assertions, that is what fixtures are for — in `tests/fixtures/`, see the next rule. A module-level `_helper()` is never the answer.
|
||||
- **Fixtures live in `tests/fixtures/` — never under `integration/tests/`.** Not in test modules, not in suite `conftest.py` files. `tests/fixtures/` is the shared library (auth, signoz, clickhouse, logs/metrics/traces seeding, …): reuse what's there before writing anything new; when a new fixture is genuinely needed, add it to the matching `tests/fixtures/` module and register new modules in `tests/conftest.py` `pytest_plugins`. **The one exception: SigNoz-level fixtures in a suite's `conftest.py`.** A suite that needs its own SigNoz spun up with different envs (`create_signoz`/`create_migrator` with `env_overrides` + `cache_key` — e.g. basepath, metricreduction, querier_json_body) keeps that in its `conftest.py`; that is always okay.
|
||||
- **Fixture only when there is a lifecycle; otherwise a plain function.** A fixture earns its indirection by owning setup/teardown (`yield` + cleanup — `insert_metrics` truncating on teardown) or by provisioning a resource (containers, SigNoz instances). A stateless action or lookup (`create_saved_view`, `find_saved_view_by_name`, wiping a resource list) is a plain importable function in the matching `tests/fixtures/` module, taking `signoz`/`token` as ordinary arguments — never wrap a plain callable in a fixture-factory just to inject `signoz`.
|
||||
- **Fixtures own their cleanup.** When a test needs seeded state, put the seed + cleanup pair in a fixture (`yield`, then tear down) so tests in the same suite don't interfere — the pattern `insert_metrics` sets: yield a callable, truncate on teardown.
|
||||
- **Fixture-factory over indirect parametrization.** A fixture that yields a callable (e.g. `insert_metrics(metrics)`) is clearer than `@pytest.mark.parametrize(..., indirect=True)` + `request.param` — the value is an explicit argument, not resolved by magic.
|
||||
- **Skip at collection, not inside the test body.** Use `pytest.param(..., marks=pytest.mark.skip(reason="…"))` so a skipped case shows as SKIPPED-with-reason **and** short-circuits before its fixtures run (no environment spin-up for a test that won't execute).
|
||||
- **Test config comes from explicit `--flags`, not the environment.** Wire configuration as pytest options declared in `tests/conftest.py` (`pytest_addoption` — e.g. `--sqlstore-provider`, `--clickhouse-version`); do **not** add `os.environ` fallbacks inside tests or fixtures.
|
||||
- **snake_case parametrize ids.** `ids=["fill_gaps", "fill_zero"]`, not camelCase.
|
||||
- **Name suite files with the two-digit prefix (`NN_*.py`).** `pyproject.toml` restricts collection to `[0-9][0-9]_*.py` (plus the bootstrap `setup.py` / `run.py`) — a file that doesn't match is silently never collected.
|
||||
- **Always run pytest from `tests/`.** `--import-mode=importlib` is what allows same-basename files across suites (`querier/01_logs.py` vs `rawexportdata/01_logs.py`), but it disables pytest's implicit `sys.path` injection — `import fixtures` only resolves via `pythonpath = ["."]` from that rootdir.
|
||||
88
.github/pull_request_template.md
vendored
88
.github/pull_request_template.md
vendored
@@ -1,85 +1,13 @@
|
||||
## Pull Request
|
||||
|
||||
---
|
||||
|
||||
### 📄 Summary
|
||||
> Why does this change exist?
|
||||
> What problem does it solve, and why is this the right approach?
|
||||
|
||||
|
||||
|
||||
#### Screenshots / Screen Recordings (if applicable)
|
||||
> Include screenshots or screen recordings that clearly show the behavior before the change and the result after the change. This helps reviewers quickly understand the impact and verify the update.
|
||||
|
||||
<!--A few plain bullets saying what changed and why, for a reviewer skimming it - not a wall of text, not a restatement of the diff, not generated boilerplate.-->
|
||||
#### Description
|
||||
|
||||
<!--Reference issues using `Closes #issue-number` to enable automatic closure on merge. -->
|
||||
#### Issues closed by this PR
|
||||
> Reference issues using `Closes #issue-number` to enable automatic closure on merge.
|
||||
|
||||
---
|
||||
<!--If applicable, include screenshots or screen recordings that clearly show the behavior before the change and the result after the change. -->
|
||||
#### Screenshots / Screen Recordings
|
||||
|
||||
### ✅ Change Type
|
||||
_Select all that apply_
|
||||
<!--Anything reviewers should keep in mind while reviewing -->
|
||||
#### Additional Information
|
||||
|
||||
- [ ] ✨ Feature
|
||||
- [ ] 🐛 Bug fix
|
||||
- [ ] ♻️ Refactor
|
||||
- [ ] 🛠️ Infra / Tooling
|
||||
- [ ] 🧪 Test-only
|
||||
|
||||
---
|
||||
|
||||
### 🐛 Bug Context
|
||||
> Required if this PR fixes a bug
|
||||
|
||||
#### Root Cause
|
||||
> What caused the issue?
|
||||
> Regression, faulty assumption, edge case, refactor, etc.
|
||||
|
||||
#### Fix Strategy
|
||||
> How does this PR address the root cause?
|
||||
|
||||
---
|
||||
|
||||
### 🧪 Testing Strategy
|
||||
> How was this change validated?
|
||||
|
||||
- Tests added/updated:
|
||||
- Manual verification:
|
||||
- Edge cases covered:
|
||||
|
||||
---
|
||||
|
||||
### ⚠️ Risk & Impact Assessment
|
||||
> What could break? How do we recover?
|
||||
|
||||
- Blast radius:
|
||||
- Potential regressions:
|
||||
- Rollback plan:
|
||||
|
||||
---
|
||||
|
||||
### 📝 Changelog
|
||||
> Fill only if this affects users, APIs, UI, or documented behavior
|
||||
> Use **N/A** for internal or non-user-facing changes
|
||||
|
||||
| Field | Value |
|
||||
|------|-------|
|
||||
| Deployment Type | Cloud / OSS / Enterprise |
|
||||
| Change Type | Feature / Bug Fix / Maintenance |
|
||||
| Description | User-facing summary |
|
||||
|
||||
---
|
||||
|
||||
### 📋 Checklist
|
||||
- [ ] Tests added or explicitly not required
|
||||
- [ ] Manually tested
|
||||
- [ ] Breaking changes documented
|
||||
- [ ] Backward compatibility considered
|
||||
|
||||
---
|
||||
|
||||
## 👀 Notes for Reviewers
|
||||
|
||||
<!-- Anything reviewers should keep in mind while reviewing -->
|
||||
|
||||
---
|
||||
<!--Please delete paragraphs that you did not use before submitting.-->
|
||||
|
||||
83
.github/workflows/docs.yml
vendored
83
.github/workflows/docs.yml
vendored
@@ -1,83 +0,0 @@
|
||||
name: "Update PR labels and Block PR until related docs are shipped for the feature"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
types: [opened, edited, labeled, unlabeled]
|
||||
|
||||
permissions:
|
||||
pull-requests: write
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
docs_label_check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check PR Title and Manage Labels
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
script: |
|
||||
const prTitle = context.payload.pull_request.title;
|
||||
const prNumber = context.payload.pull_request.number;
|
||||
const owner = context.repo.owner;
|
||||
const repo = context.repo.repo;
|
||||
|
||||
// Fetch the current PR details to get labels
|
||||
const pr = await github.rest.pulls.get({
|
||||
owner,
|
||||
repo,
|
||||
pull_number: prNumber
|
||||
});
|
||||
|
||||
const labels = pr.data.labels.map(label => label.name);
|
||||
|
||||
if (prTitle.startsWith('feat:')) {
|
||||
const hasDocsRequired = labels.includes('docs required');
|
||||
const hasDocsShipped = labels.includes('docs shipped');
|
||||
const hasDocsNotRequired = labels.includes('docs not required');
|
||||
|
||||
// If "docs not required" is present, skip the checks
|
||||
if (hasDocsNotRequired && !hasDocsRequired) {
|
||||
console.log("Skipping checks due to 'docs not required' label.");
|
||||
return; // Exit the script early
|
||||
}
|
||||
|
||||
// If "docs shipped" is present, remove "docs required" if it exists
|
||||
if (hasDocsShipped && hasDocsRequired) {
|
||||
await github.rest.issues.removeLabel({
|
||||
owner,
|
||||
repo,
|
||||
issue_number: prNumber,
|
||||
name: 'docs required'
|
||||
});
|
||||
console.log("Removed 'docs required' label.");
|
||||
}
|
||||
|
||||
// Add "docs required" label if neither "docs shipped" nor "docs required" are present
|
||||
if (!hasDocsRequired && !hasDocsShipped) {
|
||||
await github.rest.issues.addLabels({
|
||||
owner,
|
||||
repo,
|
||||
issue_number: prNumber,
|
||||
labels: ['docs required']
|
||||
});
|
||||
console.log("Added 'docs required' label.");
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch the updated labels after any changes
|
||||
const updatedPr = await github.rest.pulls.get({
|
||||
owner,
|
||||
repo,
|
||||
pull_number: prNumber
|
||||
});
|
||||
|
||||
const updatedLabels = updatedPr.data.labels.map(label => label.name);
|
||||
const updatedHasDocsRequired = updatedLabels.includes('docs required');
|
||||
const updatedHasDocsShipped = updatedLabels.includes('docs shipped');
|
||||
|
||||
// Block PR if "docs required" is still present and "docs shipped" is missing
|
||||
if (updatedHasDocsRequired && !updatedHasDocsShipped) {
|
||||
core.setFailed("This PR requires documentation. Please remove the 'docs required' label and add the 'docs shipped' label to proceed.");
|
||||
}
|
||||
15
.github/workflows/goci.yaml
vendored
15
.github/workflows/goci.yaml
vendored
@@ -53,6 +53,21 @@ jobs:
|
||||
with:
|
||||
PRIMUS_REF: main
|
||||
GO_VERSION: 1.24
|
||||
semconv-generated:
|
||||
if: |
|
||||
github.event_name == 'merge_group' ||
|
||||
(github.event_name == 'pull_request' && ! github.event.pull_request.head.repo.fork && github.event.pull_request.user.login != 'dependabot[bot]' && ! contains(github.event.pull_request.labels.*.name, 'safe-to-test')) ||
|
||||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test'))
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: self-checkout
|
||||
uses: actions/checkout@v4
|
||||
- name: go-install
|
||||
uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: "1.24"
|
||||
- name: check-semconv-generated-files
|
||||
run: go run ./scripts/semconv -check
|
||||
build:
|
||||
if: |
|
||||
github.event_name == 'merge_group' ||
|
||||
|
||||
6
.gitignore
vendored
6
.gitignore
vendored
@@ -90,8 +90,6 @@ queries.active
|
||||
.devenv/**/tmp/**
|
||||
.qodo
|
||||
|
||||
.dev
|
||||
|
||||
### Python ###
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
@@ -231,4 +229,6 @@ cython_debug/
|
||||
# LSP config files
|
||||
pyrightconfig.json
|
||||
|
||||
|
||||
# dev
|
||||
.dev/
|
||||
.claude/worktrees/
|
||||
|
||||
4
Makefile
4
Makefile
@@ -233,6 +233,10 @@ py-clean: ## Clear all pycache and pytest cache from tests directory recursively
|
||||
##############################################################
|
||||
# generate commands
|
||||
##############################################################
|
||||
.PHONY: semconv-generate
|
||||
semconv-generate: ## Regenerate semantic-convention families for Go and TypeScript
|
||||
@go run ./scripts/semconv
|
||||
|
||||
.PHONY: gen-mocks
|
||||
gen-mocks:
|
||||
@echo ">> Generating mocks"
|
||||
|
||||
@@ -7759,6 +7759,112 @@ components:
|
||||
enum:
|
||||
- basic
|
||||
type: string
|
||||
SavedviewtypesDisplay:
|
||||
properties:
|
||||
color:
|
||||
type: string
|
||||
fontSize:
|
||||
type: string
|
||||
format:
|
||||
type: string
|
||||
maxLines:
|
||||
type: integer
|
||||
type: object
|
||||
SavedviewtypesPanelType:
|
||||
enum:
|
||||
- value
|
||||
- graph
|
||||
- table
|
||||
- list
|
||||
- trace
|
||||
type: string
|
||||
SavedviewtypesPostableSavedView:
|
||||
properties:
|
||||
data:
|
||||
$ref: '#/components/schemas/SavedviewtypesSavedViewData'
|
||||
generateName:
|
||||
type: boolean
|
||||
name:
|
||||
type: string
|
||||
source:
|
||||
$ref: '#/components/schemas/SavedviewtypesSource'
|
||||
required:
|
||||
- source
|
||||
- data
|
||||
type: object
|
||||
SavedviewtypesSavedView:
|
||||
properties:
|
||||
createdAt:
|
||||
format: date-time
|
||||
type: string
|
||||
createdBy:
|
||||
type: string
|
||||
data:
|
||||
$ref: '#/components/schemas/SavedviewtypesSavedViewData'
|
||||
id:
|
||||
type: string
|
||||
name:
|
||||
type: string
|
||||
source:
|
||||
$ref: '#/components/schemas/SavedviewtypesSource'
|
||||
updatedAt:
|
||||
format: date-time
|
||||
type: string
|
||||
updatedBy:
|
||||
type: string
|
||||
required:
|
||||
- id
|
||||
type: object
|
||||
SavedviewtypesSavedViewData:
|
||||
properties:
|
||||
schemaVersion:
|
||||
type: string
|
||||
spec:
|
||||
$ref: '#/components/schemas/SavedviewtypesSavedViewSpec'
|
||||
required:
|
||||
- schemaVersion
|
||||
- spec
|
||||
type: object
|
||||
SavedviewtypesSavedViewSpec:
|
||||
properties:
|
||||
display:
|
||||
$ref: '#/components/schemas/SavedviewtypesDisplay'
|
||||
displayName:
|
||||
type: string
|
||||
panelType:
|
||||
$ref: '#/components/schemas/SavedviewtypesPanelType'
|
||||
queries:
|
||||
items:
|
||||
$ref: '#/components/schemas/Querybuildertypesv5QueryEnvelope'
|
||||
type: array
|
||||
selectedFields:
|
||||
items:
|
||||
$ref: '#/components/schemas/TelemetrytypesTelemetryFieldKey'
|
||||
type: array
|
||||
required:
|
||||
- displayName
|
||||
- panelType
|
||||
- queries
|
||||
- selectedFields
|
||||
- display
|
||||
type: object
|
||||
SavedviewtypesSource:
|
||||
enum:
|
||||
- traces
|
||||
- logs
|
||||
- metrics
|
||||
- meter
|
||||
type: string
|
||||
SavedviewtypesUpdatableSavedView:
|
||||
properties:
|
||||
data:
|
||||
$ref: '#/components/schemas/SavedviewtypesSavedViewData'
|
||||
source:
|
||||
$ref: '#/components/schemas/SavedviewtypesSource'
|
||||
required:
|
||||
- source
|
||||
- data
|
||||
type: object
|
||||
ServiceaccounttypesDeprecatedPostableServiceAccountRole:
|
||||
properties:
|
||||
id:
|
||||
@@ -22659,6 +22765,298 @@ paths:
|
||||
summary: Test alert rule
|
||||
tags:
|
||||
- rules
|
||||
/api/v2/saved_views:
|
||||
get:
|
||||
deprecated: false
|
||||
description: Returns saved views, optionally filtered by source and name.
|
||||
operationId: ListSavedViews
|
||||
parameters:
|
||||
- in: query
|
||||
name: source
|
||||
schema:
|
||||
$ref: '#/components/schemas/SavedviewtypesSource'
|
||||
- in: query
|
||||
name: name
|
||||
schema:
|
||||
type: string
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
properties:
|
||||
data:
|
||||
items:
|
||||
$ref: '#/components/schemas/SavedviewtypesSavedView'
|
||||
nullable: true
|
||||
type: array
|
||||
status:
|
||||
type: string
|
||||
required:
|
||||
- status
|
||||
- data
|
||||
type: object
|
||||
description: OK
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Bad Request
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Unauthorized
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Forbidden
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Internal Server Error
|
||||
security:
|
||||
- api_key:
|
||||
- saved-view:list
|
||||
- tokenizer:
|
||||
- saved-view:list
|
||||
summary: List saved views
|
||||
tags:
|
||||
- saved_view
|
||||
post:
|
||||
deprecated: false
|
||||
description: Persists a saved view for the explore page. Returns the id of the
|
||||
created view.
|
||||
operationId: CreateSavedView
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/SavedviewtypesPostableSavedView'
|
||||
responses:
|
||||
"201":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
properties:
|
||||
data:
|
||||
$ref: '#/components/schemas/TypesIdentifiable'
|
||||
status:
|
||||
type: string
|
||||
required:
|
||||
- status
|
||||
- data
|
||||
type: object
|
||||
description: Created
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Bad Request
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Unauthorized
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Forbidden
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Internal Server Error
|
||||
security:
|
||||
- api_key:
|
||||
- saved-view:create
|
||||
- tokenizer:
|
||||
- saved-view:create
|
||||
summary: Create saved view
|
||||
tags:
|
||||
- saved_view
|
||||
/api/v2/saved_views/{id}:
|
||||
delete:
|
||||
deprecated: false
|
||||
description: Deletes a saved view by id.
|
||||
operationId: DeleteSavedView
|
||||
parameters:
|
||||
- in: path
|
||||
name: id
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
responses:
|
||||
"204":
|
||||
description: No Content
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Bad Request
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Unauthorized
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Forbidden
|
||||
"404":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Not Found
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Internal Server Error
|
||||
security:
|
||||
- api_key:
|
||||
- saved-view:delete
|
||||
- tokenizer:
|
||||
- saved-view:delete
|
||||
summary: Delete saved view
|
||||
tags:
|
||||
- saved_view
|
||||
get:
|
||||
deprecated: false
|
||||
description: Returns a saved view by id.
|
||||
operationId: GetSavedView
|
||||
parameters:
|
||||
- in: path
|
||||
name: id
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
responses:
|
||||
"200":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
properties:
|
||||
data:
|
||||
$ref: '#/components/schemas/SavedviewtypesSavedView'
|
||||
status:
|
||||
type: string
|
||||
required:
|
||||
- status
|
||||
- data
|
||||
type: object
|
||||
description: OK
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Bad Request
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Unauthorized
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Forbidden
|
||||
"404":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Not Found
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Internal Server Error
|
||||
security:
|
||||
- api_key:
|
||||
- saved-view:read
|
||||
- tokenizer:
|
||||
- saved-view:read
|
||||
summary: Get saved view
|
||||
tags:
|
||||
- saved_view
|
||||
put:
|
||||
deprecated: false
|
||||
description: Replaces a saved view's name and query.
|
||||
operationId: UpdateSavedView
|
||||
parameters:
|
||||
- in: path
|
||||
name: id
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/SavedviewtypesUpdatableSavedView'
|
||||
responses:
|
||||
"204":
|
||||
description: No Content
|
||||
"400":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Bad Request
|
||||
"401":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Unauthorized
|
||||
"403":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Forbidden
|
||||
"404":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Not Found
|
||||
"500":
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/RenderErrorResponse'
|
||||
description: Internal Server Error
|
||||
security:
|
||||
- api_key:
|
||||
- saved-view:update
|
||||
- tokenizer:
|
||||
- saved-view:update
|
||||
summary: Update saved view
|
||||
tags:
|
||||
- saved_view
|
||||
/api/v2/sessions:
|
||||
delete:
|
||||
deprecated: false
|
||||
|
||||
490
frontend/src/api/generated/services/saved-view/index.ts
Normal file
490
frontend/src/api/generated/services/saved-view/index.ts
Normal file
@@ -0,0 +1,490 @@
|
||||
/**
|
||||
* ! Do not edit manually
|
||||
* * The file has been auto-generated using Orval for SigNoz
|
||||
* * regenerate with 'pnpm generate:api'
|
||||
* SigNoz
|
||||
*/
|
||||
import { useMutation, useQuery } from 'react-query';
|
||||
import type {
|
||||
InvalidateOptions,
|
||||
MutationFunction,
|
||||
QueryClient,
|
||||
QueryFunction,
|
||||
QueryKey,
|
||||
UseMutationOptions,
|
||||
UseMutationResult,
|
||||
UseQueryOptions,
|
||||
UseQueryResult,
|
||||
} from 'react-query';
|
||||
|
||||
import type {
|
||||
CreateSavedView201,
|
||||
DeleteSavedViewPathParameters,
|
||||
GetSavedView200,
|
||||
GetSavedViewPathParameters,
|
||||
ListSavedViews200,
|
||||
ListSavedViewsParams,
|
||||
RenderErrorResponseDTO,
|
||||
SavedviewtypesPostableSavedViewDTO,
|
||||
SavedviewtypesUpdatableSavedViewDTO,
|
||||
UpdateSavedViewPathParameters,
|
||||
} from '../sigNoz.schemas';
|
||||
|
||||
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
|
||||
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
|
||||
|
||||
/**
|
||||
* Returns saved views, optionally filtered by source and name.
|
||||
* @summary List saved views
|
||||
*/
|
||||
export const listSavedViews = (
|
||||
params?: ListSavedViewsParams,
|
||||
signal?: AbortSignal,
|
||||
) => {
|
||||
return GeneratedAPIInstance<ListSavedViews200>({
|
||||
url: `/api/v2/saved_views`,
|
||||
method: 'GET',
|
||||
params,
|
||||
signal,
|
||||
});
|
||||
};
|
||||
|
||||
export const getListSavedViewsQueryKey = (params?: ListSavedViewsParams) => {
|
||||
return [`/api/v2/saved_views`, ...(params ? [params] : [])] as const;
|
||||
};
|
||||
|
||||
export const getListSavedViewsQueryOptions = <
|
||||
TData = Awaited<ReturnType<typeof listSavedViews>>,
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
>(
|
||||
params?: ListSavedViewsParams,
|
||||
options?: {
|
||||
query?: UseQueryOptions<
|
||||
Awaited<ReturnType<typeof listSavedViews>>,
|
||||
TError,
|
||||
TData
|
||||
>;
|
||||
},
|
||||
) => {
|
||||
const { query: queryOptions } = options ?? {};
|
||||
|
||||
const queryKey = queryOptions?.queryKey ?? getListSavedViewsQueryKey(params);
|
||||
|
||||
const queryFn: QueryFunction<Awaited<ReturnType<typeof listSavedViews>>> = ({
|
||||
signal,
|
||||
}) => listSavedViews(params, signal);
|
||||
|
||||
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
|
||||
Awaited<ReturnType<typeof listSavedViews>>,
|
||||
TError,
|
||||
TData
|
||||
> & { queryKey: QueryKey };
|
||||
};
|
||||
|
||||
export type ListSavedViewsQueryResult = NonNullable<
|
||||
Awaited<ReturnType<typeof listSavedViews>>
|
||||
>;
|
||||
export type ListSavedViewsQueryError = ErrorType<RenderErrorResponseDTO>;
|
||||
|
||||
/**
|
||||
* @summary List saved views
|
||||
*/
|
||||
|
||||
export function useListSavedViews<
|
||||
TData = Awaited<ReturnType<typeof listSavedViews>>,
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
>(
|
||||
params?: ListSavedViewsParams,
|
||||
options?: {
|
||||
query?: UseQueryOptions<
|
||||
Awaited<ReturnType<typeof listSavedViews>>,
|
||||
TError,
|
||||
TData
|
||||
>;
|
||||
},
|
||||
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
|
||||
const queryOptions = getListSavedViewsQueryOptions(params, options);
|
||||
|
||||
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
|
||||
queryKey: QueryKey;
|
||||
};
|
||||
|
||||
return { ...query, queryKey: queryOptions.queryKey };
|
||||
}
|
||||
|
||||
/**
|
||||
* @summary List saved views
|
||||
*/
|
||||
export const invalidateListSavedViews = async (
|
||||
queryClient: QueryClient,
|
||||
params?: ListSavedViewsParams,
|
||||
options?: InvalidateOptions,
|
||||
): Promise<QueryClient> => {
|
||||
await queryClient.invalidateQueries(
|
||||
{ queryKey: getListSavedViewsQueryKey(params) },
|
||||
options,
|
||||
);
|
||||
|
||||
return queryClient;
|
||||
};
|
||||
|
||||
/**
|
||||
* Persists a saved view for the explore page. Returns the id of the created view.
|
||||
* @summary Create saved view
|
||||
*/
|
||||
export const createSavedView = (
|
||||
savedviewtypesPostableSavedViewDTO?: BodyType<SavedviewtypesPostableSavedViewDTO>,
|
||||
signal?: AbortSignal,
|
||||
) => {
|
||||
return GeneratedAPIInstance<CreateSavedView201>({
|
||||
url: `/api/v2/saved_views`,
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
data: savedviewtypesPostableSavedViewDTO,
|
||||
signal,
|
||||
});
|
||||
};
|
||||
|
||||
export const getCreateSavedViewMutationOptions = <
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
TContext = unknown,
|
||||
>(options?: {
|
||||
mutation?: UseMutationOptions<
|
||||
Awaited<ReturnType<typeof createSavedView>>,
|
||||
TError,
|
||||
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
|
||||
TContext
|
||||
>;
|
||||
}): UseMutationOptions<
|
||||
Awaited<ReturnType<typeof createSavedView>>,
|
||||
TError,
|
||||
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
|
||||
TContext
|
||||
> => {
|
||||
const mutationKey = ['createSavedView'];
|
||||
const { mutation: mutationOptions } = options
|
||||
? options.mutation &&
|
||||
'mutationKey' in options.mutation &&
|
||||
options.mutation.mutationKey
|
||||
? options
|
||||
: { ...options, mutation: { ...options.mutation, mutationKey } }
|
||||
: { mutation: { mutationKey } };
|
||||
|
||||
const mutationFn: MutationFunction<
|
||||
Awaited<ReturnType<typeof createSavedView>>,
|
||||
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> }
|
||||
> = (props) => {
|
||||
const { data } = props ?? {};
|
||||
|
||||
return createSavedView(data);
|
||||
};
|
||||
|
||||
return { mutationFn, ...mutationOptions };
|
||||
};
|
||||
|
||||
export type CreateSavedViewMutationResult = NonNullable<
|
||||
Awaited<ReturnType<typeof createSavedView>>
|
||||
>;
|
||||
export type CreateSavedViewMutationBody =
|
||||
| BodyType<SavedviewtypesPostableSavedViewDTO>
|
||||
| undefined;
|
||||
export type CreateSavedViewMutationError = ErrorType<RenderErrorResponseDTO>;
|
||||
|
||||
/**
|
||||
* @summary Create saved view
|
||||
*/
|
||||
export const useCreateSavedView = <
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
TContext = unknown,
|
||||
>(options?: {
|
||||
mutation?: UseMutationOptions<
|
||||
Awaited<ReturnType<typeof createSavedView>>,
|
||||
TError,
|
||||
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
|
||||
TContext
|
||||
>;
|
||||
}): UseMutationResult<
|
||||
Awaited<ReturnType<typeof createSavedView>>,
|
||||
TError,
|
||||
{ data?: BodyType<SavedviewtypesPostableSavedViewDTO> },
|
||||
TContext
|
||||
> => {
|
||||
return useMutation(getCreateSavedViewMutationOptions(options));
|
||||
};
|
||||
/**
|
||||
* Deletes a saved view by id.
|
||||
* @summary Delete saved view
|
||||
*/
|
||||
export const deleteSavedView = (
|
||||
{ id }: DeleteSavedViewPathParameters,
|
||||
signal?: AbortSignal,
|
||||
) => {
|
||||
return GeneratedAPIInstance<void>({
|
||||
url: `/api/v2/saved_views/${id}`,
|
||||
method: 'DELETE',
|
||||
signal,
|
||||
});
|
||||
};
|
||||
|
||||
export const getDeleteSavedViewMutationOptions = <
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
TContext = unknown,
|
||||
>(options?: {
|
||||
mutation?: UseMutationOptions<
|
||||
Awaited<ReturnType<typeof deleteSavedView>>,
|
||||
TError,
|
||||
{ pathParams: DeleteSavedViewPathParameters },
|
||||
TContext
|
||||
>;
|
||||
}): UseMutationOptions<
|
||||
Awaited<ReturnType<typeof deleteSavedView>>,
|
||||
TError,
|
||||
{ pathParams: DeleteSavedViewPathParameters },
|
||||
TContext
|
||||
> => {
|
||||
const mutationKey = ['deleteSavedView'];
|
||||
const { mutation: mutationOptions } = options
|
||||
? options.mutation &&
|
||||
'mutationKey' in options.mutation &&
|
||||
options.mutation.mutationKey
|
||||
? options
|
||||
: { ...options, mutation: { ...options.mutation, mutationKey } }
|
||||
: { mutation: { mutationKey } };
|
||||
|
||||
const mutationFn: MutationFunction<
|
||||
Awaited<ReturnType<typeof deleteSavedView>>,
|
||||
{ pathParams: DeleteSavedViewPathParameters }
|
||||
> = (props) => {
|
||||
const { pathParams } = props ?? {};
|
||||
|
||||
return deleteSavedView(pathParams);
|
||||
};
|
||||
|
||||
return { mutationFn, ...mutationOptions };
|
||||
};
|
||||
|
||||
export type DeleteSavedViewMutationResult = NonNullable<
|
||||
Awaited<ReturnType<typeof deleteSavedView>>
|
||||
>;
|
||||
|
||||
export type DeleteSavedViewMutationError = ErrorType<RenderErrorResponseDTO>;
|
||||
|
||||
/**
|
||||
* @summary Delete saved view
|
||||
*/
|
||||
export const useDeleteSavedView = <
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
TContext = unknown,
|
||||
>(options?: {
|
||||
mutation?: UseMutationOptions<
|
||||
Awaited<ReturnType<typeof deleteSavedView>>,
|
||||
TError,
|
||||
{ pathParams: DeleteSavedViewPathParameters },
|
||||
TContext
|
||||
>;
|
||||
}): UseMutationResult<
|
||||
Awaited<ReturnType<typeof deleteSavedView>>,
|
||||
TError,
|
||||
{ pathParams: DeleteSavedViewPathParameters },
|
||||
TContext
|
||||
> => {
|
||||
return useMutation(getDeleteSavedViewMutationOptions(options));
|
||||
};
|
||||
/**
|
||||
* Returns a saved view by id.
|
||||
* @summary Get saved view
|
||||
*/
|
||||
export const getSavedView = (
|
||||
{ id }: GetSavedViewPathParameters,
|
||||
signal?: AbortSignal,
|
||||
) => {
|
||||
return GeneratedAPIInstance<GetSavedView200>({
|
||||
url: `/api/v2/saved_views/${id}`,
|
||||
method: 'GET',
|
||||
signal,
|
||||
});
|
||||
};
|
||||
|
||||
export const getGetSavedViewQueryKey = ({ id }: GetSavedViewPathParameters) => {
|
||||
return [`/api/v2/saved_views/${id}`] as const;
|
||||
};
|
||||
|
||||
export const getGetSavedViewQueryOptions = <
|
||||
TData = Awaited<ReturnType<typeof getSavedView>>,
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
>(
|
||||
{ id }: GetSavedViewPathParameters,
|
||||
options?: {
|
||||
query?: UseQueryOptions<
|
||||
Awaited<ReturnType<typeof getSavedView>>,
|
||||
TError,
|
||||
TData
|
||||
>;
|
||||
},
|
||||
) => {
|
||||
const { query: queryOptions } = options ?? {};
|
||||
|
||||
const queryKey = queryOptions?.queryKey ?? getGetSavedViewQueryKey({ id });
|
||||
|
||||
const queryFn: QueryFunction<Awaited<ReturnType<typeof getSavedView>>> = ({
|
||||
signal,
|
||||
}) => getSavedView({ id }, signal);
|
||||
|
||||
return {
|
||||
queryKey,
|
||||
queryFn,
|
||||
enabled: !!id,
|
||||
...queryOptions,
|
||||
} as UseQueryOptions<
|
||||
Awaited<ReturnType<typeof getSavedView>>,
|
||||
TError,
|
||||
TData
|
||||
> & { queryKey: QueryKey };
|
||||
};
|
||||
|
||||
export type GetSavedViewQueryResult = NonNullable<
|
||||
Awaited<ReturnType<typeof getSavedView>>
|
||||
>;
|
||||
export type GetSavedViewQueryError = ErrorType<RenderErrorResponseDTO>;
|
||||
|
||||
/**
|
||||
* @summary Get saved view
|
||||
*/
|
||||
|
||||
export function useGetSavedView<
|
||||
TData = Awaited<ReturnType<typeof getSavedView>>,
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
>(
|
||||
{ id }: GetSavedViewPathParameters,
|
||||
options?: {
|
||||
query?: UseQueryOptions<
|
||||
Awaited<ReturnType<typeof getSavedView>>,
|
||||
TError,
|
||||
TData
|
||||
>;
|
||||
},
|
||||
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
|
||||
const queryOptions = getGetSavedViewQueryOptions({ id }, options);
|
||||
|
||||
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
|
||||
queryKey: QueryKey;
|
||||
};
|
||||
|
||||
return { ...query, queryKey: queryOptions.queryKey };
|
||||
}
|
||||
|
||||
/**
|
||||
* @summary Get saved view
|
||||
*/
|
||||
export const invalidateGetSavedView = async (
|
||||
queryClient: QueryClient,
|
||||
{ id }: GetSavedViewPathParameters,
|
||||
options?: InvalidateOptions,
|
||||
): Promise<QueryClient> => {
|
||||
await queryClient.invalidateQueries(
|
||||
{ queryKey: getGetSavedViewQueryKey({ id }) },
|
||||
options,
|
||||
);
|
||||
|
||||
return queryClient;
|
||||
};
|
||||
|
||||
/**
|
||||
* Replaces a saved view's name and query.
|
||||
* @summary Update saved view
|
||||
*/
|
||||
export const updateSavedView = (
|
||||
{ id }: UpdateSavedViewPathParameters,
|
||||
savedviewtypesUpdatableSavedViewDTO?: BodyType<SavedviewtypesUpdatableSavedViewDTO>,
|
||||
signal?: AbortSignal,
|
||||
) => {
|
||||
return GeneratedAPIInstance<void>({
|
||||
url: `/api/v2/saved_views/${id}`,
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
data: savedviewtypesUpdatableSavedViewDTO,
|
||||
signal,
|
||||
});
|
||||
};
|
||||
|
||||
export const getUpdateSavedViewMutationOptions = <
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
TContext = unknown,
|
||||
>(options?: {
|
||||
mutation?: UseMutationOptions<
|
||||
Awaited<ReturnType<typeof updateSavedView>>,
|
||||
TError,
|
||||
{
|
||||
pathParams: UpdateSavedViewPathParameters;
|
||||
data?: BodyType<SavedviewtypesUpdatableSavedViewDTO>;
|
||||
},
|
||||
TContext
|
||||
>;
|
||||
}): UseMutationOptions<
|
||||
Awaited<ReturnType<typeof updateSavedView>>,
|
||||
TError,
|
||||
{
|
||||
pathParams: UpdateSavedViewPathParameters;
|
||||
data?: BodyType<SavedviewtypesUpdatableSavedViewDTO>;
|
||||
},
|
||||
TContext
|
||||
> => {
|
||||
const mutationKey = ['updateSavedView'];
|
||||
const { mutation: mutationOptions } = options
|
||||
? options.mutation &&
|
||||
'mutationKey' in options.mutation &&
|
||||
options.mutation.mutationKey
|
||||
? options
|
||||
: { ...options, mutation: { ...options.mutation, mutationKey } }
|
||||
: { mutation: { mutationKey } };
|
||||
|
||||
const mutationFn: MutationFunction<
|
||||
Awaited<ReturnType<typeof updateSavedView>>,
|
||||
{
|
||||
pathParams: UpdateSavedViewPathParameters;
|
||||
data?: BodyType<SavedviewtypesUpdatableSavedViewDTO>;
|
||||
}
|
||||
> = (props) => {
|
||||
const { pathParams, data } = props ?? {};
|
||||
|
||||
return updateSavedView(pathParams, data);
|
||||
};
|
||||
|
||||
return { mutationFn, ...mutationOptions };
|
||||
};
|
||||
|
||||
export type UpdateSavedViewMutationResult = NonNullable<
|
||||
Awaited<ReturnType<typeof updateSavedView>>
|
||||
>;
|
||||
export type UpdateSavedViewMutationBody =
|
||||
| BodyType<SavedviewtypesUpdatableSavedViewDTO>
|
||||
| undefined;
|
||||
export type UpdateSavedViewMutationError = ErrorType<RenderErrorResponseDTO>;
|
||||
|
||||
/**
|
||||
* @summary Update saved view
|
||||
*/
|
||||
export const useUpdateSavedView = <
|
||||
TError = ErrorType<RenderErrorResponseDTO>,
|
||||
TContext = unknown,
|
||||
>(options?: {
|
||||
mutation?: UseMutationOptions<
|
||||
Awaited<ReturnType<typeof updateSavedView>>,
|
||||
TError,
|
||||
{
|
||||
pathParams: UpdateSavedViewPathParameters;
|
||||
data?: BodyType<SavedviewtypesUpdatableSavedViewDTO>;
|
||||
},
|
||||
TContext
|
||||
>;
|
||||
}): UseMutationResult<
|
||||
Awaited<ReturnType<typeof updateSavedView>>,
|
||||
TError,
|
||||
{
|
||||
pathParams: UpdateSavedViewPathParameters;
|
||||
data?: BodyType<SavedviewtypesUpdatableSavedViewDTO>;
|
||||
},
|
||||
TContext
|
||||
> => {
|
||||
return useMutation(getUpdateSavedViewMutationOptions(options));
|
||||
};
|
||||
@@ -8858,6 +8858,112 @@ export interface RuletypesRuleDTO {
|
||||
export enum RuletypesThresholdKindDTO {
|
||||
basic = 'basic',
|
||||
}
|
||||
export interface SavedviewtypesDisplayDTO {
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
color?: string;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
fontSize?: string;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
format?: string;
|
||||
/**
|
||||
* @type integer
|
||||
*/
|
||||
maxLines?: number;
|
||||
}
|
||||
|
||||
export enum SavedviewtypesPanelTypeDTO {
|
||||
value = 'value',
|
||||
graph = 'graph',
|
||||
table = 'table',
|
||||
list = 'list',
|
||||
trace = 'trace',
|
||||
}
|
||||
export interface SavedviewtypesSavedViewSpecDTO {
|
||||
display: SavedviewtypesDisplayDTO;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
displayName: string;
|
||||
panelType: SavedviewtypesPanelTypeDTO;
|
||||
/**
|
||||
* @type array
|
||||
*/
|
||||
queries: Querybuildertypesv5QueryEnvelopeDTO[];
|
||||
/**
|
||||
* @type array
|
||||
*/
|
||||
selectedFields: TelemetrytypesTelemetryFieldKeyDTO[];
|
||||
}
|
||||
|
||||
export interface SavedviewtypesSavedViewDataDTO {
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
schemaVersion: string;
|
||||
spec: SavedviewtypesSavedViewSpecDTO;
|
||||
}
|
||||
|
||||
export enum SavedviewtypesSourceDTO {
|
||||
traces = 'traces',
|
||||
logs = 'logs',
|
||||
metrics = 'metrics',
|
||||
meter = 'meter',
|
||||
}
|
||||
export interface SavedviewtypesPostableSavedViewDTO {
|
||||
data: SavedviewtypesSavedViewDataDTO;
|
||||
/**
|
||||
* @type boolean
|
||||
*/
|
||||
generateName?: boolean;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
name?: string;
|
||||
source: SavedviewtypesSourceDTO;
|
||||
}
|
||||
|
||||
export interface SavedviewtypesSavedViewDTO {
|
||||
/**
|
||||
* @type string
|
||||
* @format date-time
|
||||
*/
|
||||
createdAt?: string;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
createdBy?: string;
|
||||
data?: SavedviewtypesSavedViewDataDTO;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
id: string;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
name?: string;
|
||||
source?: SavedviewtypesSourceDTO;
|
||||
/**
|
||||
* @type string
|
||||
* @format date-time
|
||||
*/
|
||||
updatedAt?: string;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
updatedBy?: string;
|
||||
}
|
||||
|
||||
export interface SavedviewtypesUpdatableSavedViewDTO {
|
||||
data: SavedviewtypesSavedViewDataDTO;
|
||||
source: SavedviewtypesSourceDTO;
|
||||
}
|
||||
|
||||
export interface ServiceaccounttypesDeprecatedPostableServiceAccountRoleDTO {
|
||||
/**
|
||||
* @type string
|
||||
@@ -12056,6 +12162,54 @@ export type TestRule200 = {
|
||||
status: string;
|
||||
};
|
||||
|
||||
export type ListSavedViewsParams = {
|
||||
/**
|
||||
* @description undefined
|
||||
*/
|
||||
source?: SavedviewtypesSourceDTO;
|
||||
/**
|
||||
* @type string
|
||||
* @description undefined
|
||||
*/
|
||||
name?: string;
|
||||
};
|
||||
|
||||
export type ListSavedViews200 = {
|
||||
/**
|
||||
* @type array,null
|
||||
*/
|
||||
data: SavedviewtypesSavedViewDTO[] | null;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
status: string;
|
||||
};
|
||||
|
||||
export type CreateSavedView201 = {
|
||||
data: TypesIdentifiableDTO;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
status: string;
|
||||
};
|
||||
|
||||
export type DeleteSavedViewPathParameters = {
|
||||
id: string;
|
||||
};
|
||||
export type GetSavedViewPathParameters = {
|
||||
id: string;
|
||||
};
|
||||
export type GetSavedView200 = {
|
||||
data: SavedviewtypesSavedViewDTO;
|
||||
/**
|
||||
* @type string
|
||||
*/
|
||||
status: string;
|
||||
};
|
||||
|
||||
export type UpdateSavedViewPathParameters = {
|
||||
id: string;
|
||||
};
|
||||
export type GetSessionContext200 = {
|
||||
data: AuthtypesSessionContextDTO;
|
||||
/**
|
||||
|
||||
32
frontend/src/constants/generated/semconvFamilies.gen.ts
Normal file
32
frontend/src/constants/generated/semconvFamilies.gen.ts
Normal file
@@ -0,0 +1,32 @@
|
||||
// Code generated by scripts/semconv. DO NOT EDIT.
|
||||
|
||||
export type SemconvFamily = {
|
||||
readonly current: string;
|
||||
readonly old: readonly string[];
|
||||
readonly kind: 'attribute' | 'metric';
|
||||
readonly contexts: readonly string[];
|
||||
readonly signals: readonly string[];
|
||||
readonly applyToMetrics: readonly string[];
|
||||
readonly valueMap: Readonly<Record<string, string>>;
|
||||
};
|
||||
|
||||
export const SEMCONV_FAMILIES: readonly SemconvFamily[] = [
|
||||
{
|
||||
current: 'db.system.name',
|
||||
old: ['db.system'],
|
||||
kind: 'attribute',
|
||||
contexts: [],
|
||||
signals: [],
|
||||
applyToMetrics: [],
|
||||
valueMap: {},
|
||||
},
|
||||
{
|
||||
current: 'deployment.environment.name',
|
||||
old: ['deployment.environment'],
|
||||
kind: 'attribute',
|
||||
contexts: [],
|
||||
signals: [],
|
||||
applyToMetrics: [],
|
||||
valueMap: {},
|
||||
},
|
||||
] as const;
|
||||
2
go.mod
2
go.mod
@@ -4,7 +4,7 @@ go 1.25.7
|
||||
|
||||
require (
|
||||
dario.cat/mergo v1.0.2
|
||||
github.com/AfterShip/clickhouse-sql-parser v0.5.4
|
||||
github.com/AfterShip/clickhouse-sql-parser v0.5.5
|
||||
github.com/ClickHouse/clickhouse-go/v2 v2.44.0
|
||||
github.com/DATA-DOG/go-sqlmock v1.5.2
|
||||
github.com/SigNoz/clickhouse-go-mock v0.14.0
|
||||
|
||||
4
go.sum
4
go.sum
@@ -66,8 +66,8 @@ dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
|
||||
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
|
||||
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
|
||||
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
|
||||
github.com/AfterShip/clickhouse-sql-parser v0.5.4 h1:yiCQaMq8EO+dpKdnpP9YYd/ne6MSuOXgsMsNL33NiTI=
|
||||
github.com/AfterShip/clickhouse-sql-parser v0.5.4/go.mod h1:Qi3qvPTfZb/aFwI5V4WFOahgjsLJa4MzVijIAfwOhDw=
|
||||
github.com/AfterShip/clickhouse-sql-parser v0.5.5 h1:LCA23yAA4GgF73PoYXb67yzCdC4sXsj4geQz1Oij3U8=
|
||||
github.com/AfterShip/clickhouse-sql-parser v0.5.5/go.mod h1:Qi3qvPTfZb/aFwI5V4WFOahgjsLJa4MzVijIAfwOhDw=
|
||||
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible h1:fcYLmCpyNYRnvJbPerq7U0hS+6+I79yEDJBqVNcqUzU=
|
||||
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0 h1:fou+2+WFTib47nS+nz/ozhEBnvU96bKHy6LjRsY4E28=
|
||||
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0/go.mod h1:t76Ruy8AHvUAC8GfMWJMa0ElSbuIcO03NLpynfbgsPA=
|
||||
|
||||
@@ -25,6 +25,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/modules/promote"
|
||||
"github.com/SigNoz/signoz/pkg/modules/rawdataexport"
|
||||
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
|
||||
"github.com/SigNoz/signoz/pkg/modules/savedview"
|
||||
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
|
||||
"github.com/SigNoz/signoz/pkg/modules/session"
|
||||
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
|
||||
@@ -75,6 +76,7 @@ type provider struct {
|
||||
rulerHandler ruler.Handler
|
||||
llmPricingRuleHandler llmpricingrule.Handler
|
||||
statsHandler statsreporter.Handler
|
||||
savedViewHandler savedview.Handler
|
||||
}
|
||||
|
||||
func NewFactory(
|
||||
@@ -110,6 +112,7 @@ func NewFactory(
|
||||
traceDetailHandler tracedetail.Handler,
|
||||
rulerHandler ruler.Handler,
|
||||
statsHandler statsreporter.Handler,
|
||||
savedViewHandler savedview.Handler,
|
||||
) factory.ProviderFactory[apiserver.APIServer, apiserver.Config] {
|
||||
return factory.NewProviderFactory(factory.MustNewName("signoz"), func(ctx context.Context, providerSettings factory.ProviderSettings, config apiserver.Config) (apiserver.APIServer, error) {
|
||||
return newProvider(
|
||||
@@ -148,6 +151,7 @@ func NewFactory(
|
||||
traceDetailHandler,
|
||||
rulerHandler,
|
||||
statsHandler,
|
||||
savedViewHandler,
|
||||
)
|
||||
})
|
||||
}
|
||||
@@ -188,6 +192,7 @@ func newProvider(
|
||||
traceDetailHandler tracedetail.Handler,
|
||||
rulerHandler ruler.Handler,
|
||||
statsHandler statsreporter.Handler,
|
||||
savedViewHandler savedview.Handler,
|
||||
) (apiserver.APIServer, error) {
|
||||
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/apiserver/signozapiserver")
|
||||
router := mux.NewRouter().UseEncodedPath()
|
||||
@@ -227,6 +232,7 @@ func newProvider(
|
||||
rulerHandler: rulerHandler,
|
||||
llmPricingRuleHandler: llmPricingRuleHandler,
|
||||
statsHandler: statsHandler,
|
||||
savedViewHandler: savedViewHandler,
|
||||
}
|
||||
|
||||
provider.authzMiddleware = middleware.NewAuthZ(settings.Logger(), orgGetter, authzService)
|
||||
@@ -359,6 +365,10 @@ func (provider *provider) AddToRouter(router *mux.Router) error {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := provider.addSavedViewRoutes(router); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
151
pkg/apiserver/signozapiserver/savedview.go
Normal file
151
pkg/apiserver/signozapiserver/savedview.go
Normal file
@@ -0,0 +1,151 @@
|
||||
package signozapiserver
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/http/handler"
|
||||
"github.com/SigNoz/signoz/pkg/types"
|
||||
"github.com/SigNoz/signoz/pkg/types/authtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/coretypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/gorilla/mux"
|
||||
)
|
||||
|
||||
func (provider *provider) addSavedViewRoutes(router *mux.Router) error {
|
||||
if err := router.Handle("/api/v2/saved_views", handler.New(
|
||||
provider.authzMiddleware.CheckResources(provider.savedViewHandler.ListV2, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
|
||||
handler.OpenAPIDef{
|
||||
ID: "ListSavedViews",
|
||||
Tags: []string{"saved_view"},
|
||||
Summary: "List saved views",
|
||||
Description: "Returns saved views, optionally filtered by source and name.",
|
||||
Request: nil,
|
||||
RequestQuery: new(savedviewtypes.ListSavedViewsParams),
|
||||
RequestContentType: "",
|
||||
Response: new([]*savedviewtypes.SavedView),
|
||||
ResponseContentType: "application/json",
|
||||
SuccessStatusCode: http.StatusOK,
|
||||
ErrorStatusCodes: []int{http.StatusBadRequest},
|
||||
Deprecated: false,
|
||||
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSavedView.Scope(coretypes.VerbList)}),
|
||||
},
|
||||
handler.WithResourceDefs(handler.BasicResourceDef{
|
||||
Resource: coretypes.ResourceMetaResourceSavedView,
|
||||
Verb: coretypes.VerbList,
|
||||
Category: coretypes.ActionCategoryDataAccess,
|
||||
Selector: coretypes.WildcardSelector,
|
||||
}),
|
||||
)).Methods(http.MethodGet).GetError(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := router.Handle("/api/v2/saved_views", handler.New(
|
||||
provider.authzMiddleware.CheckResources(provider.savedViewHandler.CreateV2, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName),
|
||||
handler.OpenAPIDef{
|
||||
ID: "CreateSavedView",
|
||||
Tags: []string{"saved_view"},
|
||||
Summary: "Create saved view",
|
||||
Description: "Persists a saved view for the explore page. Returns the id of the created view.",
|
||||
Request: new(savedviewtypes.PostableSavedView),
|
||||
RequestContentType: "application/json",
|
||||
Response: new(types.Identifiable),
|
||||
ResponseContentType: "application/json",
|
||||
SuccessStatusCode: http.StatusCreated,
|
||||
ErrorStatusCodes: []int{http.StatusBadRequest},
|
||||
Deprecated: false,
|
||||
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSavedView.Scope(coretypes.VerbCreate)}),
|
||||
},
|
||||
handler.WithResourceDefs(handler.BasicResourceDef{
|
||||
Resource: coretypes.ResourceMetaResourceSavedView,
|
||||
Verb: coretypes.VerbCreate,
|
||||
Category: coretypes.ActionCategoryDataAccess,
|
||||
ID: coretypes.ResponseJSONPath("data.id"),
|
||||
Selector: coretypes.WildcardSelector,
|
||||
}),
|
||||
)).Methods(http.MethodPost).GetError(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := router.Handle("/api/v2/saved_views/{id}", handler.New(
|
||||
provider.authzMiddleware.CheckResources(provider.savedViewHandler.GetV2, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
|
||||
handler.OpenAPIDef{
|
||||
ID: "GetSavedView",
|
||||
Tags: []string{"saved_view"},
|
||||
Summary: "Get saved view",
|
||||
Description: "Returns a saved view by id.",
|
||||
Request: nil,
|
||||
RequestContentType: "",
|
||||
Response: new(savedviewtypes.SavedView),
|
||||
ResponseContentType: "application/json",
|
||||
SuccessStatusCode: http.StatusOK,
|
||||
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
|
||||
Deprecated: false,
|
||||
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSavedView.Scope(coretypes.VerbRead)}),
|
||||
},
|
||||
handler.WithResourceDefs(handler.BasicResourceDef{
|
||||
Resource: coretypes.ResourceMetaResourceSavedView,
|
||||
Verb: coretypes.VerbRead,
|
||||
Category: coretypes.ActionCategoryDataAccess,
|
||||
ID: coretypes.PathParam("id"),
|
||||
Selector: coretypes.IDSelector,
|
||||
}),
|
||||
)).Methods(http.MethodGet).GetError(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := router.Handle("/api/v2/saved_views/{id}", handler.New(
|
||||
provider.authzMiddleware.CheckResources(provider.savedViewHandler.UpdateV2, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName),
|
||||
handler.OpenAPIDef{
|
||||
ID: "UpdateSavedView",
|
||||
Tags: []string{"saved_view"},
|
||||
Summary: "Update saved view",
|
||||
Description: "Replaces a saved view's name and query.",
|
||||
Request: new(savedviewtypes.UpdatableSavedView),
|
||||
RequestContentType: "application/json",
|
||||
Response: nil,
|
||||
ResponseContentType: "application/json",
|
||||
SuccessStatusCode: http.StatusNoContent,
|
||||
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
|
||||
Deprecated: false,
|
||||
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSavedView.Scope(coretypes.VerbUpdate)}),
|
||||
},
|
||||
handler.WithResourceDefs(handler.BasicResourceDef{
|
||||
Resource: coretypes.ResourceMetaResourceSavedView,
|
||||
Verb: coretypes.VerbUpdate,
|
||||
Category: coretypes.ActionCategoryDataAccess,
|
||||
ID: coretypes.PathParam("id"),
|
||||
Selector: coretypes.IDSelector,
|
||||
}),
|
||||
)).Methods(http.MethodPut).GetError(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := router.Handle("/api/v2/saved_views/{id}", handler.New(
|
||||
provider.authzMiddleware.CheckResources(provider.savedViewHandler.Delete, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName),
|
||||
handler.OpenAPIDef{
|
||||
ID: "DeleteSavedView",
|
||||
Tags: []string{"saved_view"},
|
||||
Summary: "Delete saved view",
|
||||
Description: "Deletes a saved view by id.",
|
||||
Request: nil,
|
||||
RequestContentType: "",
|
||||
Response: nil,
|
||||
ResponseContentType: "application/json",
|
||||
SuccessStatusCode: http.StatusNoContent,
|
||||
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
|
||||
Deprecated: false,
|
||||
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSavedView.Scope(coretypes.VerbDelete)}),
|
||||
},
|
||||
handler.WithResourceDefs(handler.BasicResourceDef{
|
||||
Resource: coretypes.ResourceMetaResourceSavedView,
|
||||
Verb: coretypes.VerbDelete,
|
||||
Category: coretypes.ActionCategoryDataAccess,
|
||||
ID: coretypes.PathParam("id"),
|
||||
Selector: coretypes.IDSelector,
|
||||
}),
|
||||
)).Methods(http.MethodDelete).GetError(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -40,7 +40,8 @@ func (c *conditionBuilder) ConditionFor(
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
keys, warning := querybuilder.ResolveKeys(key, querybuilder.MatchingFieldKeys(key, fieldKeys))
|
||||
logicalFields, warning := querybuilder.ResolveLogicalFields(key, querybuilder.MatchingLogicalFields(key, fieldKeys))
|
||||
keys := querybuilder.SingleKeys(logicalFields)
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
|
||||
@@ -11,6 +11,8 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/modules/savedview"
|
||||
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
|
||||
"github.com/SigNoz/signoz/pkg/types/authtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/gorilla/mux"
|
||||
)
|
||||
@@ -23,6 +25,116 @@ func NewHandler(module savedview.Module) savedview.Handler {
|
||||
return &handler{module: module}
|
||||
}
|
||||
|
||||
// legacyExtraData mirrors the frontend's extraData JSON shape so /api/v1
|
||||
// responses can synthesize the same shape back for the legacy frontend.
|
||||
type legacyExtraData struct {
|
||||
Color string `json:"color,omitempty"`
|
||||
SelectColumns []telemetrytypes.TelemetryFieldKey `json:"selectColumns,omitempty"`
|
||||
Format string `json:"format,omitempty"`
|
||||
MaxLines int `json:"maxLines,omitempty"`
|
||||
FontSize string `json:"fontSize,omitempty"`
|
||||
}
|
||||
|
||||
// newPostableSavedViewFromLegacyView builds a create payload for a v1 request.
|
||||
func newPostableSavedViewFromLegacyView(v *v3.SavedView) savedviewtypes.PostableSavedView {
|
||||
var legacy legacyExtraData
|
||||
if v.ExtraData != "" {
|
||||
// Best-effort: malformed/older extraData shapes never fail the request
|
||||
_ = json.Unmarshal([]byte(v.ExtraData), &legacy)
|
||||
}
|
||||
|
||||
return savedviewtypes.PostableSavedView{
|
||||
GenerateName: true,
|
||||
Source: savedviewtypes.Source{String: valuer.NewString(v.SourcePage)},
|
||||
Data: savedviewtypes.SavedViewData{
|
||||
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
|
||||
Spec: savedviewtypes.SavedViewSpec{
|
||||
DisplayName: v.Name,
|
||||
PanelType: savedviewtypes.PanelType{String: valuer.NewString(string(v.CompositeQuery.PanelType))},
|
||||
Queries: v.CompositeQuery.Queries,
|
||||
SelectedFields: legacy.SelectColumns,
|
||||
Display: savedviewtypes.Display{
|
||||
MaxLines: legacy.MaxLines,
|
||||
FontSize: legacy.FontSize,
|
||||
Format: legacy.Format,
|
||||
Color: legacy.Color,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// newUpdatableSavedViewFromLegacyView builds an update payload for a v1 request.
|
||||
func newUpdatableSavedViewFromLegacyView(v *v3.SavedView) savedviewtypes.UpdatableSavedView {
|
||||
var legacy legacyExtraData
|
||||
if v.ExtraData != "" {
|
||||
// Best-effort: malformed/older extraData shapes never fail the request
|
||||
_ = json.Unmarshal([]byte(v.ExtraData), &legacy)
|
||||
}
|
||||
|
||||
return savedviewtypes.UpdatableSavedView{
|
||||
Source: savedviewtypes.Source{String: valuer.NewString(v.SourcePage)},
|
||||
Data: savedviewtypes.SavedViewData{
|
||||
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
|
||||
Spec: savedviewtypes.SavedViewSpec{
|
||||
DisplayName: v.Name,
|
||||
PanelType: savedviewtypes.PanelType{String: valuer.NewString(string(v.CompositeQuery.PanelType))},
|
||||
Queries: v.CompositeQuery.Queries,
|
||||
SelectedFields: legacy.SelectColumns,
|
||||
Display: savedviewtypes.Display{
|
||||
MaxLines: legacy.MaxLines,
|
||||
FontSize: legacy.FontSize,
|
||||
Format: legacy.Format,
|
||||
Color: legacy.Color,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// newLegacyViewFromSavedView renders a v2 SavedView back into the v1 shape.
|
||||
func newLegacyViewFromSavedView(v *savedviewtypes.SavedView) (*v3.SavedView, error) {
|
||||
extraData, err := json.Marshal(legacyExtraData{
|
||||
Color: v.Data.Spec.Display.Color,
|
||||
SelectColumns: v.Data.Spec.SelectedFields,
|
||||
Format: v.Data.Spec.Display.Format,
|
||||
MaxLines: v.Data.Spec.Display.MaxLines,
|
||||
FontSize: v.Data.Spec.Display.FontSize,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in marshalling extra data")
|
||||
}
|
||||
|
||||
return &v3.SavedView{
|
||||
ID: v.ID,
|
||||
Name: v.Data.Spec.DisplayName,
|
||||
CreatedAt: v.CreatedAt,
|
||||
CreatedBy: v.CreatedBy,
|
||||
UpdatedAt: v.UpdatedAt,
|
||||
UpdatedBy: v.UpdatedBy,
|
||||
SourcePage: v.Source.StringValue(),
|
||||
CompositeQuery: &v3.CompositeQuery{
|
||||
PanelType: v3.PanelType(v.Data.Spec.PanelType.StringValue()),
|
||||
// Saved views are only ever created from the explorer's builder mode.
|
||||
QueryType: v3.QueryTypeBuilder,
|
||||
Queries: v.Data.Spec.Queries,
|
||||
},
|
||||
ExtraData: string(extraData),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func newLegacyViewsFromSavedViews(views []*savedviewtypes.SavedView) ([]*v3.SavedView, error) {
|
||||
out := make([]*v3.SavedView, 0, len(views))
|
||||
for _, view := range views {
|
||||
legacyView, err := newLegacyViewFromSavedView(view)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, legacyView)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (handler *handler) Create(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
|
||||
defer cancel()
|
||||
@@ -44,7 +156,7 @@ func (handler *handler) Create(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
uuid, err := handler.module.CreateView(ctx, claims.OrgID, view)
|
||||
uuid, err := handler.module.CreateView(ctx, claims.OrgID, newPostableSavedViewFromLegacyView(&view))
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
@@ -63,7 +175,7 @@ func (handler *handler) Get(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
viewID := mux.Vars(r)["viewId"]
|
||||
viewID := mux.Vars(r)["id"]
|
||||
viewUUID, err := valuer.NewUUID(viewID)
|
||||
if err != nil {
|
||||
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse view id"))
|
||||
@@ -76,7 +188,13 @@ func (handler *handler) Get(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusOK, view)
|
||||
legacyView, err := newLegacyViewFromSavedView(view)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusOK, legacyView)
|
||||
}
|
||||
|
||||
func (handler *handler) Update(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -89,7 +207,7 @@ func (handler *handler) Update(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
viewID := mux.Vars(r)["viewId"]
|
||||
viewID := mux.Vars(r)["id"]
|
||||
viewUUID, err := valuer.NewUUID(viewID)
|
||||
if err != nil {
|
||||
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse view id"))
|
||||
@@ -106,7 +224,7 @@ func (handler *handler) Update(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
err = handler.module.UpdateView(ctx, claims.OrgID, viewUUID, view)
|
||||
err = handler.module.UpdateView(ctx, claims.OrgID, viewUUID, newUpdatableSavedViewFromLegacyView(&view))
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
@@ -125,7 +243,7 @@ func (handler *handler) Delete(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
viewID := mux.Vars(r)["viewId"]
|
||||
viewID := mux.Vars(r)["id"]
|
||||
viewUUID, err := valuer.NewUUID(viewID)
|
||||
if err != nil {
|
||||
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse view id"))
|
||||
@@ -138,7 +256,7 @@ func (handler *handler) Delete(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusOK, nil)
|
||||
render.Success(w, http.StatusNoContent, nil)
|
||||
}
|
||||
|
||||
func (handler *handler) List(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -153,13 +271,18 @@ func (handler *handler) List(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
sourcePage := r.URL.Query().Get("sourcePage")
|
||||
name := r.URL.Query().Get("name")
|
||||
category := r.URL.Query().Get("category")
|
||||
|
||||
queries, err := handler.module.GetViewsForFilters(r.Context(), claims.OrgID, sourcePage, name, category)
|
||||
views, err := handler.module.GetViewsForFilters(r.Context(), claims.OrgID, savedviewtypes.Source{String: valuer.NewString(sourcePage)}, name)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusOK, queries)
|
||||
legacyViews, err := newLegacyViewsFromSavedViews(views)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusOK, legacyViews)
|
||||
}
|
||||
|
||||
197
pkg/modules/savedview/implsavedview/handler_test.go
Normal file
197
pkg/modules/savedview/implsavedview/handler_test.go
Normal file
@@ -0,0 +1,197 @@
|
||||
package implsavedview
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func testQueries() []qbtypes.QueryEnvelope {
|
||||
return []qbtypes.QueryEnvelope{
|
||||
{
|
||||
Type: qbtypes.QueryTypeBuilder,
|
||||
Spec: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
|
||||
Signal: telemetrytypes.SignalLogs,
|
||||
Aggregations: []qbtypes.LogAggregation{{Expression: "count()"}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewPostableSavedViewFromLegacyView(t *testing.T) {
|
||||
t.Run("all fields carried over", func(t *testing.T) {
|
||||
legacy := &v3.SavedView{
|
||||
Name: "my view",
|
||||
SourcePage: "logs",
|
||||
CompositeQuery: &v3.CompositeQuery{
|
||||
PanelType: v3.PanelTypeGraph,
|
||||
Queries: testQueries(),
|
||||
},
|
||||
ExtraData: `{"color":"blue","selectColumns":[{"name":"service.name"}],"format":"table","maxLines":10,"fontSize":"large"}`,
|
||||
}
|
||||
|
||||
postable := newPostableSavedViewFromLegacyView(legacy)
|
||||
|
||||
assert.Empty(t, postable.Name, "v1 has no slug concept -- name must always be generated")
|
||||
assert.True(t, postable.GenerateName, "v1 has no slug concept -- name must always be generated")
|
||||
assert.Equal(t, "my view", postable.Data.Spec.DisplayName)
|
||||
assert.Equal(t, savedviewtypes.SourceLogs, postable.Source)
|
||||
assert.Equal(t, savedviewtypes.SavedViewSchemaVersion, postable.Data.SchemaVersion)
|
||||
assert.Equal(t, savedviewtypes.PanelTypeGraph, postable.Data.Spec.PanelType)
|
||||
assert.Equal(t, legacy.CompositeQuery.Queries, postable.Data.Spec.Queries)
|
||||
assert.Equal(t, []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}}, postable.Data.Spec.SelectedFields)
|
||||
assert.Equal(t, savedviewtypes.Display{MaxLines: 10, FontSize: "large", Format: "table", Color: "blue"}, postable.Data.Spec.Display)
|
||||
})
|
||||
|
||||
t.Run("empty extra data leaves display and selected fields zero-valued", func(t *testing.T) {
|
||||
legacy := &v3.SavedView{
|
||||
Name: "no extra data",
|
||||
SourcePage: "traces",
|
||||
CompositeQuery: &v3.CompositeQuery{
|
||||
PanelType: v3.PanelTypeTable,
|
||||
Queries: testQueries(),
|
||||
},
|
||||
ExtraData: "",
|
||||
}
|
||||
|
||||
postable := newPostableSavedViewFromLegacyView(legacy)
|
||||
|
||||
assert.Equal(t, savedviewtypes.Display{}, postable.Data.Spec.Display)
|
||||
assert.Nil(t, postable.Data.Spec.SelectedFields)
|
||||
})
|
||||
|
||||
t.Run("malformed extra data is ignored, not an error", func(t *testing.T) {
|
||||
legacy := &v3.SavedView{
|
||||
Name: "malformed extra data",
|
||||
SourcePage: "metrics",
|
||||
CompositeQuery: &v3.CompositeQuery{
|
||||
PanelType: v3.PanelTypeList,
|
||||
Queries: testQueries(),
|
||||
},
|
||||
ExtraData: `{not valid json`,
|
||||
}
|
||||
|
||||
postable := newPostableSavedViewFromLegacyView(legacy)
|
||||
|
||||
assert.Equal(t, "malformed extra data", postable.Data.Spec.DisplayName)
|
||||
assert.Equal(t, savedviewtypes.Display{}, postable.Data.Spec.Display)
|
||||
})
|
||||
}
|
||||
|
||||
func TestNewUpdatableSavedViewFromLegacyView(t *testing.T) {
|
||||
legacy := &v3.SavedView{
|
||||
Name: "renamed view",
|
||||
SourcePage: "traces",
|
||||
CompositeQuery: &v3.CompositeQuery{
|
||||
PanelType: v3.PanelTypeTable,
|
||||
Queries: testQueries(),
|
||||
},
|
||||
ExtraData: `{"color":"red"}`,
|
||||
}
|
||||
|
||||
updatable := newUpdatableSavedViewFromLegacyView(legacy)
|
||||
|
||||
assert.Equal(t, "renamed view", updatable.Data.Spec.DisplayName)
|
||||
assert.Equal(t, savedviewtypes.SourceTraces, updatable.Source)
|
||||
}
|
||||
|
||||
func TestNewLegacyViewFromSavedView(t *testing.T) {
|
||||
now := time.Now()
|
||||
savedView := &savedviewtypes.SavedView{
|
||||
Name: "my-view-abc123ef",
|
||||
Source: savedviewtypes.SourceLogs,
|
||||
Data: savedviewtypes.SavedViewData{
|
||||
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
|
||||
Spec: savedviewtypes.SavedViewSpec{
|
||||
DisplayName: "my view",
|
||||
PanelType: savedviewtypes.PanelTypeGraph,
|
||||
Queries: testQueries(),
|
||||
SelectedFields: []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}},
|
||||
Display: savedviewtypes.Display{MaxLines: 10, FontSize: "large", Format: "table", Color: "blue"},
|
||||
},
|
||||
},
|
||||
}
|
||||
savedView.ID = valuer.GenerateUUID()
|
||||
savedView.CreatedAt = now
|
||||
savedView.UpdatedAt = now
|
||||
savedView.CreatedBy = "creator@signoz.io"
|
||||
savedView.UpdatedBy = "updater@signoz.io"
|
||||
|
||||
legacy, err := newLegacyViewFromSavedView(savedView)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, savedView.ID, legacy.ID)
|
||||
assert.Equal(t, savedView.Data.Spec.DisplayName, legacy.Name)
|
||||
assert.Equal(t, savedView.CreatedAt, legacy.CreatedAt)
|
||||
assert.Equal(t, savedView.CreatedBy, legacy.CreatedBy)
|
||||
assert.Equal(t, savedView.UpdatedAt, legacy.UpdatedAt)
|
||||
assert.Equal(t, savedView.UpdatedBy, legacy.UpdatedBy)
|
||||
assert.Equal(t, "logs", legacy.SourcePage)
|
||||
assert.Equal(t, v3.PanelTypeGraph, legacy.CompositeQuery.PanelType)
|
||||
assert.Equal(t, v3.QueryTypeBuilder, legacy.CompositeQuery.QueryType)
|
||||
assert.Equal(t, savedView.Data.Spec.Queries, legacy.CompositeQuery.Queries)
|
||||
|
||||
var extra legacyExtraData
|
||||
require.NoError(t, json.Unmarshal([]byte(legacy.ExtraData), &extra))
|
||||
assert.Equal(t, "blue", extra.Color)
|
||||
assert.Equal(t, savedView.Data.Spec.SelectedFields, extra.SelectColumns)
|
||||
assert.Equal(t, "table", extra.Format)
|
||||
assert.Equal(t, 10, extra.MaxLines)
|
||||
assert.Equal(t, "large", extra.FontSize)
|
||||
}
|
||||
|
||||
func TestNewLegacyViewsFromSavedViews(t *testing.T) {
|
||||
a := &savedviewtypes.SavedView{Name: "a-slug", Source: savedviewtypes.SourceLogs, Data: savedviewtypes.SavedViewData{Spec: savedviewtypes.SavedViewSpec{DisplayName: "a", PanelType: savedviewtypes.PanelTypeGraph, Queries: testQueries()}}}
|
||||
b := &savedviewtypes.SavedView{Name: "b-slug", Source: savedviewtypes.SourceTraces, Data: savedviewtypes.SavedViewData{Spec: savedviewtypes.SavedViewSpec{DisplayName: "b", PanelType: savedviewtypes.PanelTypeTable, Queries: testQueries()}}}
|
||||
|
||||
legacyViews, err := newLegacyViewsFromSavedViews([]*savedviewtypes.SavedView{a, b})
|
||||
require.NoError(t, err)
|
||||
require.Len(t, legacyViews, 2)
|
||||
assert.Equal(t, "a", legacyViews[0].Name)
|
||||
assert.Equal(t, "b", legacyViews[1].Name)
|
||||
}
|
||||
|
||||
// TestLegacyViewRoundTrip guards the whole v1<->v2 bridge: converting a
|
||||
// SavedView to its legacy shape and back must recover the fields the legacy
|
||||
// frontend round-trips through (displayName, source, panelType, queries,
|
||||
// selectedFields, display) -- these two functions are each other's inverse
|
||||
// on the API surface, so a regression in either should fail this. The internal
|
||||
// slug (Name) is deliberately NOT part of this contract -- v1 never sees it.
|
||||
func TestLegacyViewRoundTrip(t *testing.T) {
|
||||
original := &savedviewtypes.SavedView{
|
||||
Name: "round-trip-abc123ef",
|
||||
Source: savedviewtypes.SourceMetrics,
|
||||
Data: savedviewtypes.SavedViewData{
|
||||
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
|
||||
Spec: savedviewtypes.SavedViewSpec{
|
||||
DisplayName: "round trip",
|
||||
PanelType: savedviewtypes.PanelTypeTable,
|
||||
Queries: testQueries(),
|
||||
SelectedFields: []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}},
|
||||
Display: savedviewtypes.Display{MaxLines: 5, FontSize: "small", Format: "list", Color: "red"},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
legacy, err := newLegacyViewFromSavedView(original)
|
||||
require.NoError(t, err)
|
||||
|
||||
roundTripped := newPostableSavedViewFromLegacyView(legacy)
|
||||
|
||||
assert.Empty(t, roundTripped.Name)
|
||||
assert.True(t, roundTripped.GenerateName)
|
||||
assert.Equal(t, original.Data.Spec.DisplayName, roundTripped.Data.Spec.DisplayName)
|
||||
assert.Equal(t, original.Source, roundTripped.Source)
|
||||
assert.Equal(t, original.Data.Spec.PanelType, roundTripped.Data.Spec.PanelType)
|
||||
assert.Equal(t, original.Data.Spec.Queries, roundTripped.Data.Spec.Queries)
|
||||
assert.Equal(t, original.Data.Spec.SelectedFields, roundTripped.Data.Spec.SelectedFields)
|
||||
assert.Equal(t, original.Data.Spec.Display, roundTripped.Data.Spec.Display)
|
||||
}
|
||||
135
pkg/modules/savedview/implsavedview/handler_v2.go
Normal file
135
pkg/modules/savedview/implsavedview/handler_v2.go
Normal file
@@ -0,0 +1,135 @@
|
||||
package implsavedview
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/http/binding"
|
||||
"github.com/SigNoz/signoz/pkg/http/render"
|
||||
"github.com/SigNoz/signoz/pkg/types"
|
||||
"github.com/SigNoz/signoz/pkg/types/authtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/gorilla/mux"
|
||||
)
|
||||
|
||||
func (handler *handler) CreateV2(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
claims, err := authtypes.ClaimsFromContext(ctx)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
var view savedviewtypes.PostableSavedView
|
||||
if err := binding.JSON.BindBody(r.Body, &view, binding.WithDisallowUnknownFields(true)); err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
if err := view.Validate(); err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
uuid, err := handler.module.CreateView(ctx, claims.OrgID, view)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusCreated, types.Identifiable{ID: uuid})
|
||||
}
|
||||
|
||||
func (handler *handler) GetV2(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
claims, err := authtypes.ClaimsFromContext(ctx)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
viewID := mux.Vars(r)["id"]
|
||||
viewUUID, err := valuer.NewUUID(viewID)
|
||||
if err != nil {
|
||||
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse view id"))
|
||||
return
|
||||
}
|
||||
|
||||
view, err := handler.module.GetView(ctx, claims.OrgID, viewUUID)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusOK, view)
|
||||
}
|
||||
|
||||
func (handler *handler) UpdateV2(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
claims, err := authtypes.ClaimsFromContext(ctx)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
viewID := mux.Vars(r)["id"]
|
||||
viewUUID, err := valuer.NewUUID(viewID)
|
||||
if err != nil {
|
||||
render.Error(w, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse view id"))
|
||||
return
|
||||
}
|
||||
var view savedviewtypes.UpdatableSavedView
|
||||
if err := binding.JSON.BindBody(r.Body, &view, binding.WithDisallowUnknownFields(true)); err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
if err := view.Validate(); err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
err = handler.module.UpdateView(ctx, claims.OrgID, viewUUID, view)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusNoContent, nil)
|
||||
}
|
||||
|
||||
func (handler *handler) ListV2(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
claims, err := authtypes.ClaimsFromContext(ctx)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
params := new(savedviewtypes.ListSavedViewsParams)
|
||||
if err := binding.Query.BindQuery(r.URL.Query(), params); err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
if err := params.Validate(); err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
queries, err := handler.module.GetViewsForFilters(r.Context(), claims.OrgID, params.Source, params.Name)
|
||||
if err != nil {
|
||||
render.Error(w, err)
|
||||
return
|
||||
}
|
||||
|
||||
render.Success(w, http.StatusOK, queries)
|
||||
}
|
||||
@@ -2,185 +2,59 @@ package implsavedview
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/modules/savedview"
|
||||
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/types"
|
||||
"github.com/SigNoz/signoz/pkg/types/authtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
type module struct {
|
||||
sqlstore sqlstore.SQLStore
|
||||
store savedviewtypes.Store
|
||||
}
|
||||
|
||||
func NewModule(sqlstore sqlstore.SQLStore) savedview.Module {
|
||||
return &module{sqlstore: sqlstore}
|
||||
func NewModule(store savedviewtypes.Store) savedview.Module {
|
||||
return &module{store: store}
|
||||
}
|
||||
|
||||
func (module *module) GetViewsForFilters(ctx context.Context, orgID string, sourcePage string, name string, category string) ([]*v3.SavedView, error) {
|
||||
var views []savedviewtypes.SavedView
|
||||
var err error
|
||||
if len(category) == 0 {
|
||||
err = module.sqlstore.BunDB().NewSelect().Model(&views).Where("org_id = ? AND source_page = ? AND name LIKE ?", orgID, sourcePage, "%"+name+"%").Scan(ctx)
|
||||
} else {
|
||||
err = module.sqlstore.BunDB().NewSelect().Model(&views).Where("org_id = ? AND source_page = ? AND category LIKE ? AND name LIKE ?", orgID, sourcePage, "%"+category+"%", "%"+name+"%").Scan(ctx)
|
||||
}
|
||||
func (module *module) GetViewsForFilters(ctx context.Context, orgID string, source savedviewtypes.Source, name string) ([]*savedviewtypes.SavedView, error) {
|
||||
return module.store.List(ctx, orgID, source, name)
|
||||
}
|
||||
|
||||
func (module *module) CreateView(ctx context.Context, orgID string, view savedviewtypes.PostableSavedView) (valuer.UUID, error) {
|
||||
claims, err := authtypes.ClaimsFromContext(ctx)
|
||||
if err != nil {
|
||||
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in getting saved views")
|
||||
}
|
||||
|
||||
var savedViews []*v3.SavedView
|
||||
for _, view := range views {
|
||||
var compositeQuery v3.CompositeQuery
|
||||
err = json.Unmarshal([]byte(view.Data), &compositeQuery)
|
||||
if err != nil {
|
||||
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in unmarshalling explorer query data: %s", err.Error())
|
||||
}
|
||||
savedViews = append(savedViews, &v3.SavedView{
|
||||
ID: view.ID,
|
||||
Name: view.Name,
|
||||
CreatedAt: view.CreatedAt,
|
||||
CreatedBy: view.CreatedBy,
|
||||
UpdatedAt: view.UpdatedAt,
|
||||
UpdatedBy: view.UpdatedBy,
|
||||
Tags: strings.Split(view.Tags, ","),
|
||||
SourcePage: view.SourcePage,
|
||||
CompositeQuery: &compositeQuery,
|
||||
ExtraData: view.ExtraData,
|
||||
})
|
||||
}
|
||||
return savedViews, nil
|
||||
}
|
||||
|
||||
func (module *module) CreateView(ctx context.Context, orgID string, view v3.SavedView) (valuer.UUID, error) {
|
||||
data, err := json.Marshal(view.CompositeQuery)
|
||||
if err != nil {
|
||||
return valuer.UUID{}, errors.WrapInternalf(err, errors.CodeInternal, "error in marshalling explorer query data")
|
||||
}
|
||||
|
||||
uuid := valuer.GenerateUUID()
|
||||
createdAt := time.Now()
|
||||
updatedAt := time.Now()
|
||||
|
||||
claims, errv2 := authtypes.ClaimsFromContext(ctx)
|
||||
if errv2 != nil {
|
||||
return valuer.UUID{}, errors.NewInternalf(errors.CodeInternal, "error in getting email from context")
|
||||
}
|
||||
|
||||
createBy := claims.Email
|
||||
updatedBy := claims.Email
|
||||
dbView := view.ToSavedView(orgID, claims.Email)
|
||||
|
||||
dbView := savedviewtypes.SavedView{
|
||||
TimeAuditable: types.TimeAuditable{
|
||||
CreatedAt: createdAt,
|
||||
UpdatedAt: updatedAt,
|
||||
},
|
||||
UserAuditable: types.UserAuditable{
|
||||
CreatedBy: createBy,
|
||||
UpdatedBy: updatedBy,
|
||||
},
|
||||
OrgID: orgID,
|
||||
Identifiable: types.Identifiable{
|
||||
ID: uuid,
|
||||
},
|
||||
Name: view.Name,
|
||||
Category: view.Category,
|
||||
SourcePage: view.SourcePage,
|
||||
Tags: strings.Join(view.Tags, ","),
|
||||
Data: string(data),
|
||||
ExtraData: view.ExtraData,
|
||||
if err := module.store.Create(ctx, dbView); err != nil {
|
||||
return valuer.UUID{}, err
|
||||
}
|
||||
|
||||
_, err = module.sqlstore.BunDB().NewInsert().Model(&dbView).Exec(ctx)
|
||||
if err != nil {
|
||||
return valuer.UUID{}, errors.WrapInternalf(err, errors.CodeInternal, "error in creating saved view")
|
||||
}
|
||||
return uuid, nil
|
||||
return dbView.ID, nil
|
||||
}
|
||||
|
||||
func (module *module) GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*v3.SavedView, error) {
|
||||
var view savedviewtypes.SavedView
|
||||
err := module.sqlstore.BunDB().NewSelect().Model(&view).Where("org_id = ? AND id = ?", orgID, uuid.StringValue()).Scan(ctx)
|
||||
if err != nil {
|
||||
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in getting saved view")
|
||||
}
|
||||
|
||||
var compositeQuery v3.CompositeQuery
|
||||
err = json.Unmarshal([]byte(view.Data), &compositeQuery)
|
||||
if err != nil {
|
||||
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in unmarshalling explorer query data")
|
||||
}
|
||||
return &v3.SavedView{
|
||||
ID: view.ID,
|
||||
Name: view.Name,
|
||||
Category: view.Category,
|
||||
CreatedAt: view.CreatedAt,
|
||||
CreatedBy: view.CreatedBy,
|
||||
UpdatedAt: view.UpdatedAt,
|
||||
UpdatedBy: view.UpdatedBy,
|
||||
SourcePage: view.SourcePage,
|
||||
Tags: strings.Split(view.Tags, ","),
|
||||
CompositeQuery: &compositeQuery,
|
||||
ExtraData: view.ExtraData,
|
||||
}, nil
|
||||
func (module *module) GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*savedviewtypes.SavedView, error) {
|
||||
return module.store.Get(ctx, orgID, uuid)
|
||||
}
|
||||
|
||||
func (module *module) UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view v3.SavedView) error {
|
||||
data, err := json.Marshal(view.CompositeQuery)
|
||||
func (module *module) UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view savedviewtypes.UpdatableSavedView) error {
|
||||
claims, err := authtypes.ClaimsFromContext(ctx)
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, errors.CodeInternal, "error in marshalling explorer query data")
|
||||
}
|
||||
|
||||
claims, errv2 := authtypes.ClaimsFromContext(ctx)
|
||||
if errv2 != nil {
|
||||
return errors.NewInternalf(errors.CodeInternal, "error in getting email from context")
|
||||
}
|
||||
|
||||
updatedAt := time.Now()
|
||||
updatedBy := claims.Email
|
||||
|
||||
_, err = module.sqlstore.BunDB().NewUpdate().
|
||||
Model(&savedviewtypes.SavedView{}).
|
||||
Set("updated_at = ?, updated_by = ?, name = ?, category = ?, source_page = ?, tags = ?, data = ?, extra_data = ?",
|
||||
updatedAt, updatedBy, view.Name, view.Category, view.SourcePage, strings.Join(view.Tags, ","), data, view.ExtraData).
|
||||
Where("id = ?", uuid.StringValue()).
|
||||
Where("org_id = ?", orgID).
|
||||
Exec(ctx)
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, errors.CodeInternal, "error in updating saved view")
|
||||
}
|
||||
return nil
|
||||
return module.store.Update(ctx, view.ToSavedView(uuid, orgID, claims.Email))
|
||||
}
|
||||
|
||||
func (module *module) DeleteView(ctx context.Context, orgID string, uuid valuer.UUID) error {
|
||||
_, err := module.sqlstore.BunDB().NewDelete().
|
||||
Model(&savedviewtypes.SavedView{}).
|
||||
Where("id = ?", uuid.StringValue()).
|
||||
Where("org_id = ?", orgID).
|
||||
Exec(ctx)
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, errors.CodeInternal, "error in deleting explorer query")
|
||||
}
|
||||
return nil
|
||||
return module.store.Delete(ctx, orgID, uuid)
|
||||
}
|
||||
|
||||
func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
|
||||
savedViews := []*savedviewtypes.SavedView{}
|
||||
|
||||
err := module.
|
||||
sqlstore.
|
||||
BunDB().
|
||||
NewSelect().
|
||||
Model(&savedViews).
|
||||
Where("org_id = ?", orgID).
|
||||
Scan(ctx)
|
||||
savedViews, err := module.store.List(ctx, orgID.StringValue(), savedviewtypes.Source{}, "")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
299
pkg/modules/savedview/implsavedview/module_test.go
Normal file
299
pkg/modules/savedview/implsavedview/module_test.go
Normal file
@@ -0,0 +1,299 @@
|
||||
package implsavedview_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/DATA-DOG/go-sqlmock"
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/modules/savedview"
|
||||
"github.com/SigNoz/signoz/pkg/modules/savedview/implsavedview"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstoretest"
|
||||
"github.com/SigNoz/signoz/pkg/types/authtypes"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes/savedviewtypestest"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func newTestStore() (savedview.Module, *savedviewtypestest.StoreTest) {
|
||||
sqlStore := sqlstoretest.New(sqlstore.Config{Provider: "sqlite"}, sqlmock.QueryMatcherRegexp)
|
||||
store := implsavedview.NewStore(sqlStore)
|
||||
return implsavedview.NewModule(store), savedviewtypestest.New(store, sqlStore.Mock())
|
||||
}
|
||||
|
||||
func testPostableSavedView(name string, source savedviewtypes.Source) savedviewtypes.PostableSavedView {
|
||||
return savedviewtypes.PostableSavedView{
|
||||
Name: name,
|
||||
Source: source,
|
||||
Data: savedviewtypes.SavedViewData{
|
||||
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
|
||||
Spec: savedviewtypes.SavedViewSpec{
|
||||
DisplayName: name,
|
||||
PanelType: savedviewtypes.PanelTypeGraph,
|
||||
Queries: []qbtypes.QueryEnvelope{
|
||||
{
|
||||
Type: qbtypes.QueryTypeBuilder,
|
||||
Spec: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
|
||||
Signal: telemetrytypes.SignalLogs,
|
||||
Aggregations: []qbtypes.LogAggregation{{Expression: "count()"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
SelectedFields: []telemetrytypes.TelemetryFieldKey{},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func testUpdatableSavedView(displayName string, source savedviewtypes.Source) savedviewtypes.UpdatableSavedView {
|
||||
postable := testPostableSavedView(displayName, source)
|
||||
return savedviewtypes.UpdatableSavedView{
|
||||
Source: postable.Source,
|
||||
Data: postable.Data,
|
||||
}
|
||||
}
|
||||
|
||||
func testSavedView(orgID string, id valuer.UUID, updatedBy string, view savedviewtypes.PostableSavedView) *savedviewtypes.SavedView {
|
||||
savedView := view.ToSavedView(orgID, "creator@signoz.io")
|
||||
savedView.ID = id
|
||||
savedView.UpdatedBy = updatedBy
|
||||
return savedView
|
||||
}
|
||||
|
||||
func contextWithClaims(orgID, email string) context.Context {
|
||||
return authtypes.NewContextWithClaims(context.Background(), authtypes.Claims{
|
||||
OrgID: orgID,
|
||||
Email: email,
|
||||
})
|
||||
}
|
||||
|
||||
func TestModule_CreateAndGetView(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
ctx := contextWithClaims(orgID, "creator@signoz.io")
|
||||
view := testPostableSavedView("my view", savedviewtypes.SourceLogs)
|
||||
|
||||
st.ExpectCreate()
|
||||
id, err := m.CreateView(ctx, orgID, view)
|
||||
require.NoError(t, err)
|
||||
require.False(t, id.IsZero())
|
||||
|
||||
stored := testSavedView(orgID, id, "creator@signoz.io", view)
|
||||
st.ExpectGet(orgID, id, stored)
|
||||
got, err := m.GetView(ctx, orgID, id)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, id, got.ID)
|
||||
assert.Equal(t, "my view", got.Name)
|
||||
assert.Equal(t, savedviewtypes.SourceLogs, got.Source)
|
||||
assert.Equal(t, "creator@signoz.io", got.CreatedBy)
|
||||
assert.Equal(t, "creator@signoz.io", got.UpdatedBy)
|
||||
assert.Equal(t, savedviewtypes.PanelTypeGraph, got.Data.Spec.PanelType)
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_GetView_NotFound(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
st.ExpectGet(orgID, id, nil)
|
||||
_, err := m.GetView(contextWithClaims(orgID, "someone@signoz.io"), orgID, id)
|
||||
require.Error(t, err)
|
||||
assert.True(t, errors.Ast(err, errors.TypeNotFound), "expected a not-found error, got %v", err)
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_GetView_ScopedToOrg(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgB := valuer.GenerateUUID().StringValue()
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
// The mock only has an expectation for orgB's WHERE clause; a lookup
|
||||
// scoped to org A's real id must not accidentally match it.
|
||||
st.ExpectGet(orgB, id, nil)
|
||||
_, err := m.GetView(contextWithClaims(orgB, "b@signoz.io"), orgB, id)
|
||||
require.Error(t, err, "a view created under org A must not be visible to org B")
|
||||
assert.True(t, errors.Ast(err, errors.TypeNotFound), "expected a not-found error, got %v", err)
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_UpdateView(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
existing := testSavedView(orgID, id, "creator@signoz.io", testPostableSavedView("my-view", savedviewtypes.SourceLogs))
|
||||
existingName := existing.Name
|
||||
|
||||
updated := testUpdatableSavedView("renamed", savedviewtypes.SourceTraces)
|
||||
updated.Data.Spec.PanelType = savedviewtypes.PanelTypeTable
|
||||
|
||||
st.ExpectUpdate(orgID, id, 1)
|
||||
require.NoError(t, m.UpdateView(contextWithClaims(orgID, "updater@signoz.io"), orgID, id, updated))
|
||||
|
||||
stored := testSavedView(orgID, id, "updater@signoz.io", testPostableSavedView("renamed", savedviewtypes.SourceTraces))
|
||||
stored.Name = existingName
|
||||
stored.Data.Spec.PanelType = savedviewtypes.PanelTypeTable
|
||||
st.ExpectGet(orgID, id, stored)
|
||||
got, err := m.GetView(contextWithClaims(orgID, "creator@signoz.io"), orgID, id)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, existingName, got.Name, "name must not change on update")
|
||||
assert.Equal(t, "renamed", got.Data.Spec.DisplayName)
|
||||
assert.Equal(t, savedviewtypes.SourceTraces, got.Source)
|
||||
assert.Equal(t, savedviewtypes.PanelTypeTable, got.Data.Spec.PanelType)
|
||||
assert.Equal(t, "updater@signoz.io", got.UpdatedBy)
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_UpdateView_NotFound(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
ctx := contextWithClaims(orgID, "someone@signoz.io")
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
st.ExpectUpdate(orgID, id, 0)
|
||||
err := m.UpdateView(ctx, orgID, id, testUpdatableSavedView("does-not-exist", savedviewtypes.SourceLogs))
|
||||
require.Error(t, err)
|
||||
assert.True(t, errors.Ast(err, errors.TypeNotFound), "expected a not-found error, got %v", err)
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_UpdateView_ScopedToOrg(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgB := valuer.GenerateUUID().StringValue()
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
// Only an Update scoped to orgB's WHERE clause is registered; updating org
|
||||
// A's view while authenticated as org B must not match it.
|
||||
st.ExpectUpdate(orgB, id, 0)
|
||||
err := m.UpdateView(contextWithClaims(orgB, "b@signoz.io"), orgB, id, testUpdatableSavedView("hijacked", savedviewtypes.SourceLogs))
|
||||
require.Error(t, err, "org B must not be able to update org A's view")
|
||||
assert.True(t, errors.Ast(err, errors.TypeNotFound))
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_DeleteView(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
ctx := contextWithClaims(orgID, "creator@signoz.io")
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
st.ExpectDelete(orgID, id, 1)
|
||||
require.NoError(t, m.DeleteView(ctx, orgID, id))
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_DeleteView_NotFound(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
ctx := contextWithClaims(orgID, "someone@signoz.io")
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
st.ExpectDelete(orgID, id, 0)
|
||||
err := m.DeleteView(ctx, orgID, id)
|
||||
require.Error(t, err)
|
||||
assert.True(t, errors.Ast(err, errors.TypeNotFound), "expected a not-found error, got %v", err)
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_DeleteView_ScopedToOrg(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgB := valuer.GenerateUUID().StringValue()
|
||||
id := valuer.GenerateUUID()
|
||||
|
||||
st.ExpectDelete(orgB, id, 0)
|
||||
err := m.DeleteView(contextWithClaims(orgB, "b@signoz.io"), orgB, id)
|
||||
require.Error(t, err, "org B must not be able to delete org A's view")
|
||||
assert.True(t, errors.Ast(err, errors.TypeNotFound))
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_GetViewsForFilters(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
ctx := contextWithClaims(orgID, "creator@signoz.io")
|
||||
|
||||
logsOverview := testSavedView(orgID, valuer.GenerateUUID(), "creator@signoz.io", testPostableSavedView("logs overview", savedviewtypes.SourceLogs))
|
||||
logsErrors := testSavedView(orgID, valuer.GenerateUUID(), "creator@signoz.io", testPostableSavedView("logs errors", savedviewtypes.SourceLogs))
|
||||
tracesOverview := testSavedView(orgID, valuer.GenerateUUID(), "creator@signoz.io", testPostableSavedView("traces overview", savedviewtypes.SourceTraces))
|
||||
|
||||
t.Run("filters by source page", func(t *testing.T) {
|
||||
st.ExpectList(orgID, []*savedviewtypes.SavedView{logsOverview, logsErrors})
|
||||
views, err := m.GetViewsForFilters(ctx, orgID, savedviewtypes.SourceLogs, "")
|
||||
require.NoError(t, err)
|
||||
assert.Len(t, views, 2)
|
||||
})
|
||||
|
||||
t.Run("filters by name substring", func(t *testing.T) {
|
||||
st.ExpectList(orgID, []*savedviewtypes.SavedView{logsErrors})
|
||||
views, err := m.GetViewsForFilters(ctx, orgID, savedviewtypes.SourceLogs, "errors")
|
||||
require.NoError(t, err)
|
||||
require.Len(t, views, 1)
|
||||
assert.Equal(t, "logs errors", views[0].Name)
|
||||
})
|
||||
|
||||
t.Run("omitted source page returns everything, not nothing", func(t *testing.T) {
|
||||
// Fixes a bug: source used to be an unconditional exact-match
|
||||
// clause, so a zero-value source matched zero rows -- even though
|
||||
// ListSavedViewsParams.Validate() treats a zero Source as valid
|
||||
// ("no filter"). Store.List now only applies the source clause
|
||||
// when it's non-zero.
|
||||
st.ExpectList(orgID, []*savedviewtypes.SavedView{logsOverview, logsErrors, tracesOverview})
|
||||
views, err := m.GetViewsForFilters(ctx, orgID, savedviewtypes.Source{}, "")
|
||||
require.NoError(t, err)
|
||||
assert.Len(t, views, 3)
|
||||
})
|
||||
|
||||
t.Run("scoped to org", func(t *testing.T) {
|
||||
otherOrgID := valuer.GenerateUUID().StringValue()
|
||||
st.ExpectList(otherOrgID, nil)
|
||||
views, err := m.GetViewsForFilters(ctx, otherOrgID, savedviewtypes.SourceLogs, "")
|
||||
require.NoError(t, err)
|
||||
assert.Empty(t, views)
|
||||
})
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
|
||||
func TestModule_Collect(t *testing.T) {
|
||||
m, st := newTestStore()
|
||||
|
||||
orgID := valuer.GenerateUUID()
|
||||
|
||||
logsA := testSavedView(orgID.StringValue(), valuer.GenerateUUID(), "creator@signoz.io", testPostableSavedView("logs a", savedviewtypes.SourceLogs))
|
||||
logsB := testSavedView(orgID.StringValue(), valuer.GenerateUUID(), "creator@signoz.io", testPostableSavedView("logs b", savedviewtypes.SourceLogs))
|
||||
tracesA := testSavedView(orgID.StringValue(), valuer.GenerateUUID(), "creator@signoz.io", testPostableSavedView("traces a", savedviewtypes.SourceTraces))
|
||||
|
||||
st.ExpectList(orgID.StringValue(), []*savedviewtypes.SavedView{logsA, logsB, tracesA})
|
||||
stats, err := m.Collect(context.Background(), orgID)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, int64(3), stats["savedview.count"])
|
||||
assert.Equal(t, int64(2), stats["savedview.source.logs.count"])
|
||||
assert.Equal(t, int64(1), stats["savedview.source.traces.count"])
|
||||
|
||||
require.NoError(t, st.AssertExpectations())
|
||||
}
|
||||
109
pkg/modules/savedview/implsavedview/store.go
Normal file
109
pkg/modules/savedview/implsavedview/store.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package implsavedview
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
type store struct {
|
||||
sqlstore sqlstore.SQLStore
|
||||
}
|
||||
|
||||
func NewStore(sqlstore sqlstore.SQLStore) savedviewtypes.Store {
|
||||
return &store{sqlstore: sqlstore}
|
||||
}
|
||||
|
||||
func (store *store) Create(ctx context.Context, view *savedviewtypes.SavedView) error {
|
||||
_, err := store.sqlstore.BunDB().NewInsert().Model(view).Exec(ctx)
|
||||
if err != nil {
|
||||
return store.sqlstore.WrapAlreadyExistsErrf(err, errors.CodeAlreadyExists, "saved view with name %s already exists", view.Name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *store) Get(ctx context.Context, orgID string, id valuer.UUID) (*savedviewtypes.SavedView, error) {
|
||||
var view savedviewtypes.SavedView
|
||||
err := store.sqlstore.BunDB().NewSelect().Model(&view).Where("org_id = ? AND id = ?", orgID, id.StringValue()).Scan(ctx)
|
||||
if err != nil {
|
||||
return nil, store.sqlstore.WrapNotFoundErrf(err, savedviewtypes.ErrCodeSavedViewNotFound, "saved view %s not found", id.StringValue())
|
||||
}
|
||||
|
||||
normalizeSelectedFields(&view)
|
||||
return &view, nil
|
||||
}
|
||||
|
||||
func (store *store) Update(ctx context.Context, view *savedviewtypes.SavedView) error {
|
||||
res, err := store.sqlstore.BunDB().NewUpdate().
|
||||
Model(&savedviewtypes.SavedView{}).
|
||||
Set("updated_at = ?, updated_by = ?, source = ?, data = ?",
|
||||
view.UpdatedAt, view.UpdatedBy, view.Source, view.Data).
|
||||
Where("id = ?", view.ID.StringValue()).
|
||||
Where("org_id = ?", view.OrgID).
|
||||
Exec(ctx)
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, errors.CodeInternal, "error in updating saved view")
|
||||
}
|
||||
|
||||
rowsAffected, err := res.RowsAffected()
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, errors.CodeInternal, "error in verifying the updated saved view")
|
||||
}
|
||||
if rowsAffected == 0 {
|
||||
return errors.NewNotFoundf(savedviewtypes.ErrCodeSavedViewNotFound, "saved view %s not found", view.ID.StringValue())
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *store) Delete(ctx context.Context, orgID string, id valuer.UUID) error {
|
||||
res, err := store.sqlstore.BunDB().NewDelete().
|
||||
Model(&savedviewtypes.SavedView{}).
|
||||
Where("id = ?", id.StringValue()).
|
||||
Where("org_id = ?", orgID).
|
||||
Exec(ctx)
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, errors.CodeInternal, "error in deleting saved view")
|
||||
}
|
||||
|
||||
rowsAffected, err := res.RowsAffected()
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, errors.CodeInternal, "error in verifying the deleted saved view")
|
||||
}
|
||||
if rowsAffected == 0 {
|
||||
return errors.NewNotFoundf(savedviewtypes.ErrCodeSavedViewNotFound, "saved view %s not found", id.StringValue())
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (store *store) List(ctx context.Context, orgID string, source savedviewtypes.Source, name string) ([]*savedviewtypes.SavedView, error) {
|
||||
var views []*savedviewtypes.SavedView
|
||||
q := store.sqlstore.BunDB().NewSelect().Model(&views).
|
||||
Where("org_id = ?", orgID).
|
||||
Where("name LIKE ?", "%"+name+"%")
|
||||
if !source.IsZero() {
|
||||
q = q.Where("source = ?", source)
|
||||
}
|
||||
|
||||
if err := q.Scan(ctx); err != nil {
|
||||
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in getting saved views")
|
||||
}
|
||||
|
||||
for _, view := range views {
|
||||
normalizeSelectedFields(view)
|
||||
}
|
||||
|
||||
return views, nil
|
||||
}
|
||||
|
||||
// normalizeSelectedFields fixes up a scanned row's nil SelectedFields.
|
||||
func normalizeSelectedFields(view *savedviewtypes.SavedView) {
|
||||
if view.Data.Spec.SelectedFields == nil {
|
||||
view.Data.Spec.SelectedFields = []telemetrytypes.TelemetryFieldKey{}
|
||||
}
|
||||
}
|
||||
@@ -4,19 +4,19 @@ import (
|
||||
"context"
|
||||
"net/http"
|
||||
|
||||
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
|
||||
"github.com/SigNoz/signoz/pkg/statsreporter"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
type Module interface {
|
||||
GetViewsForFilters(ctx context.Context, orgID string, sourcePage string, name string, category string) ([]*v3.SavedView, error)
|
||||
GetViewsForFilters(ctx context.Context, orgID string, source savedviewtypes.Source, name string) ([]*savedviewtypes.SavedView, error)
|
||||
|
||||
CreateView(ctx context.Context, orgID string, view v3.SavedView) (valuer.UUID, error)
|
||||
CreateView(ctx context.Context, orgID string, view savedviewtypes.PostableSavedView) (valuer.UUID, error)
|
||||
|
||||
GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*v3.SavedView, error)
|
||||
GetView(ctx context.Context, orgID string, uuid valuer.UUID) (*savedviewtypes.SavedView, error)
|
||||
|
||||
UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view v3.SavedView) error
|
||||
UpdateView(ctx context.Context, orgID string, uuid valuer.UUID, view savedviewtypes.UpdatableSavedView) error
|
||||
|
||||
DeleteView(ctx context.Context, orgID string, uuid valuer.UUID) error
|
||||
|
||||
@@ -33,9 +33,22 @@ type Handler interface {
|
||||
// Updates the saved view
|
||||
Update(http.ResponseWriter, *http.Request)
|
||||
|
||||
// Deletes the saved view
|
||||
// Deletes the saved view. Shared by both API generations -- delete has no
|
||||
// request/response body to reshape.
|
||||
Delete(http.ResponseWriter, *http.Request)
|
||||
|
||||
// Lists the saved views
|
||||
List(http.ResponseWriter, *http.Request)
|
||||
|
||||
// CreateV2 is the /api/v2/saved_views typed-spec variant of Create.
|
||||
CreateV2(http.ResponseWriter, *http.Request)
|
||||
|
||||
// GetV2 is the /api/v2/saved_views typed-spec variant of Get.
|
||||
GetV2(http.ResponseWriter, *http.Request)
|
||||
|
||||
// UpdateV2 is the /api/v2/saved_views typed-spec variant of Update.
|
||||
UpdateV2(http.ResponseWriter, *http.Request)
|
||||
|
||||
// ListV2 is the /api/v2/saved_views typed-spec variant of List.
|
||||
ListV2(http.ResponseWriter, *http.Request)
|
||||
}
|
||||
|
||||
@@ -23,7 +23,11 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
|
||||
traces uint64
|
||||
tracesLastSeenAt time.Time
|
||||
)
|
||||
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), max(timestamp) FROM %s", tracesTable)).Scan(&traces, &tracesLastSeenAt); err == nil {
|
||||
tracesLastSeenExpr := "max(timestamp)"
|
||||
if q.hasColumn(ctx, tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName, "inserted_at") {
|
||||
tracesLastSeenExpr = "max(inserted_at)"
|
||||
}
|
||||
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), %s FROM %s", tracesLastSeenExpr, tracesTable)).Scan(&traces, &tracesLastSeenAt); err == nil {
|
||||
stats["telemetry.traces.count"] = traces
|
||||
if tracesLastSeenAt.Unix() != 0 {
|
||||
stats["telemetry.traces.last_observed.time"] = tracesLastSeenAt.UTC()
|
||||
@@ -37,7 +41,11 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
|
||||
logs uint64
|
||||
logsLastSeenAt time.Time
|
||||
)
|
||||
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), fromUnixTimestamp64Nano(max(timestamp)) FROM %s", logsTable)).Scan(&logs, &logsLastSeenAt); err == nil {
|
||||
logsLastSeenExpr := "fromUnixTimestamp64Nano(max(timestamp))"
|
||||
if q.hasColumn(ctx, logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName, "inserted_at") {
|
||||
logsLastSeenExpr = "max(inserted_at)"
|
||||
}
|
||||
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), %s FROM %s", logsLastSeenExpr, logsTable)).Scan(&logs, &logsLastSeenAt); err == nil {
|
||||
stats["telemetry.logs.count"] = logs
|
||||
if logsLastSeenAt.Unix() != 0 {
|
||||
stats["telemetry.logs.last_observed.time"] = logsLastSeenAt.UTC()
|
||||
@@ -51,7 +59,11 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
|
||||
metrics uint64
|
||||
metricsLastSeenAt time.Time
|
||||
)
|
||||
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), toDateTime(max(unix_milli) / 1000) FROM %s", metricsTable)).Scan(&metrics, &metricsLastSeenAt); err == nil {
|
||||
metricsLastSeenExpr := "toDateTime(max(unix_milli) / 1000)"
|
||||
if q.hasColumn(ctx, metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName, "inserted_at_unix_milli") {
|
||||
metricsLastSeenExpr = "fromUnixTimestamp64Milli(max(inserted_at_unix_milli))"
|
||||
}
|
||||
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), %s FROM %s", metricsLastSeenExpr, metricsTable)).Scan(&metrics, &metricsLastSeenAt); err == nil {
|
||||
stats["telemetry.metrics.count"] = metrics
|
||||
if metricsLastSeenAt.Unix() != 0 {
|
||||
stats["telemetry.metrics.last_observed.time"] = metricsLastSeenAt.UTC()
|
||||
@@ -63,3 +75,12 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
|
||||
|
||||
return stats, nil
|
||||
}
|
||||
|
||||
func (q *querier) hasColumn(ctx context.Context, database, table, column string) bool {
|
||||
var exists bool
|
||||
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT hasColumnInTable(?, ?, ?)", database, table, column).Scan(&exists); err != nil {
|
||||
q.logger.DebugContext(ctx, "failed to check column existence", errors.Attr(err))
|
||||
return false
|
||||
}
|
||||
return exists
|
||||
}
|
||||
|
||||
@@ -507,9 +507,9 @@ func (aH *APIHandler) RegisterRoutes(router *mux.Router, am *middleware.AuthZ) {
|
||||
|
||||
router.HandleFunc("/api/v1/explorer/views", am.ViewAccess(aH.Signoz.Handlers.SavedView.List)).Methods(http.MethodGet)
|
||||
router.HandleFunc("/api/v1/explorer/views", am.EditAccess(aH.Signoz.Handlers.SavedView.Create)).Methods(http.MethodPost)
|
||||
router.HandleFunc("/api/v1/explorer/views/{viewId}", am.ViewAccess(aH.Signoz.Handlers.SavedView.Get)).Methods(http.MethodGet)
|
||||
router.HandleFunc("/api/v1/explorer/views/{viewId}", am.EditAccess(aH.Signoz.Handlers.SavedView.Update)).Methods(http.MethodPut)
|
||||
router.HandleFunc("/api/v1/explorer/views/{viewId}", am.EditAccess(aH.Signoz.Handlers.SavedView.Delete)).Methods(http.MethodDelete)
|
||||
router.HandleFunc("/api/v1/explorer/views/{id}", am.ViewAccess(aH.Signoz.Handlers.SavedView.Get)).Methods(http.MethodGet)
|
||||
router.HandleFunc("/api/v1/explorer/views/{id}", am.EditAccess(aH.Signoz.Handlers.SavedView.Update)).Methods(http.MethodPut)
|
||||
router.HandleFunc("/api/v1/explorer/views/{id}", am.EditAccess(aH.Signoz.Handlers.SavedView.Delete)).Methods(http.MethodDelete)
|
||||
router.HandleFunc("/api/v1/event", am.ViewAccess(aH.registerEvent)).Methods(http.MethodPost)
|
||||
|
||||
router.HandleFunc("/api/v1/services", am.ViewAccess(aH.getServices)).Methods(http.MethodPost) // Deprecated Usage, use the below endpoint /v2/services
|
||||
|
||||
17
pkg/querybuilder/clickhouse_quote.go
Normal file
17
pkg/querybuilder/clickhouse_quote.go
Normal file
@@ -0,0 +1,17 @@
|
||||
package querybuilder
|
||||
|
||||
import "strings"
|
||||
|
||||
// ClickHouseStringLiteral quotes a value for a ClickHouse string literal.
|
||||
func ClickHouseStringLiteral(value string) string {
|
||||
escaped := strings.ReplaceAll(value, `\`, `\\`)
|
||||
escaped = strings.ReplaceAll(escaped, `'`, `\'`)
|
||||
return "'" + escaped + "'"
|
||||
}
|
||||
|
||||
// ClickHouseIdentifier quotes a value for a ClickHouse identifier.
|
||||
func ClickHouseIdentifier(value string) string {
|
||||
escaped := strings.ReplaceAll(value, `\`, `\\`)
|
||||
escaped = strings.ReplaceAll(escaped, "`", "\\`")
|
||||
return "`" + escaped + "`"
|
||||
}
|
||||
17
pkg/querybuilder/clickhouse_quote_test.go
Normal file
17
pkg/querybuilder/clickhouse_quote_test.go
Normal file
@@ -0,0 +1,17 @@
|
||||
package querybuilder
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestClickHouseQuoting(t *testing.T) {
|
||||
t.Run("string literal", func(t *testing.T) {
|
||||
assert.Equal(t, `'name\'\\); SELECT 1 --'`, ClickHouseStringLiteral(`name'\); SELECT 1 --`))
|
||||
})
|
||||
|
||||
t.Run("identifier", func(t *testing.T) {
|
||||
assert.Equal(t, "`name\\`\\\\); SELECT 1 --`", ClickHouseIdentifier("name`\\); SELECT 1 --"))
|
||||
})
|
||||
}
|
||||
@@ -17,6 +17,7 @@ var (
|
||||
CodeClickHouseSQLNotSingleStatement = errors.MustNewCode("clickhouse_sql_not_single_statement")
|
||||
CodeClickHouseSQLNotSelect = errors.MustNewCode("clickhouse_sql_not_select")
|
||||
CodeClickHouseSQLTableFunction = errors.MustNewCode("clickhouse_sql_table_function")
|
||||
CodeClickHouseSQLReadingFunction = errors.MustNewCode("clickhouse_sql_reading_function")
|
||||
CodeClickHouseSQLInternalDatabase = errors.MustNewCode("clickhouse_sql_internal_database")
|
||||
CodeClickHouseSQLReadonlyOverride = errors.MustNewCode("clickhouse_sql_readonly_override")
|
||||
)
|
||||
@@ -43,6 +44,25 @@ var generatorTableFunctions = map[string]string{
|
||||
|
||||
var generatorTableFunctionsMessage = "allowed table functions are " + strings.Join(slices.Sorted(maps.Values(generatorTableFunctions)), ", ")
|
||||
|
||||
// readingFunctions reach a file, a model or the server binary while looking like ordinary
|
||||
// scalar functions. They name no table and no database, so neither of the rules above sees
|
||||
// them, and a wrapper that returns a number leaks what they read through the row count alone:
|
||||
// numbers(length(file(x))) yields one row per byte.
|
||||
//
|
||||
// Keyed by the lowercased name, since ClickHouse resolves function names case-insensitively.
|
||||
var readingFunctions = map[string]struct{}{
|
||||
"file": {},
|
||||
"catboostevaluate": {},
|
||||
"demangle": {},
|
||||
"addresstoline": {},
|
||||
"addresstolinewithinlines": {},
|
||||
"addresstosymbol": {},
|
||||
}
|
||||
|
||||
// A dictionary can be backed by HTTP, ODBC or another database, and every one of the 42
|
||||
// accessors carries this prefix.
|
||||
const dictionaryFunctionPrefix = "dict"
|
||||
|
||||
// The parser's grammar has gaps against SQL that ClickHouse itself accepts.
|
||||
func ErrIfStatementIsNotValid(query string) (err error) {
|
||||
defer func() {
|
||||
@@ -69,11 +89,23 @@ func ErrIfStatementIsNotValid(query string) (err error) {
|
||||
|
||||
visitor := &chparser.DefaultASTVisitor{Visit: func(node chparser.Expr) error {
|
||||
switch expr := node.(type) {
|
||||
case *chparser.TableFunctionExpr:
|
||||
// Source table functions remain usable in ClickHouse read-only mode. Arguments are
|
||||
// visited before this, so a read smuggled into one is already refused by the time
|
||||
// an allowed generator gets here.
|
||||
name := chparser.Format(expr.Name)
|
||||
case *chparser.TableExpr:
|
||||
// Source table functions remain usable in ClickHouse read-only mode, and only a
|
||||
// table position can be one. The parser also types a call inside a table function's
|
||||
// argument list as a TableFunctionExpr, so asking every one of those refuses the
|
||||
// numbers(intDiv(...)) that every dashboard writes. What can read from an argument
|
||||
// is caught by name below instead.
|
||||
source := expr.Expr
|
||||
if alias, ok := source.(*chparser.AliasExpr); ok {
|
||||
source = alias.Expr
|
||||
}
|
||||
|
||||
tableFunction, ok := source.(*chparser.TableFunctionExpr)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
|
||||
name := functionName(tableFunction.Name)
|
||||
if _, ok := generatorTableFunctions[strings.ToLower(name)]; ok {
|
||||
return nil
|
||||
}
|
||||
@@ -82,6 +114,25 @@ func ErrIfStatementIsNotValid(query string) (err error) {
|
||||
NewInvalidInputf(CodeClickHouseSQLTableFunction, "ClickHouse table functions are not allowed in SQL queries: %s", name).
|
||||
WithAdditional(generatorTableFunctionsMessage)
|
||||
|
||||
case *chparser.FunctionExpr:
|
||||
return errIfFunctionReads(expr.Name.Name)
|
||||
|
||||
case *chparser.TableFunctionExpr:
|
||||
// Reached for a call in an argument list, and for a table position ahead of the
|
||||
// TableExpr above, since a node is visited after its children.
|
||||
return errIfFunctionReads(functionName(expr.Name))
|
||||
|
||||
case *chparser.Path:
|
||||
// ClickHouse reads `x IN db.table` as a select from that table, and a qualified name
|
||||
// on the right of IN is a Path rather than a TableIdentifier.
|
||||
if len(expr.Fields) < 2 {
|
||||
return nil
|
||||
}
|
||||
|
||||
if _, ok := internalDatabases[strings.ToLower(expr.Fields[0].Name)]; ok {
|
||||
return errors.NewInvalidInputf(CodeClickHouseSQLInternalDatabase, "the ClickHouse %s database is not allowed in SQL queries", expr.Fields[0].Name)
|
||||
}
|
||||
|
||||
case *chparser.TableIdentifier:
|
||||
// Reading these is unaffected by ClickHouse read-only mode.
|
||||
if expr.Database == nil {
|
||||
@@ -111,3 +162,22 @@ func LogIfStatementIsNotValid(ctx context.Context, logger *slog.Logger, query st
|
||||
logger.WarnContext(ctx, "clickhouse sql is not valid", errors.Attr(err), slog.String("query", query))
|
||||
}
|
||||
}
|
||||
|
||||
func errIfFunctionReads(name string) error {
|
||||
lowered := strings.ToLower(name)
|
||||
if _, ok := readingFunctions[lowered]; !ok && !strings.HasPrefix(lowered, dictionaryFunctionPrefix) {
|
||||
return nil
|
||||
}
|
||||
|
||||
return errors.NewInvalidInputf(CodeClickHouseSQLReadingFunction, "ClickHouse functions that read outside the telemetry tables are not allowed in SQL queries: %s", name)
|
||||
}
|
||||
|
||||
// The parser spells a call's name as an Ident everywhere it can. Reading the field rather than
|
||||
// formatting the node keeps the quoting out, so `numbers`(1) matches numbers.
|
||||
func functionName(expr chparser.Expr) string {
|
||||
if ident, ok := expr.(*chparser.Ident); ok {
|
||||
return ident.Name
|
||||
}
|
||||
|
||||
return chparser.Format(expr)
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
|
||||
@@ -14,13 +15,12 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
|
||||
name string
|
||||
query string
|
||||
}{
|
||||
// Shapes a telemetry read is allowed to take.
|
||||
{"Select", "SELECT region AS r, zone FROM metrics WHERE metric_name = 'cpu' GROUP BY region, zone"},
|
||||
{"TrailingSemicolon", "SELECT count() FROM signoz_logs.distributed_logs_v2;"},
|
||||
{"CommonTableExpression", "WITH t AS (SELECT fingerprint FROM signoz_metrics.time_series_v4) SELECT * FROM t"},
|
||||
{"Join", "SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.b"},
|
||||
{"GlobalIn", "SELECT a FROM t WHERE a GLOBAL IN (SELECT b FROM t2)"},
|
||||
// GLOBAL parsed only when the join type was omitted, and only before IN. https://github.com/AfterShip/clickhouse-sql-parser/pull/293
|
||||
// https://github.com/AfterShip/clickhouse-sql-parser/pull/293
|
||||
{"GlobalLeftJoin", "SELECT * FROM t1 GLOBAL LEFT JOIN t2 ON t1.a = t2.a"},
|
||||
{"GlobalNotIn", "SELECT a FROM t WHERE a GLOBAL NOT IN (SELECT b FROM t2)"},
|
||||
{"Union", "SELECT * FROM t UNION ALL SELECT * FROM t2"},
|
||||
@@ -29,32 +29,34 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
|
||||
{"UnrelatedSetting", "SELECT * FROM t SETTINGS max_threads = 4"},
|
||||
{"TerminatedBlockComment", "SELECT /* keep me */ count() FROM t"},
|
||||
{"BlockCommentMarkerInsideStringLiteral", "SELECT count() FROM t WHERE body = '/* not a comment'"},
|
||||
// The parser used to loop forever on this; it now reads the comment to the end of
|
||||
// the input, so this doubles as a canary for that regression.
|
||||
// Looped forever before v0.5.2.
|
||||
{"TrailingUnterminatedBlockComment", "SELECT count() FROM t /* unterminated"},
|
||||
// The rule keys on the database, not on the table name.
|
||||
// Keyed on the database, not on the table name.
|
||||
{"TableNamedSystemInTelemetryDatabase", "SELECT * FROM signoz_logs.system"},
|
||||
{"SignedLiteralAfterClosingParenSpaced", "SELECT (toUnixTimestamp(now()) - 3600)*1000000000"},
|
||||
// order by interval
|
||||
{"OrderByInterval", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval ORDER BY interval"},
|
||||
{"OrderByIntervalAndDirection", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS `interval` ORDER BY `interval` ASC"},
|
||||
// `interval` is a unit keyword, so unquoting it was rejected everywhere the parser
|
||||
// expected a plain identifier. https://github.com/AfterShip/clickhouse-sql-parser/pull/296
|
||||
// https://github.com/AfterShip/clickhouse-sql-parser/pull/296
|
||||
{"OrderByUnquotedIntervalAsc", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval FROM t GROUP BY interval ORDER BY interval ASC"},
|
||||
{"OrderByUnquotedIntervalDesc", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval FROM t GROUP BY interval ORDER BY interval DESC"},
|
||||
{"UnquotedIntervalInGroupByTuple", "SELECT a FROM t GROUP BY (`service.name`, `service.version`, interval)"},
|
||||
{"UnquotedIntervalProductionQuery", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval, resource_string_service$$name AS `service.name`, attributes_string['http.route'] AS `http.route`, quantile(0.95)(duration_nano) / 1000000000 AS value FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_string_service$$name = 'svc-a' AND resources_string['deployment.environment'] = 'dev' AND attributes_string['http.route'] = '/v1' AND http_method = 'POST' AND timestamp BETWEEN toDateTime(1784601720) AND toDateTime(1784602620) AND ts_bucket_start BETWEEN 1784601720 - 1800 AND 1784602620 GROUP BY `service.name`, `http.route`, interval ORDER BY interval ASC"},
|
||||
// Separating the two readings of INTERVAL needs backtracking as per the current implementation which could have performance regressions.
|
||||
// https://github.com/AfterShip/clickhouse-sql-parser/pull/296#issuecomment-5150316367
|
||||
// The fix backtracks, so this bounds the cost. https://github.com/AfterShip/clickhouse-sql-parser/pull/296#issuecomment-5150316367
|
||||
{"UnquotedIntervalRepeatedThirtyTimes", "SELECT interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval + interval AS total FROM t WHERE interval > 0 ORDER BY interval ASC"},
|
||||
// `interval` was one of 37 such keywords. https://github.com/AfterShip/clickhouse-sql-parser/pull/305
|
||||
{"UnquotedLimitInFunctionArgument", "SELECT sum(limit) FROM t"},
|
||||
{"UnquotedLimitInArithmetic", "SELECT limit + 1 FROM t"},
|
||||
{"UnquotedLimitInNegation", "SELECT abs(-limit) FROM t"},
|
||||
{"UnquotedKeywordOperands", "SELECT sum(offset) + sum(format) + sum(settings) FROM t"},
|
||||
{"UnquotedLimitProductionQuery", "WITH limit_value AS (SELECT cluster, region, value AS limit FROM t) SELECT region AS `Region`, sum(limit) AS `Capacity` FROM limit_value GROUP BY Region"},
|
||||
{"SignedLiteralAfterClosingParenUnspaced", "SELECT now() AS ts, toFloat64(count()) AS value FROM ( SELECT attributes_string['TableName'] AS T, attributes_string['MissingId'] AS M, max(fromUnixTimestamp64Nano(timestamp)) AS last_seen, dateDiff('minute', min(fromUnixTimestamp64Nano(timestamp)), max(fromUnixTimestamp64Nano(timestamp))) AS age_min FROM signoz_logs.distributed_logs_v2 WHERE body='missing_map_record' AND timestamp >= (toUnixTimestamp(now())-3600)*1000000000 GROUP BY T, M ) WHERE age_min >= 20 AND last_seen >= now() - toIntervalMinute(8)"},
|
||||
{"SignedLiteralAfterClosingParenMinimal", "SELECT (1)-1"},
|
||||
{"TrimFunction", "SELECT trimBoth('/api/endpoint/', '/');"},
|
||||
// The SQL-standard keyword-separated argument forms, which took commas only. https://github.com/AfterShip/clickhouse-sql-parser/pull/290
|
||||
// https://github.com/AfterShip/clickhouse-sql-parser/pull/290
|
||||
{"StandardTrimSyntax", "SELECT trim(BOTH ' ' FROM body) FROM t"},
|
||||
{"StandardSubstringSyntax", "SELECT substring(body FROM 2 FOR 3) FROM t"},
|
||||
{"StandardOverlaySyntax", "SELECT overlay(body PLACING 'x' FROM 2) FROM t"},
|
||||
// Row generators compute their rows from their arguments, so they read through nothing. This is the shape they get used for: a dense interval axis to CROSS JOIN a sparse series against.
|
||||
// The shape row generators get used for: a dense interval axis to CROSS JOIN a sparse series against.
|
||||
{"NumbersTableFunction", "SELECT intervals.interval AS interval, active.cluster AS cluster, toFloat64(if(ts_data.has_data = 0, 0, 1)) AS value FROM ( SELECT DISTINCT JSONExtractString(labels, 'k8s.cluster.name') AS cluster FROM signoz_metrics.distributed_time_series_v4 WHERE metric_name = 'my_metric' AND unix_milli >= toUnixTimestamp(now() - INTERVAL 30 DAY) * 1000 HAVING cluster != '' ) AS active CROSS JOIN ( SELECT toStartOfInterval( toDateTime(toUnixTimestamp(now() - INTERVAL 30 MINUTE) + number * 60), INTERVAL 1 MINUTE ) AS interval FROM numbers(31) ) AS intervals LEFT JOIN ( SELECT toStartOfInterval( toDateTime(intDiv(s.unix_milli, 1000)), INTERVAL 1 MINUTE ) AS interval, JSONExtractString(ts.labels, 'k8s.cluster.name') AS cluster, 1 AS has_data FROM signoz_metrics.distributed_samples_v4 s INNER JOIN ( SELECT DISTINCT fingerprint, labels FROM signoz_metrics.distributed_time_series_v4 WHERE metric_name = 'my_metric' ) AS ts ON s.fingerprint = ts.fingerprint WHERE s.metric_name = 'my_metric' AND s.unix_milli >= toUnixTimestamp(now() - INTERVAL 30 MINUTE) * 1000 GROUP BY interval, cluster ) AS ts_data ON active.cluster = ts_data.cluster AND intervals.interval = ts_data.interval ORDER BY interval ASC"},
|
||||
{"NumbersMtTableFunction", "SELECT * FROM numbers_mt(31)"},
|
||||
{"ZerosTableFunction", "SELECT * FROM zeros(31)"},
|
||||
@@ -63,6 +65,16 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
|
||||
{"GenerateSeriesSnakeCaseTableFunction", "SELECT * FROM generate_series(1, 10)"},
|
||||
{"GeneratorTableFunctionUppercase", "SELECT * FROM NUMBERS(31)"},
|
||||
{"GeneratorTableFunctionParenthesisedArgument", "SELECT * FROM NUMBERS((31))"},
|
||||
// CAST in an argument was itself read as a table function. https://github.com/AfterShip/clickhouse-sql-parser/pull/307
|
||||
{"CastInGeneratorTableFunctionArgument", "SELECT * FROM numbers(CAST(10 AS UInt64))"},
|
||||
{"ScalarCallInGeneratorTableFunctionArgument", "SELECT * FROM numbers(intDiv(100, 2))"},
|
||||
{"NestedScalarCallInGeneratorTableFunctionArgument", "SELECT * FROM numbers(greatest(1, intDiv(100, 2) + 1))"},
|
||||
{"GeneratorTableFunctionProductionQuery", "WITH toInt64(1786029960000000000) AS start_ns, toInt64(1786031760000000000) AS end_ns, 300000000000 AS step_ns SELECT ts, toFloat64(sum(value)) AS value FROM (SELECT fromUnixTimestamp64Nano(start_ns + toInt64(number) * step_ns) AS ts, 0 AS value FROM numbers(greatest(1, intDiv(end_ns - start_ns, step_ns) + 1)) UNION ALL SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 5 minute) AS ts, count() AS value FROM signoz_logs.distributed_logs_v2 WHERE timestamp >= 1786029960000000000 AND timestamp <= 1786031760000000000 GROUP BY ts) GROUP BY ts ORDER BY ts"},
|
||||
// The allow list keys on the bare name, so quoting must not hide a generator from it.
|
||||
{"BacktickQuotedGeneratorTableFunction", "SELECT * FROM `numbers`(31)"},
|
||||
{"DoubleQuotedGeneratorTableFunction", "SELECT * FROM \"numbers\"(31)"},
|
||||
// Reads nothing: format builds a string, and shares its name with a table function.
|
||||
{"ScalarFunctionNamedAfterATableFunction", "SELECT format('{} {}', a, b) FROM t"},
|
||||
{"GeneratorTableFunctionInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l CROSS JOIN numbers(31) AS n"},
|
||||
{"GeneratorTableFunctionInCommonTableExpression", "WITH axis AS (SELECT number FROM numbers(31)) SELECT * FROM axis"},
|
||||
{"GeneratorTableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT number FROM numbers(31))"},
|
||||
@@ -71,8 +83,7 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
|
||||
|
||||
for _, testCase := range testCases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
// Bounded rather than called directly: a parser that backtracks without memoising
|
||||
// hangs instead of returning. Every case here parses in well under a millisecond.
|
||||
// Bounded because a parser that backtracks without memoising hangs rather than returning.
|
||||
errC := make(chan error, 1)
|
||||
go func() { errC <- ErrIfStatementIsNotValid(testCase.query) }()
|
||||
|
||||
@@ -92,46 +103,57 @@ func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
|
||||
query string
|
||||
expectedCode errors.Code
|
||||
}{
|
||||
// Not a single statement, or not a statement at all.
|
||||
{"Empty", "", CodeClickHouseSQLNotSingleStatement},
|
||||
{"UnterminatedBlockCommentOnly", "/* x", CodeClickHouseSQLUnparseable},
|
||||
{"Unparseable", "SELECT FROM WHERE", CodeClickHouseSQLUnparseable},
|
||||
{"MultipleStatements", "SELECT 1; DROP TABLE signoz_logs.logs_v2", CodeClickHouseSQLNotSingleStatement},
|
||||
// Parses, but is not a SELECT.
|
||||
{"Drop", "DROP TABLE signoz_logs.logs_v2", CodeClickHouseSQLNotSelect},
|
||||
{"Insert", "INSERT INTO signoz_logs.logs_v2 SELECT * FROM signoz_logs.logs_v2", CodeClickHouseSQLNotSelect},
|
||||
{"AlterDelete", "ALTER TABLE signoz_logs.logs_v2 DELETE WHERE 1 = 1", CodeClickHouseSQLNotSelect},
|
||||
{"CreateTable", "CREATE TABLE evil (a Int) ENGINE = Memory", CodeClickHouseSQLNotSelect},
|
||||
{"Grant", "GRANT ALL ON *.* TO admin", CodeClickHouseSQLNotSelect},
|
||||
{"Set", "SET readonly = 0", CodeClickHouseSQLNotSelect},
|
||||
// The parser still dereferences nil on a DEFAULT expression it cannot read, so the recover is what turns this into a rejection rather than a crash.
|
||||
{"UnparseableDefaultExpression", "CREATE TABLE t (a String DEFAULT foo(b FROM 2)) ENGINE = Memory", CodeClickHouseSQLParserPanic},
|
||||
// These the parser rejects outright rather than classifying.
|
||||
// Both panicked before v0.5.5. https://github.com/AfterShip/clickhouse-sql-parser/pull/306
|
||||
{"UnparseableDefaultExpression", "CREATE TABLE t (a String DEFAULT foo(b FROM 2)) ENGINE = Memory", CodeClickHouseSQLUnparseable},
|
||||
{"TrailingOperatorInDefaultExpression", "CREATE TABLE t (a String DEFAULT 1 +) ENGINE = Memory", CodeClickHouseSQLUnparseable},
|
||||
// Rejected outright rather than classified.
|
||||
{"ShowGrants", "SHOW GRANTS", CodeClickHouseSQLUnparseable},
|
||||
{"IntoOutfile", "SELECT * FROM t INTO OUTFILE '/tmp/x.csv'", CodeClickHouseSQLUnparseable},
|
||||
// Table functions, which read through something other than a telemetry table.
|
||||
{"UrlTableFunction", "SELECT * FROM url('http://attacker.example/x', CSV, 'a String')", CodeClickHouseSQLTableFunction},
|
||||
{"FileTableFunction", "SELECT * FROM file('/etc/passwd', CSV, 'a String')", CodeClickHouseSQLTableFunction},
|
||||
// file is also a scalar function, so the reading rule reaches it before the table rule does.
|
||||
{"FileTableFunction", "SELECT * FROM file('/etc/passwd', CSV, 'a String')", CodeClickHouseSQLReadingFunction},
|
||||
{"ExecutableTableFunction", "SELECT * FROM executable('script.sh', CSV, 'a String')", CodeClickHouseSQLTableFunction},
|
||||
{"TableFunctionInJoin", "SELECT * FROM t1 JOIN url('http://x', CSV, 'a String') u ON 1 = 1", CodeClickHouseSQLTableFunction},
|
||||
{"TableFunctionInCommonTableExpression", "WITH c AS (SELECT * FROM url('http://x', CSV, 'a String')) SELECT * FROM c", CodeClickHouseSQLTableFunction},
|
||||
{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM file('/etc/passwd', CSV, 'a String'))", CodeClickHouseSQLTableFunction},
|
||||
{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM url('http://x', CSV, 'a String'))", CodeClickHouseSQLTableFunction},
|
||||
{"TableFunctionInUnion", "SELECT * FROM t UNION ALL SELECT * FROM url('http://x', CSV, 'a String')", CodeClickHouseSQLTableFunction},
|
||||
// These reach the internal databases without ever naming one, so the table-function rule is the only thing that sees them.
|
||||
// Reach an internal database without naming one, so only the table-function rule sees them.
|
||||
{"MergeTableFunction", "SELECT * FROM merge('system', '.*')", CodeClickHouseSQLTableFunction},
|
||||
{"RemoteTableFunction", "SELECT * FROM remote('other-host', 'system.users')", CodeClickHouseSQLTableFunction},
|
||||
{"ClusterTableFunction", "SELECT * FROM cluster('c', 'system.users')", CodeClickHouseSQLTableFunction},
|
||||
// Pure, but excluded: generateRandom streams rows the arguments do not bound, and values has no use here that an array literal does not already cover.
|
||||
// Pure, but excluded: generateRandom is unbounded, and values adds nothing over an array literal.
|
||||
{"GenerateRandomTableFunction", "SELECT * FROM generateRandom('a UInt64')", CodeClickHouseSQLTableFunction},
|
||||
{"ValuesTableFunction", "SELECT * FROM values('a UInt64', 1, 2)", CodeClickHouseSQLTableFunction},
|
||||
// Arguments are visited before the table function itself, so allowing a generator does not give anyone a wrapper to smuggle a read through.
|
||||
// Arguments are visited first, so an allowed generator is not a wrapper to smuggle a read through.
|
||||
{"InternalDatabaseInsideAllowedTableFunction", "SELECT * FROM numbers((SELECT count() FROM system.users))", CodeClickHouseSQLInternalDatabase},
|
||||
{"InternalDatabaseJoinedOntoAllowedTableFunction", "SELECT * FROM numbers(31) AS n JOIN system.users AS u ON 1 = 1", CodeClickHouseSQLInternalDatabase},
|
||||
{"InternalDatabaseUnionedWithAllowedTableFunction", "SELECT number FROM numbers(31) UNION ALL SELECT name FROM system.users", CodeClickHouseSQLInternalDatabase},
|
||||
{"RefusedTableFunctionJoinedOntoAllowedTableFunction", "SELECT * FROM numbers(31) AS n JOIN url('http://x', CSV, 'a String') AS u ON 1 = 1", CodeClickHouseSQLTableFunction},
|
||||
{"RefusedTableFunctionInsideAllowedTableFunction", "SELECT * FROM numbers((SELECT count() FROM file('/etc/passwd', CSV, 'a String')))", CodeClickHouseSQLTableFunction},
|
||||
{"RefusedTableFunctionInsideAllowedTableFunction", "SELECT * FROM numbers((SELECT count() FROM url('http://x', CSV, 'a String')))", CodeClickHouseSQLTableFunction},
|
||||
{"InternalDatabaseInsideAllowedTableFunctionCommonTableExpression", "WITH axis AS (SELECT * FROM numbers((SELECT count() FROM system.users))) SELECT * FROM axis", CodeClickHouseSQLInternalDatabase},
|
||||
// Internal databases, which hold grants and server metadata rather than telemetry.
|
||||
// Read a file, a dictionary or the server binary without naming a table, so neither the table rule nor the database rule sees them. The row count alone is an oracle: numbers(length(file(x))) returns one row per byte.
|
||||
{"ScalarFileFunction", "SELECT file('/etc/passwd')", CodeClickHouseSQLReadingFunction},
|
||||
{"ScalarFileFunctionInWhere", "SELECT * FROM t WHERE length(file('/etc/passwd')) > 0", CodeClickHouseSQLReadingFunction},
|
||||
{"ScalarFileFunctionInGeneratorTableFunctionArgument", "SELECT * FROM numbers(length(file('/etc/passwd')))", CodeClickHouseSQLReadingFunction},
|
||||
{"DictionaryFunction", "SELECT dictGetUInt64('d', 'k', toUInt64(1))", CodeClickHouseSQLReadingFunction},
|
||||
{"DictionaryFunctionUppercase", "SELECT DICTGETSTRING('d', 'k', toUInt64(1))", CodeClickHouseSQLReadingFunction},
|
||||
{"DictionaryFunctionInGeneratorTableFunctionArgument", "SELECT * FROM numbers(dictGetUInt64('d', 'k', toUInt64(1)))", CodeClickHouseSQLReadingFunction},
|
||||
{"IntrospectionFunction", "SELECT demangle(addressToSymbol(toUInt64(1)))", CodeClickHouseSQLReadingFunction},
|
||||
{"ModelEvaluationFunction", "SELECT catboostEvaluate('/model.bin', 1)", CodeClickHouseSQLReadingFunction},
|
||||
// ClickHouse reads `x IN table` as `x IN (SELECT * FROM table)`, and a qualified name there is a Path rather than a TableIdentifier.
|
||||
{"InternalDatabaseInInOperator", "SELECT * FROM t WHERE a IN system.users", CodeClickHouseSQLInternalDatabase},
|
||||
{"InternalDatabaseInGlobalInOperator", "SELECT * FROM t WHERE a GLOBAL IN system.users", CodeClickHouseSQLInternalDatabase},
|
||||
{"InternalDatabaseInNotInOperator", "SELECT * FROM t WHERE a NOT IN system.users", CodeClickHouseSQLInternalDatabase},
|
||||
{"SystemUsers", "SELECT * FROM system.users", CodeClickHouseSQLInternalDatabase},
|
||||
{"SystemUppercase", "SELECT * FROM SYSTEM.USERS", CodeClickHouseSQLInternalDatabase},
|
||||
{"SystemQuoted", "SELECT count() FROM `system`.`tables`", CodeClickHouseSQLInternalDatabase},
|
||||
@@ -139,7 +161,7 @@ func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
|
||||
{"SystemInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l JOIN system.users AS u ON 1 = 1", CodeClickHouseSQLInternalDatabase},
|
||||
{"SystemInIntersect", "SELECT * FROM t INTERSECT SELECT * FROM system.users", CodeClickHouseSQLInternalDatabase},
|
||||
{"InformationSchema", "SELECT * FROM information_schema.tables", CodeClickHouseSQLInternalDatabase},
|
||||
// A query-level setting takes precedence over the one the caller applies.
|
||||
// Takes precedence over the setting the caller applies.
|
||||
{"ReadonlySettingOverride", "SELECT * FROM t SETTINGS readonly = 0", CodeClickHouseSQLReadonlyOverride},
|
||||
{"ReadonlySettingOverrideAmongOthers", "SELECT * FROM t SETTINGS max_threads = 4, readonly = 0", CodeClickHouseSQLReadonlyOverride},
|
||||
}
|
||||
@@ -148,7 +170,33 @@ func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
err := ErrIfStatementIsNotValid(testCase.query)
|
||||
|
||||
assert.Error(t, err)
|
||||
// Required rather than asserted: errors.Asc dereferences the error it is given.
|
||||
require.Error(t, err)
|
||||
assert.True(t, errors.Asc(err, testCase.expectedCode), "expected code %s, got %v", testCase.expectedCode, err)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrIfStatementIsNotValid_ShouldPassButFails(t *testing.T) {
|
||||
testCases := []struct {
|
||||
name string
|
||||
query string
|
||||
expectedCode errors.Code
|
||||
}{
|
||||
// The left operand commits the parser to a subquery, leaving the operator nowhere to bind. Parenthesising only the right operand is fine.
|
||||
{"ParenthesisedUnionLeftOperand", "SELECT a FROM ((SELECT 1 AS a) UNION ALL (SELECT 2 AS a))", CodeClickHouseSQLUnparseable},
|
||||
{"ParenthesisedExceptLeftOperand", "SELECT a FROM ((SELECT 1 AS a) EXCEPT (SELECT 2 AS a))", CodeClickHouseSQLUnparseable},
|
||||
{"ParenthesisedUnionLeftOperandAtStatementLevel", "(SELECT 1 AS a) UNION ALL (SELECT 2 AS a)", CodeClickHouseSQLUnparseable},
|
||||
// The one keyword PR 305 left behind, because ON also opens a join condition.
|
||||
{"UnquotedOnAsColumnName", "SELECT on + 1 FROM t", CodeClickHouseSQLUnparseable},
|
||||
}
|
||||
|
||||
for _, testCase := range testCases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
err := ErrIfStatementIsNotValid(testCase.query)
|
||||
|
||||
// Required rather than asserted: errors.Asc dereferences the error it is given.
|
||||
require.Error(t, err)
|
||||
assert.True(t, errors.Asc(err, testCase.expectedCode), "expected code %s, got %v", testCase.expectedCode, err)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -43,7 +43,7 @@ func ExistsExpression(columns []*schema.Column, key *telemetrytypes.TelemetryFie
|
||||
if len(evolutionsEntries) > 0 && evolutionsEntries[0] != nil {
|
||||
columnName = evolutionsEntries[0].ColumnName
|
||||
}
|
||||
rawPath := fmt.Sprintf("%s.`%s`", columnName, key.Name)
|
||||
rawPath := fmt.Sprintf("%s.%s", columnName, ClickHouseIdentifier(key.Name))
|
||||
if exists {
|
||||
return rawPath + " IS NOT NULL", nil
|
||||
}
|
||||
@@ -88,7 +88,7 @@ func ExistsExpression(columns []*schema.Column, key *telemetrytypes.TelemetryFie
|
||||
|
||||
switch valueType := column.Type.(schema.MapColumnType).ValueType; valueType.GetType() {
|
||||
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumBool, schema.ColumnTypeEnumFloat64:
|
||||
leftOperand := fmt.Sprintf("mapContains(%s, '%s')", column.Name, key.Name)
|
||||
leftOperand := fmt.Sprintf("mapContains(%s, %s)", column.Name, ClickHouseStringLiteral(key.Name))
|
||||
if key.Materialized {
|
||||
leftOperand = telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key)
|
||||
}
|
||||
|
||||
41
pkg/querybuilder/exists_expr_test.go
Normal file
41
pkg/querybuilder/exists_expr_test.go
Normal file
@@ -0,0 +1,41 @@
|
||||
package querybuilder
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// ExistsExpression is a per-physical-key primitive: family composition happens
|
||||
// at the logical-field layer, so this only ever sees one spelling.
|
||||
func TestExistsExpressionIsPerKey(t *testing.T) {
|
||||
columns := []*schema.Column{{
|
||||
Name: "attributes_string",
|
||||
Type: schema.MapColumnType{
|
||||
KeyType: schema.LowCardinalityColumnType{ElementType: schema.ColumnTypeString},
|
||||
ValueType: schema.ColumnTypeString,
|
||||
},
|
||||
}}
|
||||
|
||||
plain := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
expression, err := ExistsExpression(columns, plain, 0, 0, "unused", true)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "mapContains(attributes_string, 'deployment.environment')", expression)
|
||||
|
||||
materialized := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
Materialized: true,
|
||||
}
|
||||
expression, err = ExistsExpression(columns, materialized, 0, 0, "unused", false)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "NOT `attribute_string_deployment$$environment_exists`", expression)
|
||||
}
|
||||
@@ -21,24 +21,25 @@ const (
|
||||
hasTokenFunctionDocURL = "https://signoz.io/docs/userguide/functions-reference/#hastoken-function"
|
||||
)
|
||||
|
||||
// ResolveKeys picks which matching field keys a filter term builds conditions for.
|
||||
// With 0 or 1 match it returns the input unchanged and no warning. When a name is
|
||||
// ambiguous it returns a warning; a resource+attribute mix defaults to the resource
|
||||
// keys (the common intent), noted in the warning.
|
||||
func ResolveKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeysForName []*telemetrytypes.TelemetryFieldKey) ([]*telemetrytypes.TelemetryFieldKey, string) {
|
||||
if len(fieldKeysForName) <= 1 {
|
||||
return fieldKeysForName, ""
|
||||
// ResolveLogicalFields picks which logical fields a filter term builds conditions
|
||||
// for. With 0 or 1 field it returns the input unchanged and no warning. When a
|
||||
// name is ambiguous (several logical fields — a family is one field and never
|
||||
// ambiguous with itself) it returns a warning; a resource+attribute mix defaults
|
||||
// to the resource fields (the common intent), noted in the warning.
|
||||
func ResolveLogicalFields(field *telemetrytypes.TelemetryFieldKey, logicalFields []*telemetrytypes.LogicalField) ([]*telemetrytypes.LogicalField, string) {
|
||||
if len(logicalFields) <= 1 {
|
||||
return logicalFields, ""
|
||||
}
|
||||
|
||||
warning := fmt.Sprintf(
|
||||
"Key `%s` is ambiguous, found %d different combinations of field context / data type: %v.",
|
||||
field.Name,
|
||||
len(fieldKeysForName),
|
||||
fieldKeysForName,
|
||||
len(logicalFields),
|
||||
logicalFields,
|
||||
)
|
||||
|
||||
hasResource, hasAttribute := false, false
|
||||
for _, item := range fieldKeysForName {
|
||||
for _, item := range logicalFields {
|
||||
switch item.FieldContext {
|
||||
case telemetrytypes.FieldContextResource:
|
||||
hasResource = true
|
||||
@@ -49,18 +50,28 @@ func ResolveKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeysForName []*te
|
||||
|
||||
// when there is both resource and attribute context, default to resource only
|
||||
if hasResource && hasAttribute {
|
||||
filteredKeys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(fieldKeysForName))
|
||||
for _, item := range fieldKeysForName {
|
||||
filtered := make([]*telemetrytypes.LogicalField, 0, len(logicalFields))
|
||||
for _, item := range logicalFields {
|
||||
if item.FieldContext == telemetrytypes.FieldContextResource {
|
||||
filteredKeys = append(filteredKeys, item)
|
||||
filtered = append(filtered, item)
|
||||
}
|
||||
}
|
||||
fieldKeysForName = filteredKeys
|
||||
logicalFields = filtered
|
||||
warning += " " + "Using `resource` context by default. To query attributes explicitly, " +
|
||||
fmt.Sprintf("use the fully qualified name (e.g., 'attribute.%s')", field.Name)
|
||||
}
|
||||
|
||||
return fieldKeysForName, warning
|
||||
return logicalFields, warning
|
||||
}
|
||||
|
||||
// WrapAsLogicalFields wraps physical keys (candidate or synthesized) as
|
||||
// single-member logical fields addressed by the requested spelling.
|
||||
func WrapAsLogicalFields(requestedName string, keys []*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.LogicalField {
|
||||
fields := make([]*telemetrytypes.LogicalField, 0, len(keys))
|
||||
for _, key := range keys {
|
||||
fields = append(fields, telemetrytypes.SingleLogicalField(requestedName, key))
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
// NewKeyNotFoundError builds the error a condition builder returns when a filter term
|
||||
@@ -175,3 +186,15 @@ func NewFunctionUnsupportedError(operator qbtypes.FilterOperator) error {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// SingleKeys flattens single-member logical fields to their member keys. It is
|
||||
// the adapter for signals whose fields are never families (everything except
|
||||
// traces today); a family in the input would be silently narrowed, so callers
|
||||
// must be gated signals.
|
||||
func SingleKeys(fields []*telemetrytypes.LogicalField) []*telemetrytypes.TelemetryFieldKey {
|
||||
keys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(fields))
|
||||
for _, logical := range fields {
|
||||
keys = append(keys, logical.Single())
|
||||
}
|
||||
return keys
|
||||
}
|
||||
|
||||
53
pkg/querybuilder/semconv_materialization_test.go
Normal file
53
pkg/querybuilder/semconv_materialization_test.go
Normal file
@@ -0,0 +1,53 @@
|
||||
package querybuilder_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/tracestelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// A promoted historical member keeps its materialized column inside the family
|
||||
// expression: the member key carries its own Materialized state, so the
|
||||
// logical-field merge needs no sibling bookkeeping.
|
||||
func TestTraceFamilyUsesMaterializedHistoricalMember(t *testing.T) {
|
||||
current := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
historical := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
Materialized: true,
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
current.Name,
|
||||
telemetrytypes.FieldContextAttribute,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
|
||||
matches := querybuilder.MatchingLogicalFields(requested, map[string][]*telemetrytypes.TelemetryFieldKey{
|
||||
current.Name: {current},
|
||||
historical.Name: {historical},
|
||||
})
|
||||
require.Len(t, matches, 1, "family metadata should resolve to one logical field")
|
||||
|
||||
expression, err := tracestelemetryschema.NewFieldMapper().FieldForLogical(context.Background(), valuer.UUID{}, 0, 0, matches[0])
|
||||
require.NoError(t, err, "resolved trace family should map to a value expression")
|
||||
|
||||
assert.Equal(
|
||||
t,
|
||||
"COALESCE(NULLIF(attributes_string['deployment.environment.name'], ''), NULLIF(`attribute_string_deployment$$environment`, ''), '')",
|
||||
expression,
|
||||
"family expression should retain the promoted historical member",
|
||||
)
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
|
||||
"github.com/SigNoz/signoz/pkg/semconv"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -360,7 +361,7 @@ func (v *filterExpressionVisitor) VisitPrimary(ctx *grammar.PrimaryContext) any
|
||||
return ErrorConditionLiteral
|
||||
}
|
||||
}
|
||||
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.TelemetryFieldKey{v.fullTextColumn}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(searchText))
|
||||
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.LogicalField{telemetrytypes.SingleLogicalField(v.fullTextColumn.Name, v.fullTextColumn)}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(searchText))
|
||||
if !ok {
|
||||
return ErrorConditionLiteral
|
||||
}
|
||||
@@ -379,7 +380,7 @@ func (v *filterExpressionVisitor) VisitPrimary(ctx *grammar.PrimaryContext) any
|
||||
// VisitComparison handles all comparison operators.
|
||||
func (v *filterExpressionVisitor) VisitComparison(ctx *grammar.ComparisonContext) any {
|
||||
key := v.Visit(ctx.Key()).(*telemetrytypes.TelemetryFieldKey)
|
||||
matching := MatchingFieldKeys(key, v.fieldKeys)
|
||||
matching := MatchingLogicalFields(key, v.fieldKeys)
|
||||
|
||||
// Handle EXISTS specially
|
||||
if ctx.EXISTS() != nil {
|
||||
@@ -675,7 +676,7 @@ func (v *filterExpressionVisitor) VisitFullText(ctx *grammar.FullTextContext) an
|
||||
v.errors = append(v.errors, "full text search is not supported")
|
||||
return ErrorConditionLiteral
|
||||
}
|
||||
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.TelemetryFieldKey{v.fullTextColumn}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(text))
|
||||
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.LogicalField{telemetrytypes.SingleLogicalField(v.fullTextColumn.Name, v.fullTextColumn)}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(text))
|
||||
if !ok {
|
||||
return ErrorConditionLiteral
|
||||
}
|
||||
@@ -730,7 +731,7 @@ func (v *filterExpressionVisitor) VisitFunctionCall(ctx *grammar.FunctionCallCon
|
||||
return ErrorConditionLiteral
|
||||
}
|
||||
|
||||
conds, ok := v.buildConditions(key, MatchingFieldKeys(key, v.fieldKeys), operator, value)
|
||||
conds, ok := v.buildConditions(key, MatchingLogicalFields(key, v.fieldKeys), operator, value)
|
||||
if !ok {
|
||||
return ErrorConditionLiteral
|
||||
}
|
||||
@@ -922,7 +923,7 @@ func (v *filterExpressionVisitor) VisitKey(ctx *grammar.KeyContext) any {
|
||||
|
||||
// buildConditions invokes the condition builder for a filter term, folding its
|
||||
// warnings/errors into visitor state; returns false if an error was recorded.
|
||||
func (v *filterExpressionVisitor) buildConditions(key *telemetrytypes.TelemetryFieldKey, matching []*telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) ([]string, bool) {
|
||||
func (v *filterExpressionVisitor) buildConditions(key *telemetrytypes.TelemetryFieldKey, matching []*telemetrytypes.LogicalField, op qbtypes.FilterOperator, value any) ([]string, bool) {
|
||||
conds, warns, err := v.conditionBuilder.ConditionFor(v.context, v.orgID, v.startNs, v.endNs, key, v.fieldKeys, qbtypes.ConditionBuilderOptions{SkipResourceFilter: v.skipResourceFilter}, op, value, v.builder)
|
||||
if err != nil {
|
||||
_, _, _, _, errURL, _ := errors.Unwrapb(err)
|
||||
@@ -979,30 +980,123 @@ func assignIfEmpty(s *string, value string) {
|
||||
}
|
||||
}
|
||||
|
||||
// MatchingFieldKeys returns the field keys from the map that match the given key,
|
||||
// honoring any context/data type the user specified.
|
||||
func MatchingFieldKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
|
||||
fieldKeysForName := []*telemetrytypes.TelemetryFieldKey{}
|
||||
// familyMemberNames returns the physical spellings to look up for the
|
||||
// referenced key: the semantic-convention family members (current-first) when
|
||||
// the key can resolve to traces, else just the requested name. Only trace
|
||||
// field mappers understand families today; logs and metrics keep the
|
||||
// requested spelling until theirs land.
|
||||
func familyMemberNames(field *telemetrytypes.TelemetryFieldKey) []string {
|
||||
if field.Signal != telemetrytypes.SignalUnspecified && field.Signal != telemetrytypes.SignalTraces {
|
||||
return []string{field.Name}
|
||||
}
|
||||
return semconv.Members(semconv.KindAttribute, telemetrytypes.FieldKeySelector{
|
||||
Name: field.Name,
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: field.FieldContext,
|
||||
})
|
||||
}
|
||||
|
||||
// match by name; keep items whose context and data type match (unspecified matches any)
|
||||
for _, item := range fieldKeys[field.Name] {
|
||||
if (field.FieldContext == telemetrytypes.FieldContextUnspecified || field.FieldContext == item.FieldContext) &&
|
||||
(field.FieldDataType == telemetrytypes.FieldDataTypeUnspecified || field.FieldDataType == item.FieldDataType) {
|
||||
fieldKeysForName = append(fieldKeysForName, item)
|
||||
}
|
||||
// MatchingLogicalFields resolves the referenced key against the metadata map
|
||||
// into logical fields, honoring any context/data type the user specified.
|
||||
//
|
||||
// Physical keys that are members of one semantic-convention family (traces
|
||||
// only today) group into a single logical field per (signal, context, data
|
||||
// type) identity, members ordered current-first. Every other matching key
|
||||
// becomes its own single-member logical field. Ambiguity is therefore the
|
||||
// length of the returned slice, and a family is never ambiguous with itself.
|
||||
// Members alias the metadata map entries; nothing is copied or mutated.
|
||||
func MatchingLogicalFields(field *telemetrytypes.TelemetryFieldKey, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.LogicalField {
|
||||
members := familyMemberNames(field)
|
||||
memberRank := make(map[string]int, len(members))
|
||||
for i, member := range members {
|
||||
memberRank[member] = i
|
||||
}
|
||||
|
||||
// A context may have been split off a name that legitimately contained it (e.g.
|
||||
// `attribute.key`); also look up the context-prefixed name so both readings resolve.
|
||||
if field.FieldContext != telemetrytypes.FieldContextUnspecified {
|
||||
contextPrefixedFieldName := fmt.Sprintf("%s.%s", field.FieldContext.StringValue(), field.Name)
|
||||
for _, item := range fieldKeys[contextPrefixedFieldName] {
|
||||
// Context already matched via the lookup key; only data type needs checking.
|
||||
if field.FieldDataType == telemetrytypes.FieldDataTypeUnspecified || item.FieldDataType == field.FieldDataType {
|
||||
fieldKeysForName = append(fieldKeysForName, item)
|
||||
fields := make([]*telemetrytypes.LogicalField, 0)
|
||||
indexByIdentity := make(map[string]int)
|
||||
// rank of the family member each physical key matched under; the stored
|
||||
// name of a context-prefixed match differs from the member name.
|
||||
ranks := make(map[*telemetrytypes.TelemetryFieldKey]int)
|
||||
|
||||
appendMatches := func(lookupName string, memberName string, contextAlreadyMatched bool) {
|
||||
for _, item := range fieldKeys[lookupName] {
|
||||
if !contextAlreadyMatched && field.FieldContext != telemetrytypes.FieldContextUnspecified && field.FieldContext != item.FieldContext {
|
||||
continue
|
||||
}
|
||||
if field.FieldDataType != telemetrytypes.FieldDataTypeUnspecified && field.FieldDataType != item.FieldDataType {
|
||||
continue
|
||||
}
|
||||
|
||||
// A wildcard lookup may have found a same-named field in a scope where
|
||||
// this family does not apply. Keep exact names, but reject cross-member
|
||||
// matches outside the generated family scope.
|
||||
traceFamilyMatch := len(members) > 1 && item.Signal == telemetrytypes.SignalTraces
|
||||
if memberName != field.Name {
|
||||
if !traceFamilyMatch {
|
||||
continue
|
||||
}
|
||||
itemSelector := telemetrytypes.FieldKeySelector{
|
||||
Name: field.Name,
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: item.FieldContext,
|
||||
}
|
||||
if !slices.Contains(semconv.Members(semconv.KindAttribute, itemSelector), memberName) {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
if !traceFamilyMatch {
|
||||
fields = append(fields, telemetrytypes.SingleLogicalField(field.Name, item))
|
||||
continue
|
||||
}
|
||||
|
||||
identity := item.Signal.StringValue() + ";" + item.FieldContext.StringValue() + ";" + item.FieldDataType.StringValue()
|
||||
index, found := indexByIdentity[identity]
|
||||
if !found {
|
||||
index = len(fields)
|
||||
indexByIdentity[identity] = index
|
||||
fields = append(fields, &telemetrytypes.LogicalField{
|
||||
Name: field.Name,
|
||||
Signal: item.Signal,
|
||||
FieldContext: item.FieldContext,
|
||||
FieldDataType: item.FieldDataType,
|
||||
})
|
||||
}
|
||||
logical := fields[index]
|
||||
duplicate := false
|
||||
for _, existing := range logical.Members {
|
||||
if existing.Name == item.Name {
|
||||
duplicate = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !duplicate {
|
||||
ranks[item] = memberRank[memberName]
|
||||
logical.Members = append(logical.Members, item)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return fieldKeysForName
|
||||
for _, member := range members {
|
||||
appendMatches(member, member, false)
|
||||
}
|
||||
|
||||
// A context may have been split off a name that legitimately contained it
|
||||
// (e.g. `attribute.key`); preserve that historical alternate reading for
|
||||
// every family member.
|
||||
if field.FieldContext != telemetrytypes.FieldContextUnspecified {
|
||||
for _, member := range members {
|
||||
appendMatches(fmt.Sprintf("%s.%s", field.FieldContext.StringValue(), member), member, true)
|
||||
}
|
||||
}
|
||||
|
||||
// Precedence is a property of the family, not of arrival order: members
|
||||
// sort current-first no matter which lookup pass found them.
|
||||
for _, logical := range fields {
|
||||
slices.SortStableFunc(logical.Members, func(a, b *telemetrytypes.TelemetryFieldKey) int {
|
||||
return ranks[a] - ranks[b]
|
||||
})
|
||||
}
|
||||
|
||||
return fields
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/antlr4-go/antlr/v4"
|
||||
sqlbuilder "github.com/huandu/go-sqlbuilder"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestPrepareWhereClause_EmptyVariableList ensures PrepareWhereClause errors when a variable has an empty list value.
|
||||
@@ -589,8 +590,8 @@ func TestVisitKey(t *testing.T) {
|
||||
// VisitKey only parses; the condition builder matches, resolves ambiguity
|
||||
// and decides not-found handling. Replay that here against the generic
|
||||
// builder behavior (error unless the key is ignored).
|
||||
matching := MatchingFieldKeys(key, tt.fieldKeys)
|
||||
keys, warning := ResolveKeys(key, matching)
|
||||
matching := MatchingLogicalFields(key, tt.fieldKeys)
|
||||
keys, warning := ResolveLogicalFields(key, matching)
|
||||
|
||||
var gotErrors []string
|
||||
var gotMainErrURL, gotMainWrnURL string
|
||||
@@ -612,15 +613,19 @@ func TestVisitKey(t *testing.T) {
|
||||
t.Errorf("expected %d keys, got %d", len(tt.expectedKeys), len(keys))
|
||||
}
|
||||
|
||||
// Check each expected key matches name, field context, and data type
|
||||
// Check each expected key matches a member's stored name plus the
|
||||
// logical field's context and data type (the logical Name is the
|
||||
// requested spelling, members keep the stored spellings).
|
||||
for _, expectedKey := range tt.expectedKeys {
|
||||
found := false
|
||||
for _, key := range keys {
|
||||
if key.Name == expectedKey.Name &&
|
||||
key.FieldContext == expectedKey.FieldContext &&
|
||||
key.FieldDataType == expectedKey.FieldDataType {
|
||||
found = true
|
||||
break
|
||||
for _, logical := range keys {
|
||||
for _, member := range logical.Members {
|
||||
if member.Name == expectedKey.Name &&
|
||||
logical.FieldContext == expectedKey.FieldContext &&
|
||||
logical.FieldDataType == expectedKey.FieldDataType {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
@@ -685,6 +690,156 @@ func TestVisitKey(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func memberNames(logical *telemetrytypes.LogicalField) []string {
|
||||
names := make([]string, 0, len(logical.Members))
|
||||
for _, member := range logical.Members {
|
||||
names = append(names, member.Name)
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
func TestMatchingLogicalFieldsResolvesCurrentTraceNameFromOldMetadata(t *testing.T) {
|
||||
old := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Description: "old metadata",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
"deployment.environment.name",
|
||||
telemetrytypes.FieldContextResource,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
|
||||
matches := MatchingLogicalFields(requested, map[string][]*telemetrytypes.TelemetryFieldKey{old.Name: {old}})
|
||||
|
||||
require.Len(t, matches, 1, "a family is one logical field, not an ambiguity")
|
||||
assert.Equal(t, "deployment.environment.name", matches[0].Name, "the requested spelling is the response identity")
|
||||
assert.Equal(t, []string{"deployment.environment"}, memberNames(matches[0]))
|
||||
assert.Same(t, old, matches[0].Members[0], "members alias metadata entries; nothing is copied")
|
||||
}
|
||||
|
||||
func TestMatchingLogicalFieldsGroupsFamilyMembersCurrentFirst(t *testing.T) {
|
||||
current := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
old := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
old.Name,
|
||||
telemetrytypes.FieldContextResource,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
|
||||
matches := MatchingLogicalFields(requested, map[string][]*telemetrytypes.TelemetryFieldKey{
|
||||
current.Name: {current},
|
||||
old.Name: {old},
|
||||
})
|
||||
|
||||
require.Len(t, matches, 1, "a family is one logical field, not an ambiguity")
|
||||
assert.Equal(t, old.Name, matches[0].Name, "the requested spelling is the response identity")
|
||||
assert.True(t, matches[0].IsFamily())
|
||||
assert.Equal(t, []string{current.Name, old.Name}, memberNames(matches[0]), "members order current-first")
|
||||
}
|
||||
|
||||
// Precedence is a property of the family, not of arrival order: a member that
|
||||
// only exists under its context-prefixed stored spelling still sorts by its
|
||||
// family rank.
|
||||
func TestMatchingLogicalFieldsOrdersMembersByFamilyRank(t *testing.T) {
|
||||
old := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
prefixedCurrent := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "resource.deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
"deployment.environment.name",
|
||||
telemetrytypes.FieldContextResource,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
|
||||
matches := MatchingLogicalFields(requested, map[string][]*telemetrytypes.TelemetryFieldKey{
|
||||
old.Name: {old},
|
||||
prefixedCurrent.Name: {prefixedCurrent},
|
||||
})
|
||||
|
||||
require.Len(t, matches, 1)
|
||||
assert.Equal(t, []string{prefixedCurrent.Name, old.Name}, memberNames(matches[0]),
|
||||
"the current-spelling member must coalesce before the old one")
|
||||
}
|
||||
|
||||
func TestMatchingLogicalFieldsKeepsLogSemconvNamesLiteral(t *testing.T) {
|
||||
current := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalLogs,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
old := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalLogs,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
current.Name,
|
||||
telemetrytypes.FieldContextResource,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
|
||||
matches := MatchingLogicalFields(requested, map[string][]*telemetrytypes.TelemetryFieldKey{
|
||||
current.Name: {current},
|
||||
old.Name: {old},
|
||||
})
|
||||
|
||||
require.Len(t, matches, 1, "log lookup must keep the requested spelling literal")
|
||||
assert.False(t, matches[0].IsFamily())
|
||||
assert.Equal(t, current.Name, matches[0].Single().Name)
|
||||
}
|
||||
|
||||
func TestMatchingLogicalFieldsKeepsMetricSemconvNamesLiteral(t *testing.T) {
|
||||
current := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalMetrics,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
old := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalMetrics,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
current.Name,
|
||||
telemetrytypes.FieldContextResource,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
|
||||
matches := MatchingLogicalFields(requested, map[string][]*telemetrytypes.TelemetryFieldKey{
|
||||
current.Name: {current},
|
||||
old.Name: {old},
|
||||
})
|
||||
|
||||
require.Len(t, matches, 1, "metric lookup must keep the requested spelling literal")
|
||||
assert.False(t, matches[0].IsFamily())
|
||||
assert.Equal(t, current.Name, matches[0].Single().Name)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// TestVisitComparison
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -766,7 +921,7 @@ func (b *resourceConditionBuilder) ConditionFor(
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
keys, warning := ResolveKeys(key, MatchingFieldKeys(key, fieldKeys))
|
||||
keys, warning := ResolveLogicalFields(key, MatchingLogicalFields(key, fieldKeys))
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
@@ -774,11 +929,11 @@ func (b *resourceConditionBuilder) ConditionFor(
|
||||
|
||||
var conds []string
|
||||
for _, k := range keys {
|
||||
// only resource keys contribute; others (and unknown keys) are ignored
|
||||
// only resource fields contribute; others (and unknown keys) are ignored
|
||||
if k.FieldContext != telemetrytypes.FieldContextResource {
|
||||
continue
|
||||
}
|
||||
conds = append(conds, fmt.Sprintf("%s_cond", k.Name))
|
||||
conds = append(conds, fmt.Sprintf("%s_cond", k.Single().Name))
|
||||
}
|
||||
return conds, warnings, nil
|
||||
}
|
||||
@@ -808,7 +963,7 @@ func (b *conditionBuilder) ConditionFor(
|
||||
return []string{fmt.Sprintf("%s_cond", key.Name)}, nil, nil
|
||||
}
|
||||
|
||||
keys, warning := ResolveKeys(key, MatchingFieldKeys(key, fieldKeys))
|
||||
keys, warning := ResolveLogicalFields(key, MatchingLogicalFields(key, fieldKeys))
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
@@ -820,7 +975,7 @@ func (b *conditionBuilder) ConditionFor(
|
||||
|
||||
// A resource sub-query already covers the term; drop resource keys from the main query.
|
||||
if options.SkipResourceFilter {
|
||||
filtered := make([]*telemetrytypes.TelemetryFieldKey, 0, len(keys))
|
||||
filtered := make([]*telemetrytypes.LogicalField, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
if k.FieldContext != telemetrytypes.FieldContextResource {
|
||||
filtered = append(filtered, k)
|
||||
|
||||
22
pkg/semconv/families_gen.go
Normal file
22
pkg/semconv/families_gen.go
Normal file
@@ -0,0 +1,22 @@
|
||||
// Code generated by scripts/semconv. DO NOT EDIT.
|
||||
|
||||
package semconv
|
||||
|
||||
var families = []Family{
|
||||
{
|
||||
Current: "db.system.name",
|
||||
Old: []string{"db.system"},
|
||||
Kind: KindAttribute,
|
||||
Contexts: nil,
|
||||
Signals: nil,
|
||||
ApplyToMetrics: nil,
|
||||
},
|
||||
{
|
||||
Current: "deployment.environment.name",
|
||||
Old: []string{"deployment.environment"},
|
||||
Kind: KindAttribute,
|
||||
Contexts: nil,
|
||||
Signals: nil,
|
||||
ApplyToMetrics: nil,
|
||||
},
|
||||
}
|
||||
127
pkg/semconv/semconv.go
Normal file
127
pkg/semconv/semconv.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package semconv
|
||||
|
||||
import (
|
||||
"slices"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
//go:generate go run ../../scripts/semconv
|
||||
|
||||
// Kind identifies whether a family describes an attribute or a metric name.
|
||||
type Kind struct {
|
||||
valuer.String
|
||||
}
|
||||
|
||||
// Family is one logical telemetry field. Old is ordered from the most recent
|
||||
// predecessor to the oldest one and therefore also defines fallback order.
|
||||
type Family struct {
|
||||
Current string
|
||||
Old []string
|
||||
Kind Kind
|
||||
Contexts []telemetrytypes.FieldContext
|
||||
Signals []telemetrytypes.Signal
|
||||
ApplyToMetrics []string
|
||||
ValueMap map[string]string
|
||||
}
|
||||
|
||||
var (
|
||||
KindAttribute = Kind{String: valuer.NewString("attribute")}
|
||||
KindMetric = Kind{String: valuer.NewString("metric")}
|
||||
)
|
||||
|
||||
var memberToFamilies, familyMembers = buildIndexes()
|
||||
|
||||
// Enum returns the acceptable values for Kind.
|
||||
func (Kind) Enum() []any {
|
||||
return []any{KindAttribute, KindMetric}
|
||||
}
|
||||
|
||||
// Lookup returns the enabled family containing selector.Name for kind. The
|
||||
// returned family must not be modified.
|
||||
func Lookup(kind Kind, selector telemetrytypes.FieldKeySelector) (Family, bool) {
|
||||
idx, ok := lookupIndex(kind, selector)
|
||||
if !ok {
|
||||
return Family{}, false
|
||||
}
|
||||
return families[idx], true
|
||||
}
|
||||
|
||||
// Members returns the current name first, followed by historical names in
|
||||
// fallback order. A name outside an enabled family is returned unchanged. The
|
||||
// returned slice must not be modified.
|
||||
func Members(kind Kind, selector telemetrytypes.FieldKeySelector) []string {
|
||||
idx, ok := lookupIndex(kind, selector)
|
||||
if !ok {
|
||||
return []string{selector.Name}
|
||||
}
|
||||
return familyMembers[idx]
|
||||
}
|
||||
|
||||
// Current returns the current name for selector.Name, or the input name when
|
||||
// it does not belong to an enabled family.
|
||||
func Current(kind Kind, selector telemetrytypes.FieldKeySelector) string {
|
||||
idx, ok := lookupIndex(kind, selector)
|
||||
if !ok {
|
||||
return selector.Name
|
||||
}
|
||||
return families[idx].Current
|
||||
}
|
||||
|
||||
// All returns every enabled family. The returned slice and families must not be
|
||||
// modified.
|
||||
func All() []Family {
|
||||
return families
|
||||
}
|
||||
|
||||
func buildIndexes() (map[string][]int, [][]string) {
|
||||
index := make(map[string][]int)
|
||||
members := make([][]string, len(families))
|
||||
for i, family := range families {
|
||||
members[i] = make([]string, 0, len(family.Old)+1)
|
||||
members[i] = append(members[i], family.Current)
|
||||
members[i] = append(members[i], family.Old...)
|
||||
index[family.Current] = append(index[family.Current], i)
|
||||
for _, old := range family.Old {
|
||||
index[old] = append(index[old], i)
|
||||
}
|
||||
}
|
||||
return index, members
|
||||
}
|
||||
|
||||
func lookupIndex(kind Kind, selector telemetrytypes.FieldKeySelector) (int, bool) {
|
||||
for _, idx := range memberToFamilies[selector.Name] {
|
||||
if matchesSelector(families[idx], kind, selector) {
|
||||
return idx, true
|
||||
}
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
func matchesSelector(family Family, kind Kind, selector telemetrytypes.FieldKeySelector) bool {
|
||||
if family.Kind != kind {
|
||||
return false
|
||||
}
|
||||
|
||||
if selector.Signal != telemetrytypes.SignalUnspecified && len(family.Signals) > 0 {
|
||||
if !slices.Contains(family.Signals, selector.Signal) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
if selector.FieldContext != telemetrytypes.FieldContextUnspecified && len(family.Contexts) > 0 {
|
||||
if !slices.Contains(family.Contexts, selector.FieldContext) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
if selector.Signal == telemetrytypes.SignalMetrics && len(family.ApplyToMetrics) > 0 {
|
||||
if selector.MetricContext == nil {
|
||||
return false
|
||||
}
|
||||
return slices.Contains(family.ApplyToMetrics, selector.MetricContext.MetricName)
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
80
pkg/semconv/semconv_test.go
Normal file
80
pkg/semconv/semconv_test.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package semconv
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestMembersReturnsCurrentBeforeHistoricalName(t *testing.T) {
|
||||
selector := telemetrytypes.FieldKeySelector{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
}
|
||||
|
||||
assert.Equal(t,
|
||||
[]string{"deployment.environment.name", "deployment.environment"},
|
||||
Members(KindAttribute, selector),
|
||||
"members should use current-first fallback order",
|
||||
)
|
||||
}
|
||||
|
||||
func TestCurrentReturnsCanonicalName(t *testing.T) {
|
||||
selector := telemetrytypes.FieldKeySelector{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
}
|
||||
|
||||
assert.Equal(t,
|
||||
"deployment.environment.name",
|
||||
Current(KindAttribute, selector),
|
||||
"historical name should resolve to the current family name",
|
||||
)
|
||||
}
|
||||
|
||||
func TestAllScopedFamilyMatchesSupportedScopes(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
signal telemetrytypes.Signal
|
||||
fieldContext telemetrytypes.FieldContext
|
||||
}{
|
||||
{name: "trace resource", signal: telemetrytypes.SignalTraces, fieldContext: telemetrytypes.FieldContextResource},
|
||||
{name: "trace attribute", signal: telemetrytypes.SignalTraces, fieldContext: telemetrytypes.FieldContextAttribute},
|
||||
{name: "log resource", signal: telemetrytypes.SignalLogs, fieldContext: telemetrytypes.FieldContextResource},
|
||||
{name: "log attribute", signal: telemetrytypes.SignalLogs, fieldContext: telemetrytypes.FieldContextAttribute},
|
||||
{name: "metric resource", signal: telemetrytypes.SignalMetrics, fieldContext: telemetrytypes.FieldContextResource},
|
||||
{name: "metric attribute", signal: telemetrytypes.SignalMetrics, fieldContext: telemetrytypes.FieldContextAttribute},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
selector := telemetrytypes.FieldKeySelector{
|
||||
Name: "deployment.environment",
|
||||
Signal: test.signal,
|
||||
FieldContext: test.fieldContext,
|
||||
}
|
||||
|
||||
assert.Equal(t,
|
||||
"deployment.environment.name",
|
||||
Current(KindAttribute, selector),
|
||||
"an all-scoped family should match every supported signal and attribute context",
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMembersReturnsInputWhenKindDoesNotMatch(t *testing.T) {
|
||||
selector := telemetrytypes.FieldKeySelector{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
}
|
||||
|
||||
assert.Equal(t,
|
||||
[]string{"deployment.environment"},
|
||||
Members(KindMetric, selector),
|
||||
"an attribute family must not match a metric-name lookup",
|
||||
)
|
||||
}
|
||||
@@ -139,7 +139,7 @@ func NewModules(
|
||||
OrgGetter: orgGetter,
|
||||
OrgSetter: orgSetter,
|
||||
Preference: implpreference.NewModule(implpreference.NewStore(sqlstore), preferencetypes.NewAvailablePreference()),
|
||||
SavedView: implsavedview.NewModule(sqlstore),
|
||||
SavedView: implsavedview.NewModule(implsavedview.NewStore(sqlstore)),
|
||||
Apdex: implapdex.NewModule(sqlstore),
|
||||
Dashboard: dashboard,
|
||||
UserSetter: userSetter,
|
||||
|
||||
@@ -30,6 +30,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/modules/promote"
|
||||
"github.com/SigNoz/signoz/pkg/modules/rawdataexport"
|
||||
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
|
||||
"github.com/SigNoz/signoz/pkg/modules/savedview"
|
||||
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
|
||||
"github.com/SigNoz/signoz/pkg/modules/session"
|
||||
"github.com/SigNoz/signoz/pkg/modules/spanmapper"
|
||||
@@ -88,6 +89,7 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
|
||||
struct{ tracedetail.Handler }{},
|
||||
struct{ ruler.Handler }{},
|
||||
struct{ statsreporter.Handler }{},
|
||||
struct{ savedview.Handler }{},
|
||||
).New(ctx, instrumentation.ToProviderSettings(), apiserver.Config{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -235,6 +235,8 @@ func NewSQLMigrationProviderFactories(
|
||||
sqlmigration.NewFillDashboardSpecCollectionsFactory(sqlstore, dashboardStore),
|
||||
sqlmigration.NewScrubEmailChannelTransportFactory(sqlstore),
|
||||
sqlmigration.NewAddDashboardTuplesFactory(sqlstore),
|
||||
sqlmigration.NewRestructureSavedViewSpecFactory(sqlstore, sqlschema),
|
||||
sqlmigration.NewAddSavedViewTuplesFactory(sqlstore),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -335,6 +337,7 @@ func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.Au
|
||||
handlers.TraceDetail,
|
||||
handlers.RulerHandler,
|
||||
handlers.StatsHandler,
|
||||
handlers.SavedView,
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
304
pkg/sqlmigration/109_restructure_saved_view_spec.go
Normal file
304
pkg/sqlmigration/109_restructure_saved_view_spec.go
Normal file
@@ -0,0 +1,304 @@
|
||||
package sqlmigration
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/uptrace/bun"
|
||||
"github.com/uptrace/bun/migrate"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/sqlschema"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/types"
|
||||
)
|
||||
|
||||
type restructureSavedViewSpec struct {
|
||||
store sqlstore.SQLStore
|
||||
sqlschema sqlschema.SQLSchema
|
||||
settings factory.ProviderSettings
|
||||
}
|
||||
|
||||
func NewRestructureSavedViewSpecFactory(store sqlstore.SQLStore, sqlschema sqlschema.SQLSchema) factory.ProviderFactory[SQLMigration, Config] {
|
||||
return factory.NewProviderFactory(factory.MustNewName("restructure_saved_view_spec"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
|
||||
return &restructureSavedViewSpec{store: store, sqlschema: sqlschema, settings: ps}, nil
|
||||
})
|
||||
}
|
||||
|
||||
func (migration *restructureSavedViewSpec) Register(migrations *migrate.Migrations) error {
|
||||
return migrations.Register(migration.Up, migration.Down)
|
||||
}
|
||||
|
||||
// legacySavedViewCompositeQuery is the bare shape saved_view.data held
|
||||
// before this migration -- just the relevant fields of composite query.
|
||||
// Queries is kept as raw JSON since the migration only needs to relocate it, not interpret it.
|
||||
type legacySavedViewCompositeQuery struct {
|
||||
PanelType string `json:"panelType"`
|
||||
Queries json.RawMessage `json:"queries"`
|
||||
}
|
||||
|
||||
// legacySavedViewExtraData mirrors the frontend defined extraData JSON shape.
|
||||
type legacySavedViewExtraData struct {
|
||||
Color string `json:"color,omitempty"`
|
||||
SelectColumns json.RawMessage `json:"selectColumns,omitempty"`
|
||||
Format string `json:"format,omitempty"`
|
||||
MaxLines int `json:"maxLines,omitempty"`
|
||||
FontSize string `json:"fontSize,omitempty"`
|
||||
}
|
||||
|
||||
type savedViewDisplay struct {
|
||||
MaxLines int `json:"maxLines"`
|
||||
FontSize string `json:"fontSize"`
|
||||
Format string `json:"format"`
|
||||
Color string `json:"color"`
|
||||
}
|
||||
|
||||
type savedViewSpec struct {
|
||||
DisplayName string `json:"displayName"`
|
||||
PanelType string `json:"panelType"`
|
||||
Queries json.RawMessage `json:"queries"`
|
||||
SelectedFields json.RawMessage `json:"selectedFields"`
|
||||
Display savedViewDisplay `json:"display"`
|
||||
}
|
||||
|
||||
type savedViewData struct {
|
||||
SchemaVersion string `json:"schemaVersion"`
|
||||
Spec savedViewSpec `json:"spec"`
|
||||
}
|
||||
|
||||
const migrationSavedViewNameSuffixLen = 8
|
||||
|
||||
// slugifySavedViewName turns a pre-existing free-text saved view name and is copy of
|
||||
// dashboardtypes.generateDashboardName.
|
||||
func slugifySavedViewName(displayName string) string {
|
||||
const dns1123LabelMaxLen = 63
|
||||
suffixAlphabet := []byte("abcdefghijklmnopqrstuvwxyz0123456789")
|
||||
|
||||
var b strings.Builder
|
||||
b.Grow(len(displayName))
|
||||
prevHyphen := false
|
||||
for _, r := range strings.ToLower(displayName) {
|
||||
switch {
|
||||
case (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9'):
|
||||
b.WriteRune(r)
|
||||
prevHyphen = false
|
||||
case b.Len() > 0 && !prevHyphen:
|
||||
b.WriteByte('-')
|
||||
prevHyphen = true
|
||||
}
|
||||
}
|
||||
prefix := strings.TrimRight(b.String(), "-")
|
||||
|
||||
suffix := make([]byte, migrationSavedViewNameSuffixLen)
|
||||
if _, err := rand.Read(suffix); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
for i := range suffix {
|
||||
suffix[i] = suffixAlphabet[int(suffix[i])%len(suffixAlphabet)]
|
||||
}
|
||||
|
||||
maxPrefix := dns1123LabelMaxLen - 1 - migrationSavedViewNameSuffixLen
|
||||
if len(prefix) > maxPrefix {
|
||||
prefix = strings.TrimRight(prefix[:maxPrefix], "-")
|
||||
}
|
||||
if prefix == "" {
|
||||
return string(suffix)
|
||||
}
|
||||
return prefix + "-" + string(suffix)
|
||||
}
|
||||
|
||||
// storableLegacySavedView is the shape of the `saved_views` table before this migration.
|
||||
type storableLegacySavedView struct {
|
||||
bun.BaseModel `bun:"table:saved_views"`
|
||||
|
||||
ID string `bun:"id"`
|
||||
Name string `bun:"name"`
|
||||
SourcePage string `bun:"source_page"`
|
||||
Data string `bun:"data"`
|
||||
ExtraData string `bun:"extra_data"`
|
||||
OrgID string `bun:"org_id"`
|
||||
CreatedAt time.Time `bun:"created_at"`
|
||||
UpdatedAt time.Time `bun:"updated_at"`
|
||||
CreatedBy string `bun:"created_by"`
|
||||
UpdatedBy string `bun:"updated_by"`
|
||||
}
|
||||
|
||||
// storableSavedView is the shape of the `saved_view` table this migration creates.
|
||||
type storableSavedView struct {
|
||||
bun.BaseModel `bun:"table:saved_view"`
|
||||
|
||||
ID string `bun:"id,pk,type:text"`
|
||||
OrgID string `bun:"org_id,type:text,notnull"`
|
||||
Name string `bun:"name,type:text,notnull"`
|
||||
Source string `bun:"source,type:text,notnull"`
|
||||
Data string `bun:"data,type:text,notnull"`
|
||||
CreatedAt time.Time `bun:"created_at,notnull"`
|
||||
UpdatedAt time.Time `bun:"updated_at,notnull"`
|
||||
CreatedBy string `bun:"created_by,type:text,notnull"`
|
||||
UpdatedBy string `bun:"updated_by,type:text,notnull"`
|
||||
}
|
||||
|
||||
func (migration *restructureSavedViewSpec) Up(ctx context.Context, db *bun.DB) error {
|
||||
// check if the `saved_view` table already exists
|
||||
if _, _, err := migration.sqlschema.GetTable(ctx, sqlschema.TableName("saved_view")); err == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
savedViewsTable, _, err := migration.sqlschema.GetTable(ctx, sqlschema.TableName("saved_views"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
tx, err := db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
var oldSavedViews []*storableLegacySavedView
|
||||
if err := tx.NewSelect().Model(&oldSavedViews).Scan(ctx); err != nil && err != sql.ErrNoRows {
|
||||
return err
|
||||
}
|
||||
|
||||
var orgIDs []string
|
||||
if err := tx.NewSelect().Model((*types.Organization)(nil)).Column("id").Scan(ctx, &orgIDs); err != nil {
|
||||
return err
|
||||
}
|
||||
validOrgIDs := make(map[string]struct{}, len(orgIDs))
|
||||
for _, id := range orgIDs {
|
||||
validOrgIDs[id] = struct{}{}
|
||||
}
|
||||
|
||||
// drop table `saved_views`
|
||||
for _, sql := range migration.sqlschema.Operator().DropTable(savedViewsTable) {
|
||||
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// create table `saved_view` with the final required schema
|
||||
for _, sql := range migration.sqlschema.Operator().CreateTable(&sqlschema.Table{
|
||||
Name: "saved_view",
|
||||
Columns: []*sqlschema.Column{
|
||||
{Name: "id", DataType: sqlschema.DataTypeText, Nullable: false},
|
||||
{Name: "org_id", DataType: sqlschema.DataTypeText, Nullable: false},
|
||||
{Name: "name", DataType: sqlschema.DataTypeText, Nullable: false},
|
||||
{Name: "source", DataType: sqlschema.DataTypeText, Nullable: false},
|
||||
{Name: "data", DataType: sqlschema.DataTypeText, Nullable: false},
|
||||
{Name: "created_at", DataType: sqlschema.DataTypeTimestamp, Nullable: false},
|
||||
{Name: "updated_at", DataType: sqlschema.DataTypeTimestamp, Nullable: false},
|
||||
{Name: "created_by", DataType: sqlschema.DataTypeText, Nullable: false},
|
||||
{Name: "updated_by", DataType: sqlschema.DataTypeText, Nullable: false},
|
||||
},
|
||||
PrimaryKeyConstraint: &sqlschema.PrimaryKeyConstraint{
|
||||
ColumnNames: []sqlschema.ColumnName{"id"},
|
||||
},
|
||||
ForeignKeyConstraints: []*sqlschema.ForeignKeyConstraint{
|
||||
{
|
||||
ReferencingColumnName: sqlschema.ColumnName("org_id"),
|
||||
ReferencedTableName: sqlschema.TableName("organizations"),
|
||||
ReferencedColumnName: sqlschema.ColumnName("id"),
|
||||
},
|
||||
},
|
||||
}) {
|
||||
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// convert old saved views to the new shape
|
||||
newSavedViews := make([]*storableSavedView, 0, len(oldSavedViews))
|
||||
var skipped, failed int
|
||||
for _, old := range oldSavedViews {
|
||||
if old.OrgID == "" {
|
||||
skipped++
|
||||
continue // orphaned row from a pre-existing org_id backfill gap; nothing sane to attach it to
|
||||
}
|
||||
|
||||
// to avoid foreign key constraint issues
|
||||
if _, ok := validOrgIDs[old.OrgID]; !ok {
|
||||
skipped++
|
||||
migration.settings.Logger.WarnContext(ctx, "saved view references an org that no longer exists, skipping", slog.String("org_id", old.OrgID), slog.String("saved_view_id", old.ID))
|
||||
continue
|
||||
}
|
||||
|
||||
var compositeQuery legacySavedViewCompositeQuery
|
||||
if err := json.Unmarshal([]byte(old.Data), &compositeQuery); err != nil {
|
||||
failed++
|
||||
migration.settings.Logger.WarnContext(ctx, "failed to unmarshal saved view data, skipping", slog.String("org_id", old.OrgID), slog.String("saved_view_id", old.ID), slog.Any("error", err))
|
||||
continue // skip the row on error rather than fail the whole migration
|
||||
}
|
||||
|
||||
var extraData legacySavedViewExtraData
|
||||
if old.ExtraData != "" {
|
||||
// best-effort: malformed/older extraData shapes never fail the migration,
|
||||
// they just leave selectedFields/display empty.
|
||||
_ = json.Unmarshal([]byte(old.ExtraData), &extraData)
|
||||
}
|
||||
|
||||
dataJSON, err := json.Marshal(savedViewData{
|
||||
SchemaVersion: "v2",
|
||||
Spec: savedViewSpec{
|
||||
DisplayName: old.Name,
|
||||
PanelType: compositeQuery.PanelType,
|
||||
Queries: compositeQuery.Queries,
|
||||
SelectedFields: extraData.SelectColumns,
|
||||
Display: savedViewDisplay{
|
||||
MaxLines: extraData.MaxLines,
|
||||
FontSize: extraData.FontSize,
|
||||
Format: extraData.Format,
|
||||
Color: extraData.Color,
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Existing names were free text (no slug constraints); the free-text
|
||||
// value is preserved verbatim as data.spec.displayName above, and name is
|
||||
// replaced with a fresh slug so it satisfies the new DNS-1123 + (org_id,
|
||||
// name) uniqueness rules.
|
||||
newSavedViews = append(newSavedViews, &storableSavedView{
|
||||
ID: old.ID,
|
||||
OrgID: old.OrgID,
|
||||
Name: slugifySavedViewName(old.Name),
|
||||
Source: old.SourcePage,
|
||||
Data: string(dataJSON),
|
||||
CreatedAt: old.CreatedAt,
|
||||
UpdatedAt: old.UpdatedAt,
|
||||
CreatedBy: old.CreatedBy,
|
||||
UpdatedBy: old.UpdatedBy,
|
||||
})
|
||||
}
|
||||
|
||||
if len(newSavedViews) > 0 {
|
||||
if _, err := tx.NewInsert().Model(&newSavedViews).Exec(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
migration.settings.Logger.InfoContext(ctx, "restructured saved views", slog.Int("total", len(oldSavedViews)), slog.Int("migrated", len(newSavedViews)), slog.Int("skipped", skipped), slog.Int("failed", failed))
|
||||
|
||||
// add unique index on (org_id, name)
|
||||
for _, sql := range migration.sqlschema.Operator().CreateIndex(&sqlschema.UniqueIndex{
|
||||
TableName: "saved_view",
|
||||
ColumnNames: []sqlschema.ColumnName{"org_id", "name"},
|
||||
}) {
|
||||
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func (migration *restructureSavedViewSpec) Down(context.Context, *bun.DB) error {
|
||||
// this migration is not reversible as we're transforming the structure
|
||||
return nil
|
||||
}
|
||||
144
pkg/sqlmigration/110_add_saved_view_tuples.go
Normal file
144
pkg/sqlmigration/110_add_saved_view_tuples.go
Normal file
@@ -0,0 +1,144 @@
|
||||
package sqlmigration
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/sqlstore"
|
||||
"github.com/SigNoz/signoz/pkg/types/authtypes"
|
||||
"github.com/oklog/ulid/v2"
|
||||
"github.com/uptrace/bun"
|
||||
"github.com/uptrace/bun/dialect"
|
||||
"github.com/uptrace/bun/migrate"
|
||||
)
|
||||
|
||||
type addSavedViewTuples struct {
|
||||
sqlstore sqlstore.SQLStore
|
||||
}
|
||||
|
||||
func NewAddSavedViewTuplesFactory(sqlstore sqlstore.SQLStore) factory.ProviderFactory[SQLMigration, Config] {
|
||||
return factory.NewProviderFactory(factory.MustNewName("add_saved_view_tuples"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
|
||||
return &addSavedViewTuples{sqlstore: sqlstore}, nil
|
||||
})
|
||||
}
|
||||
|
||||
func (migration *addSavedViewTuples) Register(migrations *migrate.Migrations) error {
|
||||
return migrations.Register(migration.Up, migration.Down)
|
||||
}
|
||||
|
||||
func (migration *addSavedViewTuples) Up(ctx context.Context, db *bun.DB) error {
|
||||
tx, err := db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
var storeID string
|
||||
err = tx.QueryRowContext(ctx, `SELECT id FROM store WHERE name = ? LIMIT 1`, "signoz").Scan(&storeID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var orgIDs []string
|
||||
err = tx.NewSelect().
|
||||
Table("organizations").
|
||||
Column("id").
|
||||
Scan(ctx, &orgIDs)
|
||||
if err != nil && err != sql.ErrNoRows {
|
||||
return err
|
||||
}
|
||||
|
||||
isPG := migration.sqlstore.BunDB().Dialect().Name() == dialect.PG
|
||||
|
||||
// saved-view moved from the legacy ViewAccess/EditAccess role gate to
|
||||
// CheckResources, which on enterprise requires real tuples -- existing orgs
|
||||
// never had these written, only new orgs get them from the registry at bootstrap.
|
||||
tuples := []migrationTuple{
|
||||
{authtypes.SigNozAdminRoleName, "metaresource", "saved-view", "create"},
|
||||
{authtypes.SigNozAdminRoleName, "metaresource", "saved-view", "read"},
|
||||
{authtypes.SigNozAdminRoleName, "metaresource", "saved-view", "update"},
|
||||
{authtypes.SigNozAdminRoleName, "metaresource", "saved-view", "delete"},
|
||||
{authtypes.SigNozAdminRoleName, "metaresource", "saved-view", "list"},
|
||||
{authtypes.SigNozEditorRoleName, "metaresource", "saved-view", "create"},
|
||||
{authtypes.SigNozEditorRoleName, "metaresource", "saved-view", "read"},
|
||||
{authtypes.SigNozEditorRoleName, "metaresource", "saved-view", "update"},
|
||||
{authtypes.SigNozEditorRoleName, "metaresource", "saved-view", "delete"},
|
||||
{authtypes.SigNozEditorRoleName, "metaresource", "saved-view", "list"},
|
||||
{authtypes.SigNozViewerRoleName, "metaresource", "saved-view", "read"},
|
||||
{authtypes.SigNozViewerRoleName, "metaresource", "saved-view", "list"},
|
||||
}
|
||||
|
||||
for _, orgID := range orgIDs {
|
||||
for _, tuple := range tuples {
|
||||
entropy := ulid.DefaultEntropy()
|
||||
now := time.Now().UTC()
|
||||
tupleID := ulid.MustNew(ulid.Timestamp(now), entropy).String()
|
||||
|
||||
objectID := "organization/" + orgID + "/" + tuple.objectName + "/*"
|
||||
roleSubject := "organization/" + orgID + "/role/" + tuple.roleName
|
||||
|
||||
if isPG {
|
||||
user := "role:" + roleSubject + "#assignee"
|
||||
result, err := tx.ExecContext(ctx, `
|
||||
INSERT INTO tuple (store, object_type, object_id, relation, _user, user_type, ulid, inserted_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (store, object_type, object_id, relation, _user) DO NOTHING`,
|
||||
storeID, tuple.objectType, objectID, tuple.relation, user, "userset", tupleID, now,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rowsAffected, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if rowsAffected == 0 {
|
||||
continue
|
||||
}
|
||||
_, err = tx.ExecContext(ctx, `
|
||||
INSERT INTO changelog (store, object_type, object_id, relation, _user, operation, ulid, inserted_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
|
||||
storeID, tuple.objectType, objectID, tuple.relation, user, 0, tupleID, now,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
result, err := tx.ExecContext(ctx, `
|
||||
INSERT INTO tuple (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, user_type, ulid, inserted_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation) DO NOTHING`,
|
||||
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", "userset", tupleID, now,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rowsAffected, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if rowsAffected == 0 {
|
||||
continue
|
||||
}
|
||||
_, err = tx.ExecContext(ctx, `
|
||||
INSERT INTO changelog (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, operation, ulid, inserted_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
|
||||
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", 0, tupleID, now,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
func (migration *addSavedViewTuples) Down(context.Context, *bun.DB) error {
|
||||
return nil
|
||||
}
|
||||
@@ -1,155 +0,0 @@
|
||||
package aistatementbuilder
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// Span list with a mixed filter: gen_ai spans matching the span-level part, in
|
||||
// traces whose window-clipped aggregates satisfy the trace-level part (the
|
||||
// __trace_scope qualification on the delegated path).
|
||||
func TestBuild_FullSQL_SpanList_TraceScoped(t *testing.T) {
|
||||
b := newTestBuilder(t)
|
||||
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
|
||||
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini' AND trace.output_tokens > 1000"},
|
||||
Limit: 10,
|
||||
}, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
assertSQLEqual(t, `
|
||||
WITH __trace_scope AS (
|
||||
SELECT trace_id,
|
||||
sum(multiIf(mapContains(attributes_number, 'gen_ai.usage.output_tokens'), toFloat64(attributes_number['gen_ai.usage.output_tokens']), NULL)) AS output_tokens
|
||||
FROM signoz_traces.distributed_signoz_index_v3
|
||||
WHERE timestamp >= '1747947419000000000'
|
||||
AND timestamp < '1747983448000000000'
|
||||
AND ts_bucket_start >= 1747945619
|
||||
AND ts_bucket_start <= 1747983448
|
||||
AND ((mapContains(attributes_string, 'gen_ai.request.model') OR mapContains(attributes_string, 'gen_ai.tool.name') OR mapContains(attributes_string, 'gen_ai.agent.name')))
|
||||
GROUP BY trace_id
|
||||
HAVING output_tokens > 1000
|
||||
)
|
||||
SELECT timestamp AS __SELECT_KEY_0_timestamp, trace_id AS __SELECT_KEY_1_trace_id, span_id AS __SELECT_KEY_2_span_id,
|
||||
trace_state AS __SELECT_KEY_3_trace_state, parent_span_id AS __SELECT_KEY_4_parent_span_id, flags AS __SELECT_KEY_5_flags,
|
||||
name AS __SELECT_KEY_6_name, kind AS __SELECT_KEY_7_kind, kind_string AS __SELECT_KEY_8_kind_string, duration_nano AS __SELECT_KEY_9_duration_nano,
|
||||
status_code AS __SELECT_KEY_10_status_code, status_message AS __SELECT_KEY_11_status_message,
|
||||
status_code_string AS __SELECT_KEY_12_status_code_string, events AS __SELECT_KEY_13_events, links AS __SELECT_KEY_14_links,
|
||||
response_status_code AS __SELECT_KEY_15_response_status_code, external_http_url AS __SELECT_KEY_16_external_http_url,
|
||||
http_url AS __SELECT_KEY_17_http_url, external_http_method AS __SELECT_KEY_18_external_http_method,
|
||||
http_method AS __SELECT_KEY_19_http_method, http_host AS __SELECT_KEY_20_http_host, db_name AS __SELECT_KEY_21_db_name,
|
||||
db_operation AS __SELECT_KEY_22_db_operation, has_error AS __SELECT_KEY_23_has_error, is_remote AS __SELECT_KEY_24_is_remote,
|
||||
attributes_string, attributes_number, attributes_bool, resources_string
|
||||
FROM signoz_traces.distributed_signoz_index_v3
|
||||
WHERE trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)
|
||||
AND (((mapContains(attributes_string, 'gen_ai.request.model')
|
||||
OR mapContains(attributes_string, 'gen_ai.tool.name')
|
||||
OR mapContains(attributes_string, 'gen_ai.agent.name')))
|
||||
AND ((attributes_string['gen_ai.request.model'] = 'gpt-4o-mini'
|
||||
AND mapContains(attributes_string, 'gen_ai.request.model'))))
|
||||
AND timestamp >= '1747947419000000000'
|
||||
AND timestamp < '1747983448000000000'
|
||||
AND ts_bucket_start >= 1747945619
|
||||
AND ts_bucket_start <= 1747983448
|
||||
LIMIT 10
|
||||
`, stmt)
|
||||
}
|
||||
|
||||
// A raw filter mixing a resource attribute with a trace-level condition: the resource
|
||||
// part flows through the delegate's fingerprint machinery (__resource_filter CTE),
|
||||
// the trace-level part becomes the __trace_scope qualification.
|
||||
func TestBuild_SpanList_ResourcePlusTraceFilter(t *testing.T) {
|
||||
b := newTestBuilder(t)
|
||||
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
|
||||
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
Filter: &qbtypes.Filter{Expression: "resource.service.name = 'checkout' AND trace.output_tokens > 1000"},
|
||||
Limit: 10,
|
||||
}, nil)
|
||||
require.NoError(t, err)
|
||||
|
||||
got := renderSQL(t, stmt)
|
||||
require.Contains(t, got, "__resource_filter AS (")
|
||||
require.Contains(t, got, "resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter)")
|
||||
require.Contains(t, got, "__trace_scope AS (")
|
||||
require.Contains(t, got, "trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)")
|
||||
require.Contains(t, got, "HAVING output_tokens > 1000")
|
||||
}
|
||||
|
||||
// Without a trace-level condition nothing changes: the span list stays a single
|
||||
// gated span scan (no __trace_scope CTE).
|
||||
func TestBuild_SpanList_NoTraceFilter_NoScope(t *testing.T) {
|
||||
b := newTestBuilder(t)
|
||||
stmt, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
|
||||
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
Filter: &qbtypes.Filter{Expression: "gen_ai.request.model = 'gpt-4o-mini'"},
|
||||
Limit: 10,
|
||||
}, nil)
|
||||
require.NoError(t, err)
|
||||
require.NotContains(t, stmt.Query, "__trace_scope")
|
||||
}
|
||||
|
||||
// The span-list trace-level filter shares the trace list's rules: output-only
|
||||
// aggregates are rejected, OR-mixing the two classes is rejected, and explicitly
|
||||
// trace-level order keys get a targeted error — while bare span columns that happen
|
||||
// to share a name with an aggregate alias (duration_nano) stay orderable.
|
||||
func TestBuild_SpanList_TraceFilter_Validation(t *testing.T) {
|
||||
b := newTestBuilder(t)
|
||||
build := func(q qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) error {
|
||||
q.Signal = telemetrytypes.SignalTraces
|
||||
_, err := b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw, q, nil)
|
||||
return err
|
||||
}
|
||||
|
||||
err := build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Filter: &qbtypes.Filter{Expression: "trace.span_count > 3"},
|
||||
})
|
||||
require.ErrorContains(t, err, `aggregate "span_count" cannot be used`)
|
||||
|
||||
err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Filter: &qbtypes.Filter{Expression: "trace.output_tokens > 1000 OR kind_string = 'Client'"},
|
||||
})
|
||||
require.ErrorContains(t, err, "cannot be combined")
|
||||
|
||||
err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "trace.output_tokens"}}}},
|
||||
})
|
||||
require.ErrorContains(t, err, `ordering the span list by trace-level aggregate "trace.output_tokens" is not supported`)
|
||||
|
||||
err = build(qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Order: []qbtypes.OrderBy{{Key: qbtypes.OrderByKey{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "duration_nano"}}, Direction: qbtypes.OrderDirectionDesc}},
|
||||
Limit: 10,
|
||||
})
|
||||
require.NoError(t, err, "bare duration_nano is a span column, not a trace-level key")
|
||||
}
|
||||
|
||||
// Variables in a trace-level condition on the span list get the trace list's
|
||||
// treatment: substituted as literals, __all__ drops the condition (no scope CTE).
|
||||
func TestBuild_SpanList_TraceFilter_Variables(t *testing.T) {
|
||||
b := newTestBuilder(t)
|
||||
build := func(expr string, vars map[string]qbtypes.VariableItem) (*qbtypes.Statement, error) {
|
||||
return b.Build(context.Background(), valuer.UUID{}, testStartMs, testEndMs, qbtypes.RequestTypeRaw,
|
||||
qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
Filter: &qbtypes.Filter{Expression: expr},
|
||||
Limit: 10,
|
||||
}, vars)
|
||||
}
|
||||
|
||||
stmt, err := build("trace.output_tokens > $threshold",
|
||||
map[string]qbtypes.VariableItem{"threshold": {Value: 700}})
|
||||
require.NoError(t, err)
|
||||
require.Contains(t, stmt.Query, "HAVING output_tokens > 700")
|
||||
|
||||
stmt, err = build("trace.output_tokens > $threshold",
|
||||
map[string]qbtypes.VariableItem{"threshold": {Type: qbtypes.DynamicVariableType, Value: "__all__"}})
|
||||
require.NoError(t, err)
|
||||
require.NotContains(t, stmt.Query, "__trace_scope")
|
||||
}
|
||||
@@ -13,12 +13,14 @@ import (
|
||||
)
|
||||
|
||||
type defaultConditionBuilder struct {
|
||||
fm qbtypes.FieldMapper
|
||||
// The builder composes family expressions, so it needs this package's
|
||||
// mapper, not the narrower qbtypes.FieldMapper.
|
||||
fm *defaultFieldMapper
|
||||
}
|
||||
|
||||
var _ qbtypes.ConditionBuilder = (*defaultConditionBuilder)(nil)
|
||||
|
||||
func NewConditionBuilder(fm qbtypes.FieldMapper) *defaultConditionBuilder {
|
||||
func NewConditionBuilder(fm *defaultFieldMapper) *defaultConditionBuilder {
|
||||
return &defaultConditionBuilder{fm: fm}
|
||||
}
|
||||
|
||||
@@ -44,6 +46,66 @@ func keyIndexFilter(key *telemetrytypes.TelemetryFieldKey) any {
|
||||
return fmt.Sprintf(`%%%s%%`, key.Name)
|
||||
}
|
||||
|
||||
func keyIndexCondition(sb *sqlbuilder.SelectBuilder, column string, members []*telemetrytypes.TelemetryFieldKey) string {
|
||||
conditions := make([]string, 0, len(members))
|
||||
for _, member := range members {
|
||||
conditions = append(conditions, sb.Like(column, keyIndexFilter(member)))
|
||||
}
|
||||
if len(conditions) == 1 {
|
||||
return conditions[0]
|
||||
}
|
||||
return sb.Or(conditions...)
|
||||
}
|
||||
|
||||
func valueIndexCondition(
|
||||
sb *sqlbuilder.SelectBuilder,
|
||||
column string,
|
||||
members []*telemetrytypes.TelemetryFieldKey,
|
||||
op qbtypes.FilterOperator,
|
||||
value any,
|
||||
caseInsensitive bool,
|
||||
) string {
|
||||
conditions := make([]string, 0, len(members))
|
||||
for _, member := range members {
|
||||
patterns := valueForIndexFilter(op, member, value)
|
||||
switch values := patterns.(type) {
|
||||
case []string:
|
||||
for _, pattern := range values {
|
||||
conditions = append(conditions, sb.Like(column, pattern))
|
||||
}
|
||||
default:
|
||||
if caseInsensitive {
|
||||
conditions = append(conditions, sb.ILike(column, values))
|
||||
} else {
|
||||
conditions = append(conditions, sb.Like(column, values))
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(conditions) == 1 {
|
||||
return conditions[0]
|
||||
}
|
||||
return sb.Or(conditions...)
|
||||
}
|
||||
|
||||
func memberPresenceCondition(sb *sqlbuilder.SelectBuilder, column string, members []*telemetrytypes.TelemetryFieldKey, exists bool) string {
|
||||
conditions := make([]string, 0, len(members))
|
||||
for _, member := range members {
|
||||
field := fmt.Sprintf("simpleJSONHas(%s, %s)", column, querybuilder.ClickHouseStringLiteral(member.Name))
|
||||
if exists {
|
||||
conditions = append(conditions, sb.E(field, true))
|
||||
} else {
|
||||
conditions = append(conditions, sb.NE(field, true))
|
||||
}
|
||||
}
|
||||
if exists {
|
||||
if len(conditions) == 1 {
|
||||
return conditions[0]
|
||||
}
|
||||
return sb.Or(conditions...)
|
||||
}
|
||||
return sb.And(conditions...)
|
||||
}
|
||||
|
||||
// SkipResourceFilter is not applicable here: the fingerprint table only stores resource attributes.
|
||||
func (b *defaultConditionBuilder) ConditionFor(
|
||||
ctx context.Context,
|
||||
@@ -57,7 +119,7 @@ func (b *defaultConditionBuilder) ConditionFor(
|
||||
value any,
|
||||
sb *sqlbuilder.SelectBuilder,
|
||||
) ([]string, []string, error) {
|
||||
matches := querybuilder.MatchingFieldKeys(key, fieldKeys)
|
||||
matches := querybuilder.MatchingLogicalFields(key, fieldKeys)
|
||||
|
||||
// has/hasAny/hasAll/hasToken are logs-body-only functions; they never apply to the
|
||||
// resource fingerprint table, so skip them (the main query still evaluates them).
|
||||
@@ -65,21 +127,21 @@ func (b *defaultConditionBuilder) ConditionFor(
|
||||
return nil, nil, nil
|
||||
}
|
||||
|
||||
keys, warning := querybuilder.ResolveKeys(key, matches)
|
||||
logicalFields, warning := querybuilder.ResolveLogicalFields(key, matches)
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
}
|
||||
|
||||
conds := make([]string, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
// the resource fingerprint table only stores resource attributes; keys from
|
||||
conds := make([]string, 0, len(logicalFields))
|
||||
for _, logical := range logicalFields {
|
||||
// the resource fingerprint table only stores resource attributes; fields from
|
||||
// any other context contribute no condition and are omitted. An empty result
|
||||
// (including an unknown key) lets the caller skip this filter entirely.
|
||||
if k.FieldContext != telemetrytypes.FieldContextResource {
|
||||
if logical.FieldContext != telemetrytypes.FieldContextResource {
|
||||
continue
|
||||
}
|
||||
cond, err := b.conditionForKey(ctx, startNs, endNs, k, op, value, sb)
|
||||
cond, err := b.conditionForLogicalField(ctx, startNs, endNs, logical, op, value, sb)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
@@ -88,11 +150,11 @@ func (b *defaultConditionBuilder) ConditionFor(
|
||||
return conds, warnings, nil
|
||||
}
|
||||
|
||||
func (b *defaultConditionBuilder) conditionForKey(
|
||||
func (b *defaultConditionBuilder) conditionForLogicalField(
|
||||
ctx context.Context,
|
||||
startNs uint64,
|
||||
endNs uint64,
|
||||
key *telemetrytypes.TelemetryFieldKey,
|
||||
logical *telemetrytypes.LogicalField,
|
||||
op qbtypes.FilterOperator,
|
||||
value any,
|
||||
sb *sqlbuilder.SelectBuilder,
|
||||
@@ -102,7 +164,7 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
// as we store resource values as string
|
||||
formattedValue := querybuilder.FormatValueForContains(value)
|
||||
|
||||
columns, err := b.fm.ColumnFor(ctx, valuer.UUID{}, startNs, endNs, key)
|
||||
columns, err := b.fm.ColumnFor(ctx, valuer.UUID{}, startNs, endNs, logical.Single())
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
@@ -115,10 +177,12 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
// as we have not changed the resource column in the resource fingerprint table.
|
||||
column := columns[0]
|
||||
|
||||
keyIdxFilter := sb.Like(column.Name, keyIndexFilter(key))
|
||||
valueForIndexFilter := valueForIndexFilter(op, key, value)
|
||||
members := logical.Members
|
||||
isFamily := logical.IsFamily()
|
||||
keyIdxFilter := keyIndexCondition(sb, column.Name, members)
|
||||
singleValueIndexFilter := valueForIndexFilter(op, members[0], value)
|
||||
|
||||
fieldName, err := b.fm.FieldFor(ctx, valuer.UUID{}, startNs, endNs, key)
|
||||
fieldName, err := b.fm.FieldForLogical(ctx, valuer.UUID{}, startNs, endNs, logical)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
@@ -128,12 +192,15 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
return sb.And(
|
||||
sb.E(fieldName, formattedValue),
|
||||
keyIdxFilter,
|
||||
sb.Like(column.Name, valueForIndexFilter),
|
||||
valueIndexCondition(sb, column.Name, members, op, value, false),
|
||||
), nil
|
||||
case qbtypes.FilterOperatorNotEqual:
|
||||
if isFamily {
|
||||
return sb.NE(fieldName, formattedValue), nil
|
||||
}
|
||||
return sb.And(
|
||||
sb.NE(fieldName, formattedValue),
|
||||
sb.NotLike(column.Name, valueForIndexFilter),
|
||||
sb.NotLike(column.Name, singleValueIndexFilter),
|
||||
), nil
|
||||
case qbtypes.FilterOperatorGreaterThan:
|
||||
return sb.And(sb.GT(fieldName, formattedValue), keyIdxFilter), nil
|
||||
@@ -148,7 +215,7 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
return sb.And(
|
||||
sb.ILike(fieldName, formattedValue),
|
||||
keyIdxFilter,
|
||||
sb.ILike(column.Name, valueForIndexFilter),
|
||||
valueIndexCondition(sb, column.Name, members, op, value, true),
|
||||
), nil
|
||||
case qbtypes.FilterOperatorNotLike, qbtypes.FilterOperatorNotILike:
|
||||
// no index filter: as cannot apply `not contains x%y` as y can be somewhere else
|
||||
@@ -185,13 +252,11 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
inConditions = append(inConditions, sb.E(fieldName, querybuilder.FormatValueForContains(v)))
|
||||
}
|
||||
mainCondition := sb.Or(inConditions...)
|
||||
valConditions := make([]string, 0, len(values))
|
||||
if valuesForIndexFilter, ok := valueForIndexFilter.([]string); ok {
|
||||
for _, v := range valuesForIndexFilter {
|
||||
valConditions = append(valConditions, sb.Like(column.Name, v))
|
||||
}
|
||||
}
|
||||
mainCondition = sb.And(mainCondition, keyIdxFilter, sb.Or(valConditions...))
|
||||
mainCondition = sb.And(
|
||||
mainCondition,
|
||||
keyIdxFilter,
|
||||
valueIndexCondition(sb, column.Name, members, op, value, false),
|
||||
)
|
||||
|
||||
return mainCondition, nil
|
||||
case qbtypes.FilterOperatorNotIn:
|
||||
@@ -204,8 +269,11 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
notInConditions = append(notInConditions, sb.NE(fieldName, querybuilder.FormatValueForContains(v)))
|
||||
}
|
||||
mainCondition := sb.And(notInConditions...)
|
||||
if isFamily {
|
||||
return mainCondition, nil
|
||||
}
|
||||
valConditions := make([]string, 0, len(values))
|
||||
if valuesForIndexFilter, ok := valueForIndexFilter.([]string); ok {
|
||||
if valuesForIndexFilter, ok := singleValueIndexFilter.([]string); ok {
|
||||
for _, v := range valuesForIndexFilter {
|
||||
valConditions = append(valConditions, sb.NotLike(column.Name, v))
|
||||
}
|
||||
@@ -215,13 +283,11 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
|
||||
case qbtypes.FilterOperatorExists:
|
||||
return sb.And(
|
||||
sb.E(fmt.Sprintf("simpleJSONHas(%s, '%s')", column.Name, key.Name), true),
|
||||
memberPresenceCondition(sb, column.Name, members, true),
|
||||
keyIdxFilter,
|
||||
), nil
|
||||
case qbtypes.FilterOperatorNotExists:
|
||||
return sb.And(
|
||||
sb.NE(fmt.Sprintf("simpleJSONHas(%s, '%s')", column.Name, key.Name), true),
|
||||
), nil
|
||||
return memberPresenceCondition(sb, column.Name, members, false), nil
|
||||
|
||||
case qbtypes.FilterOperatorRegexp:
|
||||
return sb.And(
|
||||
@@ -237,7 +303,7 @@ func (b *defaultConditionBuilder) conditionForKey(
|
||||
return sb.And(
|
||||
sb.ILike(fieldName, fmt.Sprintf(`%%%s%%`, formattedValue)),
|
||||
keyIdxFilter,
|
||||
sb.ILike(column.Name, valueForIndexFilter),
|
||||
valueIndexCondition(sb, column.Name, members, op, value, true),
|
||||
), nil
|
||||
case qbtypes.FilterOperatorNotContains:
|
||||
// no index filter: as cannot apply `not contains x%y` as y can be somewhere else
|
||||
|
||||
@@ -220,3 +220,129 @@ func TestConditionBuilder(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The family tests drive resolution through the metadata map, exactly as
|
||||
// production does: two plain member keys in the map, one requested spelling.
|
||||
// The keys carry no family bookkeeping — grouping is the resolver's job.
|
||||
func familyConditionSQL(t *testing.T, requestedName string, memberNames []string, op qbtypes.FilterOperator, value any) (string, []any) {
|
||||
t.Helper()
|
||||
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{}
|
||||
for _, name := range memberNames {
|
||||
fieldKeys[name] = []*telemetrytypes.TelemetryFieldKey{{
|
||||
Name: name,
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}}
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
requestedName,
|
||||
telemetrytypes.FieldContextResource,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
conditions, _, err := NewConditionBuilder(NewFieldMapper()).ConditionFor(
|
||||
context.Background(), valuer.UUID{}, 0, 0, requested,
|
||||
fieldKeys,
|
||||
qbtypes.ConditionBuilderOptions{}, op, value, sb,
|
||||
)
|
||||
require.NoError(t, err)
|
||||
sb.Where(conditions...)
|
||||
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
}
|
||||
|
||||
var deploymentFamilyMembers = []string{"deployment.environment.name", "deployment.environment"}
|
||||
|
||||
func TestFamilyPositiveFilterExcludesKeylessRows(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorEqual, "production")
|
||||
|
||||
assert.Contains(t, sql, "COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') = ? AND (labels LIKE ? OR labels LIKE ?) AND (labels LIKE ? OR labels LIKE ?)")
|
||||
assert.Equal(t, []any{
|
||||
"production",
|
||||
"%deployment.environment.name%",
|
||||
"%deployment.environment%",
|
||||
`%deployment.environment.name":"production%`,
|
||||
`%deployment.environment":"production%`,
|
||||
}, args)
|
||||
}
|
||||
|
||||
func TestFamilyNotEqualIncludesKeylessRows(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorNotEqual, "staging")
|
||||
|
||||
assert.Contains(t, sql, "COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') <> ?")
|
||||
assert.Equal(t, []any{"staging"}, args)
|
||||
}
|
||||
|
||||
func TestFamilyNotInIncludesKeylessRows(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorNotIn, []any{"staging", "dev"})
|
||||
|
||||
assert.Contains(t, sql, "(COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') <> ? AND COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') <> ?)")
|
||||
assert.Equal(t, []any{"staging", "dev"}, args)
|
||||
}
|
||||
|
||||
func TestFamilyNotLikeIncludesKeylessRows(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorNotLike, "%stag%")
|
||||
|
||||
assert.Contains(t, sql, "LOWER(COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '')) NOT LIKE LOWER(?)")
|
||||
assert.Equal(t, []any{"%stag%"}, args)
|
||||
}
|
||||
|
||||
func TestFamilyNotContainsIncludesKeylessRows(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorNotContains, "stag")
|
||||
|
||||
assert.Contains(t, sql, "LOWER(COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '')) NOT LIKE LOWER(?)")
|
||||
assert.Equal(t, []any{"%stag%"}, args)
|
||||
}
|
||||
|
||||
func TestFamilyNotRegexpIncludesKeylessRows(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorNotRegexp, "stag.*")
|
||||
|
||||
assert.Contains(t, sql, "NOT match(COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), ''), ?)")
|
||||
assert.Equal(t, []any{"stag.*"}, args)
|
||||
}
|
||||
|
||||
func TestFamilyExistsChecksEveryMember(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorExists, nil)
|
||||
|
||||
assert.Contains(t, sql, "(simpleJSONHas(labels, 'deployment.environment.name') = ? OR simpleJSONHas(labels, 'deployment.environment') = ?) AND (labels LIKE ? OR labels LIKE ?)")
|
||||
assert.Equal(t, []any{true, true, "%deployment.environment.name%", "%deployment.environment%"}, args)
|
||||
}
|
||||
|
||||
func TestFamilyNotExistsChecksEveryMember(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment.name", deploymentFamilyMembers, qbtypes.FilterOperatorNotExists, nil)
|
||||
|
||||
assert.Contains(t, sql, "simpleJSONHas(labels, 'deployment.environment.name') <> ? AND simpleJSONHas(labels, 'deployment.environment') <> ?")
|
||||
assert.Equal(t, []any{true, true}, args)
|
||||
}
|
||||
|
||||
// The old-name request with only the current spelling in metadata prunes to a
|
||||
// single member: plain single-key SQL, no coalesce.
|
||||
func TestFamilyPrunesToPresentMembers(t *testing.T) {
|
||||
sql, args := familyConditionSQL(t, "deployment.environment", []string{"deployment.environment.name"}, qbtypes.FilterOperatorEqual, "production")
|
||||
|
||||
assert.Contains(t, sql, "simpleJSONExtractString(labels, 'deployment.environment.name') = ? AND labels LIKE ? AND labels LIKE ?")
|
||||
assert.Equal(t, []any{"production", "%deployment.environment.name%", `%deployment.environment.name":"production%`}, args)
|
||||
}
|
||||
|
||||
func TestLogSemconvNameStaysLiteral(t *testing.T) {
|
||||
key := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalLogs,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
|
||||
conditions, _, err := NewConditionBuilder(NewFieldMapper()).ConditionFor(
|
||||
context.Background(), valuer.UUID{}, 0, 0, key,
|
||||
map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {key}},
|
||||
qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "production", sb,
|
||||
)
|
||||
require.NoError(t, err)
|
||||
sb.Where(conditions...)
|
||||
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
|
||||
assert.Contains(t, sql, "simpleJSONExtractString(labels, 'deployment.environment.name') = ? AND labels LIKE ? AND labels LIKE ?")
|
||||
assert.NotContains(t, sql, "deployment.environment')")
|
||||
assert.Equal(t, []any{"production", "%deployment.environment.name%", `%deployment.environment.name":"production%`}, args)
|
||||
}
|
||||
|
||||
@@ -3,8 +3,10 @@ package resourcefilter
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -32,6 +34,31 @@ func NewFieldMapper() *defaultFieldMapper {
|
||||
return &defaultFieldMapper{}
|
||||
}
|
||||
|
||||
// FieldForLogical returns the value expression for a resolved logical field:
|
||||
// the member's own expression for a single-member field, and a current-first
|
||||
// merge for a family. Resource label values are strings, so the merge is a
|
||||
// coalesce with a trailing '' that keeps single-key semantics for rows
|
||||
// without any member (see AddDefaultExistsFilter).
|
||||
func (m *defaultFieldMapper) FieldForLogical(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
tsStart, tsEnd uint64,
|
||||
logical *telemetrytypes.LogicalField,
|
||||
) (string, error) {
|
||||
if !logical.IsFamily() {
|
||||
return m.FieldFor(ctx, orgID, tsStart, tsEnd, logical.Single())
|
||||
}
|
||||
values := make([]string, 0, len(logical.Members))
|
||||
for _, member := range logical.Members {
|
||||
expr, err := m.FieldFor(ctx, orgID, tsStart, tsEnd, member)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
values = append(values, fmt.Sprintf("NULLIF(%s, '')", expr))
|
||||
}
|
||||
return "COALESCE(" + strings.Join(values, ", ") + ", '')", nil
|
||||
}
|
||||
|
||||
func (m *defaultFieldMapper) getColumn(
|
||||
_ context.Context,
|
||||
_, _ uint64,
|
||||
@@ -66,7 +93,7 @@ func (m *defaultFieldMapper) FieldFor(
|
||||
return "", err
|
||||
}
|
||||
if key.FieldContext == telemetrytypes.FieldContextResource {
|
||||
return fmt.Sprintf("simpleJSONExtractString(%s, '%s')", columns[0].Name, key.Name), nil
|
||||
return fmt.Sprintf("simpleJSONExtractString(%s, %s)", columns[0].Name, querybuilder.ClickHouseStringLiteral(key.Name)), nil
|
||||
}
|
||||
return columns[0].Name, nil
|
||||
}
|
||||
|
||||
24
pkg/statementbuilder/resourcefilter/field_mapper_test.go
Normal file
24
pkg/statementbuilder/resourcefilter/field_mapper_test.go
Normal file
@@ -0,0 +1,24 @@
|
||||
package resourcefilter
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestFieldForQuotesRequestKeyName(t *testing.T) {
|
||||
key := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "name'\\); SELECT 1 --",
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
}
|
||||
|
||||
expression, err := NewFieldMapper().FieldFor(context.Background(), valuer.UUID{}, 0, 0, key)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, "simpleJSONExtractString(labels, "+querybuilder.ClickHouseStringLiteral(key.Name)+")", expression)
|
||||
}
|
||||
@@ -31,24 +31,15 @@ var (
|
||||
// (e.g. gen_ai spans); the TraceScope decides which spans are in scope and which
|
||||
// per-trace columns to compute.
|
||||
type scopedTraceStatementBuilder struct {
|
||||
logger *slog.Logger
|
||||
metadataStore telemetrytypes.MetadataStore
|
||||
fm qbtypes.FieldMapper
|
||||
cb qbtypes.ConditionBuilder
|
||||
scope TraceScope
|
||||
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
|
||||
// scopedDelegate is traceStmtBuilder's trace-scoping capability, resolved once at
|
||||
// construction; nil when the delegate cannot constrain a query by trace ids.
|
||||
scopedDelegate traceScopedStatementBuilder
|
||||
logger *slog.Logger
|
||||
metadataStore telemetrytypes.MetadataStore
|
||||
fm qbtypes.FieldMapper
|
||||
cb qbtypes.ConditionBuilder
|
||||
scope TraceScope
|
||||
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
|
||||
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
|
||||
}
|
||||
|
||||
// traceScopedStatementBuilder is the delegate's optional capability of constraining a
|
||||
// query to a set of trace ids (implemented by the traces statement builder).
|
||||
type traceScopedStatementBuilder interface {
|
||||
BuildTraceScoped(ctx context.Context, orgID valuer.UUID, start, end uint64, requestType qbtypes.RequestType, query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation], variables map[string]qbtypes.VariableItem, traceScope *qbtypes.Statement) (*qbtypes.Statement, error)
|
||||
}
|
||||
|
||||
var _ qbtypes.StatementBuilder[qbtypes.TraceAggregation] = (*scopedTraceStatementBuilder)(nil)
|
||||
|
||||
// NewFactory returns a provider factory for a scoped trace statement builder. The
|
||||
@@ -99,7 +90,6 @@ func NewScopedTraceStatementBuilder(
|
||||
fl,
|
||||
)
|
||||
|
||||
scopedDelegate, _ := traceStmtBuilder.(traceScopedStatementBuilder)
|
||||
return &scopedTraceStatementBuilder{
|
||||
logger: scopedSettings.Logger(),
|
||||
metadataStore: metadataStore,
|
||||
@@ -107,7 +97,6 @@ func NewScopedTraceStatementBuilder(
|
||||
cb: conditionBuilder,
|
||||
scope: scope,
|
||||
traceStmtBuilder: traceStmtBuilder,
|
||||
scopedDelegate: scopedDelegate,
|
||||
resourceFilterStmtBuilder: resourceFilterStmtBuilder,
|
||||
}
|
||||
}
|
||||
@@ -125,18 +114,14 @@ func (b *scopedTraceStatementBuilder) Build(
|
||||
case qbtypes.RequestTypeTrace:
|
||||
return b.buildTraceListQuery(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), query, variables)
|
||||
case qbtypes.RequestTypeRaw:
|
||||
if err := b.validateRawOrderKeys(query); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return b.buildDelegated(ctx, orgID, start, end, requestType, query, variables)
|
||||
default:
|
||||
return nil, ErrUnsupportedRequestType
|
||||
}
|
||||
}
|
||||
|
||||
// buildDelegated serves the span-list / raw path: the gate is ANDed into the filter's
|
||||
// span-level part and the query delegates to the standard trace builder; a trace-level
|
||||
// part becomes a qualification the delegate constrains trace_id by.
|
||||
// buildDelegated ANDs the base gate into the user filter and delegates to the
|
||||
// standard trace builder (the span-list / raw path).
|
||||
func (b *scopedTraceStatementBuilder) buildDelegated(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
@@ -145,127 +130,17 @@ func (b *scopedTraceStatementBuilder) buildDelegated(
|
||||
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
|
||||
variables map[string]qbtypes.VariableItem,
|
||||
) (*qbtypes.Statement, error) {
|
||||
var spanExpr, traceExpr string
|
||||
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
|
||||
var err error
|
||||
spanExpr, traceExpr, err = querybuilder.SplitFilterForAggregates(query.Filter.Expression, b.aggregateAliasSet())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
traceExpr, err := substituteTraceLevelVariables(traceExpr, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
gate := b.scope.FilterExpression
|
||||
expr := gate
|
||||
if strings.TrimSpace(spanExpr) != "" {
|
||||
expr = fmt.Sprintf("(%s) AND (%s)", gate, spanExpr)
|
||||
if query.Filter != nil && strings.TrimSpace(query.Filter.Expression) != "" {
|
||||
expr = fmt.Sprintf("(%s) AND (%s)", gate, query.Filter.Expression)
|
||||
}
|
||||
|
||||
// shallow copy; only Filter is replaced, caller's query untouched
|
||||
gated := query
|
||||
gated.Filter = &qbtypes.Filter{Expression: expr}
|
||||
|
||||
if strings.TrimSpace(traceExpr) == "" {
|
||||
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
|
||||
}
|
||||
|
||||
if b.scopedDelegate == nil {
|
||||
return nil, errors.NewInternalf(errors.CodeInternal, "trace statement builder does not support trace-scoped queries")
|
||||
}
|
||||
scope, err := b.buildTraceScopeStatement(ctx, orgID, querybuilder.ToNanoSecs(start), querybuilder.ToNanoSecs(end), traceExpr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return b.scopedDelegate.BuildTraceScoped(ctx, orgID, start, end, requestType, gated, variables, scope)
|
||||
}
|
||||
|
||||
// validateRawOrderKeys rejects ordering the span list by an explicitly trace-level
|
||||
// aggregate (trace. prefix or trace field context) — the per-trace value does not
|
||||
// exist on span rows. Bare names pass through: they may legitimately be span columns
|
||||
// (duration_nano, timestamp).
|
||||
func (b *scopedTraceStatementBuilder) validateRawOrderKeys(query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]) error {
|
||||
aliases := b.aggregateAliasSet()
|
||||
for _, o := range query.Order {
|
||||
key := telemetrytypes.GetFieldKeyFromKeyText(o.Key.Name)
|
||||
if _, ok := aliases[key.Name]; !ok {
|
||||
continue
|
||||
}
|
||||
if key.FieldContext == telemetrytypes.FieldContextTrace || o.Key.FieldContext == telemetrytypes.FieldContextTrace {
|
||||
return errors.NewInvalidInputf(errors.CodeInvalidInput,
|
||||
"ordering the span list by trace-level aggregate %q is not supported; order by span columns instead (e.g. timestamp, duration_nano)", o.Key.Name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// substituteTraceLevelVariables resolves query variables in a trace-level expression.
|
||||
// The span-level parts bind variables via PrepareWhereClause; trace-level parts are
|
||||
// text rewrites, so variables become literals (a dynamic __all__ drops its condition).
|
||||
func substituteTraceLevelVariables(expr string, variables map[string]qbtypes.VariableItem) (string, error) {
|
||||
if strings.TrimSpace(expr) == "" || len(variables) == 0 {
|
||||
return expr, nil
|
||||
}
|
||||
return qbvariables.ReplaceVariablesInExpression(expr, variables)
|
||||
}
|
||||
|
||||
// buildTraceScopeStatement builds the __trace_scope statement: trace ids whose
|
||||
// window-clipped per-trace aggregates satisfy traceExpr. The same scan as the matched
|
||||
// CTE, minus its span-filter widening, resource prune, ordering and pagination.
|
||||
// start/end are ns.
|
||||
func (b *scopedTraceStatementBuilder) buildTraceScopeStatement(ctx context.Context, orgID valuer.UUID, start, end uint64, traceExpr string) (*qbtypes.Statement, error) {
|
||||
keys, err := b.fetchKeys(ctx, orgID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
maskExpr, resolved, err := b.resolveFor(ctx, orgID, start, end, keys, sb)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
orderableSet := orderableAliasSet(resolved)
|
||||
if err := validateAggregateFilter(traceExpr, orderableSet); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
needed := neededMatchedAliases(nil, traceExpr, orderableSet)
|
||||
selects := []string{"trace_id"}
|
||||
for _, rc := range resolved {
|
||||
if _, ok := needed[rc.alias]; !ok {
|
||||
continue
|
||||
}
|
||||
selects = append(selects, rc.expr+" AS "+quoteAlias(rc.alias))
|
||||
}
|
||||
sb.Select(selects...)
|
||||
sb.From(fmt.Sprintf("%s.%s", tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName))
|
||||
sb.Where(
|
||||
sb.GE("timestamp", fmt.Sprintf("%d", start)),
|
||||
sb.L("timestamp", fmt.Sprintf("%d", end)),
|
||||
sb.GE("ts_bucket_start", start/querybuilder.NsToSeconds-querybuilder.BucketAdjustment),
|
||||
sb.LE("ts_bucket_start", end/querybuilder.NsToSeconds),
|
||||
"("+maskExpr+")",
|
||||
)
|
||||
sb.GroupBy("trace_id")
|
||||
|
||||
// the rewriter matches raw key text, so map the trace. form alongside the bare name
|
||||
columnMap := make(map[string]string, len(orderableSet)*2)
|
||||
for a := range orderableSet {
|
||||
columnMap[a] = quoteAlias(a)
|
||||
columnMap[telemetrytypes.FieldContextTrace.StringValue()+"."+a] = quoteAlias(a)
|
||||
}
|
||||
hv, err := querybuilder.NewHavingExpressionRewriter().Rewrite(traceExpr, columnMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if hv != "" {
|
||||
// escape user text so a literal $ isn't read as an arg marker
|
||||
sb.Having(sqlbuilder.Escape(hv))
|
||||
}
|
||||
|
||||
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
return &qbtypes.Statement{Query: sql, Args: args}, nil
|
||||
return b.traceStmtBuilder.Build(ctx, orgID, start, end, requestType, gated, variables)
|
||||
}
|
||||
|
||||
// buildTraceListQuery wires the CTE pipeline (start/end are nanoseconds):
|
||||
|
||||
@@ -32,9 +32,6 @@ type traceQueryStatementBuilder struct {
|
||||
resourceFilterResolver *resourcefilter.ResourceFingerprintResolver[qbtypes.TraceAggregation]
|
||||
aggExprRewriter qbtypes.AggExprRewriter
|
||||
skipResourceFingerprintEnabled bool
|
||||
// traceScope is set only on the per-call copy made by BuildTraceScoped; it
|
||||
// constrains the query to trace ids selected by the __trace_scope CTE.
|
||||
traceScope *qbtypes.Statement
|
||||
}
|
||||
|
||||
var _ qbtypes.StatementBuilder[qbtypes.TraceAggregation] = (*traceQueryStatementBuilder)(nil)
|
||||
@@ -98,38 +95,6 @@ func NewTraceQueryStatementBuilder(
|
||||
}
|
||||
}
|
||||
|
||||
// BuildTraceScoped is Build additionally constrained to spans whose trace_id is
|
||||
// selected by traceScope. The receiver is copied so the shared builder stays stateless.
|
||||
func (b *traceQueryStatementBuilder) BuildTraceScoped(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
start uint64,
|
||||
end uint64,
|
||||
requestType qbtypes.RequestType,
|
||||
query qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation],
|
||||
variables map[string]qbtypes.VariableItem,
|
||||
traceScope *qbtypes.Statement,
|
||||
) (*qbtypes.Statement, error) {
|
||||
// The scope is wired into the list query only; reject other request types rather
|
||||
// than silently dropping the constraint.
|
||||
if requestType != qbtypes.RequestTypeRaw {
|
||||
return nil, errors.NewInternalf(errors.CodeInternal, "trace-scoped build supports only the raw request type, got %s", requestType.StringValue())
|
||||
}
|
||||
scoped := *b
|
||||
scoped.traceScope = traceScope
|
||||
return scoped.Build(ctx, orgID, start, end, requestType, query, variables)
|
||||
}
|
||||
|
||||
// attachTraceScope adds the trace-scope condition to sb and returns the CTE fragment
|
||||
// + args to prepend; both empty when no scope is set.
|
||||
func (b *traceQueryStatementBuilder) attachTraceScope(sb *sqlbuilder.SelectBuilder) (string, []any) {
|
||||
if b.traceScope == nil {
|
||||
return "", nil
|
||||
}
|
||||
sb.Where("trace_id GLOBAL IN (SELECT trace_id FROM __trace_scope)")
|
||||
return fmt.Sprintf("__trace_scope AS (%s)", b.traceScope.Query), b.traceScope.Args
|
||||
}
|
||||
|
||||
// Build builds a SQL query for traces based on the given parameters.
|
||||
func (b *traceQueryStatementBuilder) Build(
|
||||
ctx context.Context,
|
||||
@@ -353,11 +318,6 @@ func (b *traceQueryStatementBuilder) buildListQuery(
|
||||
cteArgs = append(cteArgs, args)
|
||||
}
|
||||
|
||||
if scopeFrag, scopeArgs := b.attachTraceScope(sb); scopeFrag != "" {
|
||||
cteFragments = append(cteFragments, scopeFrag)
|
||||
cteArgs = append(cteArgs, scopeArgs)
|
||||
}
|
||||
|
||||
for i, field := range query.SelectFields {
|
||||
expr, err := b.fm.ColumnExpressionFor(ctx, orgID, start, end, &field, telemetrytypes.FieldDataTypeUnspecified, keys)
|
||||
if err != nil {
|
||||
|
||||
@@ -39,7 +39,8 @@ func (c *conditionBuilder) ConditionFor(
|
||||
}
|
||||
|
||||
// an unknown key simply yields no condition rather than an error.
|
||||
keys, warning := querybuilder.ResolveKeys(key, querybuilder.MatchingFieldKeys(key, fieldKeys))
|
||||
logicalFields, warning := querybuilder.ResolveLogicalFields(key, querybuilder.MatchingLogicalFields(key, fieldKeys))
|
||||
keys := querybuilder.SingleKeys(logicalFields)
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/factory"
|
||||
"github.com/SigNoz/signoz/pkg/flagger"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/semconv"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/audittelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/logstelemetryschema"
|
||||
"github.com/SigNoz/signoz/pkg/telemetryschema/metertelemetryschema"
|
||||
@@ -151,6 +152,14 @@ func (t *telemetryMetaStore) tracesTblStatementToFieldKeys(ctx context.Context)
|
||||
return materialisedKeys, nil
|
||||
}
|
||||
|
||||
func traceSemconvMembers(name string, fieldContext telemetrytypes.FieldContext) []string {
|
||||
return semconv.Members(semconv.KindAttribute, telemetrytypes.FieldKeySelector{
|
||||
Name: name,
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: fieldContext,
|
||||
})
|
||||
}
|
||||
|
||||
// getTracesKeys returns the keys from the spans that match the field selection criteria.
|
||||
func (t *telemetryMetaStore) getTracesKeys(ctx context.Context, fieldKeySelectors []*telemetrytypes.FieldKeySelector) ([]*telemetrytypes.TelemetryFieldKey, bool, error) {
|
||||
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
|
||||
@@ -1320,6 +1329,17 @@ func (t *telemetryMetaStore) GetKeysMulti(ctx context.Context, orgID valuer.UUID
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
// GetKeys backs key suggestions and remains literal. The internal multi-key
|
||||
// lookup expands only trace selectors so query builders see stored family members.
|
||||
expandedTraceSelectors := make([]*telemetrytypes.FieldKeySelector, 0, len(tracesSelectors))
|
||||
for _, selector := range tracesSelectors {
|
||||
for _, member := range traceSemconvMembers(selector.Name, selector.FieldContext) {
|
||||
memberSelector := selector.Copy()
|
||||
memberSelector.Name = member
|
||||
expandedTraceSelectors = append(expandedTraceSelectors, memberSelector)
|
||||
}
|
||||
}
|
||||
tracesSelectors = expandedTraceSelectors
|
||||
tracesKeys, tracesComplete, err := t.getTracesKeys(ctx, tracesSelectors)
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
@@ -1542,7 +1562,16 @@ func (t *telemetryMetaStore) getSpanFieldValues(ctx context.Context, fieldValueS
|
||||
sb := sqlbuilder.Select("DISTINCT string_value, number_value").From(t.tracesDBName + "." + t.tracesFieldsTblName)
|
||||
|
||||
if fieldValueSelector.Name != "" {
|
||||
sb.Where(sb.E("tag_key", fieldValueSelector.Name))
|
||||
members := traceSemconvMembers(fieldValueSelector.Name, fieldValueSelector.FieldContext)
|
||||
if len(members) == 1 {
|
||||
sb.Where(sb.E("tag_key", members[0]))
|
||||
} else {
|
||||
memberValues := make([]any, 0, len(members))
|
||||
for _, member := range members {
|
||||
memberValues = append(memberValues, member)
|
||||
}
|
||||
sb.Where(sb.In("tag_key", memberValues...))
|
||||
}
|
||||
}
|
||||
|
||||
// now look at the field context
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
@@ -83,3 +84,38 @@ func TestGetFirstSeenFromMetricMetadata(t *testing.T) {
|
||||
t.Errorf("there were unfulfilled expectations: %s", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetAllValuesReturnsValuesFromEveryTraceSemconvFamilyMember(t *testing.T) {
|
||||
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, ®exMatcher{})
|
||||
mock := mockTelemetryStore.Mock()
|
||||
|
||||
metadata := NewTelemetryMetaStore(
|
||||
instrumentationtest.New().ToProviderSettings(),
|
||||
mockTelemetryStore,
|
||||
flaggertest.New(t),
|
||||
)
|
||||
|
||||
mock.ExpectQuery(`SELECT DISTINCT string_value, number_value FROM signoz_traces\.distributed_tag_attributes_v2 WHERE tag_key IN \(\?, \?\) AND tag_type = \? AND tag_data_type = \? LIMIT \?`).
|
||||
WithArgs("deployment.environment.name", "deployment.environment", "resource", "string", 51).
|
||||
WillReturnRows(cmock.NewRows([]cmock.ColumnType{
|
||||
{Name: "string_value", Type: "String"},
|
||||
{Name: "number_value", Type: "Float64"},
|
||||
}, [][]any{
|
||||
{"production", float64(0)},
|
||||
{"staging", float64(0)},
|
||||
{"production", float64(0)},
|
||||
}))
|
||||
|
||||
values, complete, err := metadata.GetAllValues(context.Background(), valuer.UUID{}, &telemetrytypes.FieldValueSelector{
|
||||
FieldKeySelector: &telemetrytypes.FieldKeySelector{
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
Name: "deployment.environment",
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.True(t, complete)
|
||||
assert.Equal(t, []string{"production", "staging"}, values.StringValues)
|
||||
assert.NoError(t, mock.ExpectationsWereMet(), "all expected metadata queries should be executed")
|
||||
}
|
||||
|
||||
@@ -139,7 +139,8 @@ func (c *conditionBuilder) ConditionFor(
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
keys, warning := querybuilder.ResolveKeys(key, querybuilder.MatchingFieldKeys(key, fieldKeys))
|
||||
logicalFields, warning := querybuilder.ResolveLogicalFields(key, querybuilder.MatchingLogicalFields(key, fieldKeys))
|
||||
keys := querybuilder.SingleKeys(logicalFields)
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
|
||||
@@ -452,7 +452,7 @@ func (c *conditionBuilder) ConditionFor(
|
||||
value any,
|
||||
sb *sqlbuilder.SelectBuilder,
|
||||
) ([]string, []string, error) {
|
||||
matches := querybuilder.MatchingFieldKeys(key, fieldKeys)
|
||||
matches := querybuilder.MatchingLogicalFields(key, fieldKeys)
|
||||
skipResourceFilter := options.SkipResourceFilter
|
||||
|
||||
// search() resolves its own (optional) scope; handle it before key resolution.
|
||||
@@ -460,7 +460,8 @@ func (c *conditionBuilder) ConditionFor(
|
||||
return c.conditionForSearch(ctx, orgID, key, value, sb)
|
||||
}
|
||||
|
||||
keys, warning := querybuilder.ResolveKeys(key, matches)
|
||||
logicalFields, warning := querybuilder.ResolveLogicalFields(key, matches)
|
||||
keys := querybuilder.SingleKeys(logicalFields)
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
|
||||
@@ -162,7 +162,7 @@ func (c *conditionBuilder) ConditionFor(
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
keys := querybuilder.MatchingFieldKeys(key, fieldKeys)
|
||||
keys := querybuilder.SingleKeys(querybuilder.MatchingLogicalFields(key, fieldKeys))
|
||||
var warnings []string
|
||||
if len(keys) == 0 {
|
||||
if _, isColumn := timeSeriesV4Columns[key.Name]; isColumn {
|
||||
|
||||
@@ -18,12 +18,14 @@ import (
|
||||
)
|
||||
|
||||
type conditionBuilder struct {
|
||||
fm qbtypes.FieldMapper
|
||||
// The builder composes family expressions, so it needs this package's
|
||||
// mapper, not the narrower qbtypes.FieldMapper.
|
||||
fm *fieldMapper
|
||||
}
|
||||
|
||||
var _ qbtypes.ConditionBuilder = (*conditionBuilder)(nil)
|
||||
|
||||
func NewConditionBuilder(fm qbtypes.FieldMapper) *conditionBuilder {
|
||||
func NewConditionBuilder(fm *fieldMapper) *conditionBuilder {
|
||||
return &conditionBuilder{fm: fm}
|
||||
}
|
||||
|
||||
@@ -32,7 +34,7 @@ func (c *conditionBuilder) conditionFor(
|
||||
orgID valuer.UUID,
|
||||
startNs uint64,
|
||||
endNs uint64,
|
||||
key *telemetrytypes.TelemetryFieldKey,
|
||||
logical *telemetrytypes.LogicalField,
|
||||
operator qbtypes.FilterOperator,
|
||||
value any,
|
||||
sb *sqlbuilder.SelectBuilder,
|
||||
@@ -42,13 +44,13 @@ func (c *conditionBuilder) conditionFor(
|
||||
value = querybuilder.FormatValueForContains(value)
|
||||
}
|
||||
|
||||
fieldExpression, err := c.fm.FieldFor(ctx, orgID, startNs, endNs, key)
|
||||
fieldExpression, err := c.fm.FieldForLogical(ctx, orgID, startNs, endNs, logical)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// TODO(srikanthccv): maybe extend this to every possible attribute
|
||||
if key.Name == "duration_nano" || key.Name == "durationNano" { // QoL improvement
|
||||
if logical.Name == "duration_nano" || logical.Name == "durationNano" { // QoL improvement
|
||||
switch v := value.(type) {
|
||||
case string:
|
||||
if duration, err := time.ParseDuration(v); err == nil {
|
||||
@@ -65,7 +67,7 @@ func (c *conditionBuilder) conditionFor(
|
||||
}
|
||||
}
|
||||
|
||||
fieldExpression, value = querybuilder.DataTypeCollisionHandledFieldName(key, value, fieldExpression, operator)
|
||||
fieldExpression, value = querybuilder.DataTypeCollisionHandledFieldName(logical.Single(), value, fieldExpression, operator)
|
||||
|
||||
// regular operators
|
||||
switch operator {
|
||||
@@ -154,11 +156,7 @@ func (c *conditionBuilder) conditionFor(
|
||||
// in the query builder, `exists` and `not exists` are used for
|
||||
// key membership checks, so depending on the column type, the condition changes
|
||||
case qbtypes.FilterOperatorExists, qbtypes.FilterOperatorNotExists:
|
||||
columns, err := c.fm.ColumnFor(ctx, orgID, startNs, endNs, key)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
pred, err := querybuilder.ExistsExpression(columns, key, startNs, endNs, fieldExpression, operator == qbtypes.FilterOperatorExists)
|
||||
pred, err := c.fm.ExistsForLogical(ctx, orgID, startNs, endNs, logical, operator == qbtypes.FilterOperatorExists)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
@@ -210,10 +208,10 @@ func (c *conditionBuilder) ConditionFor(
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
matches := querybuilder.MatchingFieldKeys(key, fieldKeys)
|
||||
matches := querybuilder.MatchingLogicalFields(key, fieldKeys)
|
||||
skipResourceFilter := options.SkipResourceFilter
|
||||
|
||||
keys, warning := querybuilder.ResolveKeys(key, matches)
|
||||
logicalFields, warning := querybuilder.ResolveLogicalFields(key, matches)
|
||||
var warnings []string
|
||||
if warning != "" {
|
||||
warnings = append(warnings, warning)
|
||||
@@ -221,10 +219,10 @@ func (c *conditionBuilder) ConditionFor(
|
||||
// A bare key that names a real column filters on the column too — first. When metadata
|
||||
// only knows the name under other contexts, prepend the column and keep metadata matches
|
||||
// only where their type is consistent with it (a corrupt entry can't degrade the column).
|
||||
if key.FieldContext == telemetrytypes.FieldContextUnspecified && len(keys) > 0 {
|
||||
if key.FieldContext == telemetrytypes.FieldContextUnspecified && len(logicalFields) > 0 {
|
||||
hasColumn := false
|
||||
for _, k := range keys {
|
||||
if k.FieldContext == telemetrytypes.FieldContextSpan {
|
||||
for _, logical := range logicalFields {
|
||||
if logical.FieldContext == telemetrytypes.FieldContextSpan {
|
||||
hasColumn = true
|
||||
break
|
||||
}
|
||||
@@ -232,49 +230,49 @@ func (c *conditionBuilder) ConditionFor(
|
||||
if !hasColumn {
|
||||
probe := telemetrytypes.NewTelemetryFieldKey(key.Name, telemetrytypes.FieldContextSpan, key.FieldDataType)
|
||||
if cols, colErr := c.fm.ColumnFor(ctx, orgID, startNs, endNs, probe); colErr == nil && len(cols) > 0 {
|
||||
combined := make([]*telemetrytypes.TelemetryFieldKey, 0, len(keys)+1)
|
||||
combined = append(combined, probe)
|
||||
for _, k := range keys {
|
||||
if columnMatchesDataType(cols[0], k.FieldDataType) {
|
||||
combined = append(combined, k)
|
||||
combined := make([]*telemetrytypes.LogicalField, 0, len(logicalFields)+1)
|
||||
combined = append(combined, telemetrytypes.SingleLogicalField(key.Name, probe))
|
||||
for _, logical := range logicalFields {
|
||||
if columnMatchesDataType(cols[0], logical.FieldDataType) {
|
||||
combined = append(combined, logical)
|
||||
}
|
||||
}
|
||||
keys = combined
|
||||
logicalFields = combined
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
synthesized := false
|
||||
if len(keys) == 0 {
|
||||
if len(logicalFields) == 0 {
|
||||
// Not in metadata. CandidateKeys resolves it: fold contexts (span/trace) get the
|
||||
// metadata map so it can honor a real column, correct to a stripped-name metadata
|
||||
// match, or synthesize; strict contexts pass nil and keep their synthesize path.
|
||||
keys = c.fm.CandidateKeys(ctx, orgID, key, value, candidateLookupKeys(key, fieldKeys))
|
||||
if len(keys) == 0 {
|
||||
logicalFields = querybuilder.WrapAsLogicalFields(key.Name, c.fm.CandidateKeys(ctx, orgID, key, value, candidateLookupKeys(key, fieldKeys)))
|
||||
if len(logicalFields) == 0 {
|
||||
return nil, warnings, querybuilder.NewKeyNotFoundError(key.Name)
|
||||
}
|
||||
synthesized = true
|
||||
warnings = append(warnings, querybuilder.NewKeyNotFoundWarning(key.Name))
|
||||
}
|
||||
|
||||
// When a resource sub-query already covers the term, drop resource keys from the main
|
||||
// When a resource sub-query already covers the term, drop resource fields from the main
|
||||
// query. Synthesized keys are exempt: the sub-query skips keys absent from metadata.
|
||||
if skipResourceFilter && !synthesized {
|
||||
filtered := make([]*telemetrytypes.TelemetryFieldKey, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
if k.FieldContext != telemetrytypes.FieldContextResource {
|
||||
filtered = append(filtered, k)
|
||||
filtered := make([]*telemetrytypes.LogicalField, 0, len(logicalFields))
|
||||
for _, logical := range logicalFields {
|
||||
if logical.FieldContext != telemetrytypes.FieldContextResource {
|
||||
filtered = append(filtered, logical)
|
||||
}
|
||||
}
|
||||
if len(filtered) == 0 {
|
||||
return nil, warnings, nil
|
||||
}
|
||||
keys = filtered
|
||||
logicalFields = filtered
|
||||
}
|
||||
|
||||
conds := make([]string, 0, len(keys))
|
||||
for _, k := range keys {
|
||||
cond, err := c.conditionForKey(ctx, orgID, startNs, endNs, k, operator, value, sb)
|
||||
conds := make([]string, 0, len(logicalFields))
|
||||
for _, logical := range logicalFields {
|
||||
cond, err := c.conditionForLogicalField(ctx, orgID, startNs, endNs, logical, operator, value, sb)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
@@ -283,28 +281,28 @@ func (c *conditionBuilder) ConditionFor(
|
||||
return conds, warnings, nil
|
||||
}
|
||||
|
||||
func (c *conditionBuilder) conditionForKey(
|
||||
func (c *conditionBuilder) conditionForLogicalField(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
startNs uint64,
|
||||
endNs uint64,
|
||||
key *telemetrytypes.TelemetryFieldKey,
|
||||
logical *telemetrytypes.LogicalField,
|
||||
operator qbtypes.FilterOperator,
|
||||
value any,
|
||||
sb *sqlbuilder.SelectBuilder,
|
||||
) (string, error) {
|
||||
if c.isSpanScopeField(key.Name) {
|
||||
return c.buildSpanScopeCondition(key, operator, value, startNs)
|
||||
if c.isSpanScopeField(logical.Name) {
|
||||
return c.buildSpanScopeCondition(logical.Single(), operator, value, startNs)
|
||||
}
|
||||
|
||||
condition, err := c.conditionFor(ctx, orgID, startNs, endNs, key, operator, value, sb)
|
||||
condition, err := c.conditionFor(ctx, orgID, startNs, endNs, logical, operator, value, sb)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
if operator.AddDefaultExistsFilter() {
|
||||
// skip adding exists filter for intrinsic fields
|
||||
field, _ := c.fm.FieldFor(ctx, orgID, startNs, endNs, key)
|
||||
field, _ := c.fm.FieldFor(ctx, orgID, startNs, endNs, logical.Single())
|
||||
if slices.Contains(maps.Keys(IntrinsicFields), field) ||
|
||||
slices.Contains(maps.Keys(IntrinsicFieldsDeprecated), field) ||
|
||||
slices.Contains(maps.Keys(CalculatedFields), field) ||
|
||||
@@ -312,7 +310,7 @@ func (c *conditionBuilder) conditionForKey(
|
||||
return condition, nil
|
||||
}
|
||||
|
||||
existsCondition, err := c.conditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorExists, nil, sb)
|
||||
existsCondition, err := c.conditionFor(ctx, orgID, startNs, endNs, logical, qbtypes.FilterOperatorExists, nil, sb)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
@@ -308,6 +308,93 @@ func TestConditionFor(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The family tests drive resolution through the metadata map, exactly as
|
||||
// production does: plain member keys in, the resolver groups them, and the
|
||||
// builder composes one condition per logical field.
|
||||
func traceFamilyConditionSQL(t *testing.T, requestedName string, members []*telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) (string, []any, []string) {
|
||||
t.Helper()
|
||||
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{}
|
||||
for _, member := range members {
|
||||
fieldKeys[member.Name] = []*telemetrytypes.TelemetryFieldKey{member}
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(
|
||||
requestedName,
|
||||
telemetrytypes.FieldContextAttribute,
|
||||
telemetrytypes.FieldDataTypeString,
|
||||
)
|
||||
sb := sqlbuilder.NewSelectBuilder()
|
||||
conditions, warnings, err := NewConditionBuilder(NewFieldMapper()).ConditionFor(
|
||||
context.Background(), valuer.UUID{}, 0, 0, requested,
|
||||
fieldKeys,
|
||||
qbtypes.ConditionBuilderOptions{}, op, value, sb,
|
||||
)
|
||||
require.NoError(t, err)
|
||||
sb.Where(conditions...)
|
||||
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
return sql, args, warnings
|
||||
}
|
||||
|
||||
func traceAttrMember(name string, materialized bool) *telemetrytypes.TelemetryFieldKey {
|
||||
return &telemetrytypes.TelemetryFieldKey{
|
||||
Name: name,
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
Materialized: materialized,
|
||||
}
|
||||
}
|
||||
|
||||
func TestConditionForSemconvFamilyPositiveFilterChecksPresence(t *testing.T) {
|
||||
sql, args, warnings := traceFamilyConditionSQL(t,
|
||||
"deployment.environment.name",
|
||||
[]*telemetrytypes.TelemetryFieldKey{
|
||||
traceAttrMember("deployment.environment.name", false),
|
||||
traceAttrMember("deployment.environment", false),
|
||||
},
|
||||
qbtypes.FilterOperatorEqual, "production",
|
||||
)
|
||||
|
||||
assert.Empty(t, warnings, "a family is one logical field, never an ambiguity warning")
|
||||
assert.Contains(t, sql, "COALESCE(NULLIF(attributes_string['deployment.environment.name'], ''), NULLIF(attributes_string['deployment.environment'], ''), '') = ? AND (mapContains(attributes_string, 'deployment.environment.name') OR mapContains(attributes_string, 'deployment.environment'))")
|
||||
assert.Equal(t, []any{"production"}, args)
|
||||
}
|
||||
|
||||
func TestNewConditionBuilderTakesThisPackagesMapper(t *testing.T) {
|
||||
// The builder composes family expressions, so it is deliberately coupled
|
||||
// to this package's mapper rather than the narrower qbtypes.FieldMapper.
|
||||
require.NotNil(t, NewConditionBuilder(NewFieldMapper()))
|
||||
}
|
||||
|
||||
func TestConditionForSemconvFamilyPreservesMaterializedMemberExistsColumn(t *testing.T) {
|
||||
sql, args, warnings := traceFamilyConditionSQL(t,
|
||||
"deployment.environment.name",
|
||||
[]*telemetrytypes.TelemetryFieldKey{
|
||||
traceAttrMember("deployment.environment.name", false),
|
||||
traceAttrMember("deployment.environment", true),
|
||||
},
|
||||
qbtypes.FilterOperatorEqual, "production",
|
||||
)
|
||||
|
||||
assert.Empty(t, warnings)
|
||||
assert.Contains(t, sql, "`attribute_string_deployment$$environment_exists`")
|
||||
assert.NotContains(t, sql, "`attribute_string_deployment$environment_exists`")
|
||||
assert.Equal(t, []any{"production"}, args)
|
||||
}
|
||||
|
||||
func TestConditionForSemconvFamilyNotExistsChecksEveryMember(t *testing.T) {
|
||||
sql, _, warnings := traceFamilyConditionSQL(t,
|
||||
"deployment.environment",
|
||||
[]*telemetrytypes.TelemetryFieldKey{
|
||||
traceAttrMember("deployment.environment.name", false),
|
||||
traceAttrMember("deployment.environment", false),
|
||||
},
|
||||
qbtypes.FilterOperatorNotExists, nil,
|
||||
)
|
||||
|
||||
assert.Empty(t, warnings)
|
||||
assert.Contains(t, sql, "NOT (mapContains(attributes_string, 'deployment.environment.name') OR mapContains(attributes_string, 'deployment.environment'))")
|
||||
}
|
||||
|
||||
func TestConditionForResourceWithEvolution(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
releaseTime := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||
|
||||
@@ -167,6 +167,145 @@ func NewFieldMapper() *fieldMapper {
|
||||
return &fieldMapper{}
|
||||
}
|
||||
|
||||
// FieldForLogical returns the value expression for a resolved logical field:
|
||||
// the member's own expression for a single-member field, and a current-first
|
||||
// merge across the members' expressions for a family. Each member expression
|
||||
// comes from FieldFor and therefore honors that member's own materialization
|
||||
// and evolution state — a family never needs sibling information on a key.
|
||||
func (m *fieldMapper) FieldForLogical(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
tsStart, tsEnd uint64,
|
||||
logical *telemetrytypes.LogicalField,
|
||||
) (string, error) {
|
||||
if !logical.IsFamily() {
|
||||
return m.FieldFor(ctx, orgID, tsStart, tsEnd, logical.Single())
|
||||
}
|
||||
|
||||
memberExprs := make([]string, 0, len(logical.Members))
|
||||
for _, member := range logical.Members {
|
||||
expr, err := m.FieldFor(ctx, orgID, tsStart, tsEnd, member)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
memberExprs = append(memberExprs, expr)
|
||||
}
|
||||
|
||||
if logical.FieldDataType == telemetrytypes.FieldDataTypeString {
|
||||
// The trailing '' keeps single-key semantics for rows without any
|
||||
// member: string maps read '' for an absent key, and negative
|
||||
// operators must keep including such rows (see AddDefaultExistsFilter).
|
||||
values := make([]string, 0, len(memberExprs))
|
||||
for _, expr := range memberExprs {
|
||||
values = append(values, fmt.Sprintf("NULLIF(%s, '')", expr))
|
||||
}
|
||||
return "COALESCE(" + strings.Join(values, ", ") + ", '')", nil
|
||||
}
|
||||
|
||||
// Numeric and boolean maps return zero for an absent key. If a family of
|
||||
// either type is enabled, this tail must become zero too.
|
||||
branches := make([]string, 0, len(logical.Members)*2)
|
||||
for i, member := range logical.Members {
|
||||
guard, err := m.existsExpressionFor(ctx, orgID, tsStart, tsEnd, member, true)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
branches = append(branches, guard, memberExprs[i])
|
||||
}
|
||||
return "multiIf(" + strings.Join(branches, ", ") + ", NULL)", nil
|
||||
}
|
||||
|
||||
// ExistsForLogical renders the existence predicate for a resolved logical
|
||||
// field: a member's own predicate for a single-member field, presence of any
|
||||
// member for a family.
|
||||
func (m *fieldMapper) ExistsForLogical(
|
||||
ctx context.Context,
|
||||
orgID valuer.UUID,
|
||||
tsStart, tsEnd uint64,
|
||||
logical *telemetrytypes.LogicalField,
|
||||
exists bool,
|
||||
) (string, error) {
|
||||
if !logical.IsFamily() {
|
||||
return m.existsExpressionFor(ctx, orgID, tsStart, tsEnd, logical.Single(), exists)
|
||||
}
|
||||
|
||||
guards := make([]string, 0, len(logical.Members))
|
||||
for _, member := range logical.Members {
|
||||
guard, err := m.existsExpressionFor(ctx, orgID, tsStart, tsEnd, member, true)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
guards = append(guards, guard)
|
||||
}
|
||||
combined := "(" + strings.Join(guards, " OR ") + ")"
|
||||
if exists {
|
||||
return combined, nil
|
||||
}
|
||||
return "NOT " + combined, nil
|
||||
}
|
||||
|
||||
// logicalForResolvedColumn upgrades a directly-resolvable key (the FieldFor
|
||||
// probe succeeded) to its family when the metadata map proves membership;
|
||||
// otherwise the key stays a single-member logical field.
|
||||
func logicalForResolvedColumn(field *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) *telemetrytypes.LogicalField {
|
||||
for _, logical := range querybuilder.MatchingLogicalFields(field, keys) {
|
||||
if logical.IsFamily() &&
|
||||
logical.FieldContext == field.FieldContext &&
|
||||
(field.FieldDataType == telemetrytypes.FieldDataTypeUnspecified || logical.FieldDataType == field.FieldDataType) {
|
||||
return logical
|
||||
}
|
||||
}
|
||||
return telemetrytypes.SingleLogicalField(field.Name, field)
|
||||
}
|
||||
|
||||
// upgradeToFamilies swaps single-member candidates for their family when the
|
||||
// metadata map proves membership. Candidate order and every non-family
|
||||
// candidate stay exactly as the legacy flow produced them; sibling candidates
|
||||
// of an already-emitted family are dropped rather than duplicated.
|
||||
func upgradeToFamilies(field *telemetrytypes.TelemetryFieldKey, candidates []*telemetrytypes.LogicalField, keys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.LogicalField {
|
||||
var families []*telemetrytypes.LogicalField
|
||||
for _, logical := range querybuilder.MatchingLogicalFields(field, keys) {
|
||||
if logical.IsFamily() {
|
||||
families = append(families, logical)
|
||||
}
|
||||
}
|
||||
if len(families) == 0 {
|
||||
return candidates
|
||||
}
|
||||
|
||||
out := make([]*telemetrytypes.LogicalField, 0, len(candidates))
|
||||
emitted := make(map[*telemetrytypes.LogicalField]bool)
|
||||
for _, candidate := range candidates {
|
||||
var family *telemetrytypes.LogicalField
|
||||
for _, fam := range families {
|
||||
if fam.FieldContext != candidate.FieldContext || fam.FieldDataType != candidate.FieldDataType {
|
||||
continue
|
||||
}
|
||||
memberOfFamily := candidate.Single().Name == field.Name
|
||||
for _, member := range fam.Members {
|
||||
if member.Name == candidate.Single().Name {
|
||||
memberOfFamily = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if memberOfFamily {
|
||||
family = fam
|
||||
break
|
||||
}
|
||||
}
|
||||
if family == nil {
|
||||
out = append(out, candidate)
|
||||
continue
|
||||
}
|
||||
if emitted[family] {
|
||||
continue
|
||||
}
|
||||
emitted[family] = true
|
||||
out = append(out, family)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (m *fieldMapper) getColumn(
|
||||
_ context.Context,
|
||||
_, _ uint64,
|
||||
@@ -292,9 +431,9 @@ func (m *fieldMapper) resolveColumnExprs(
|
||||
return nil, nil, nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource context fields are supported for json columns, got %s", key.FieldContext.String)
|
||||
}
|
||||
// have to add ::string as clickHouse throws an error :- data types Variant/Dynamic are not allowed in GROUP BY
|
||||
// once clickHouse dependency is updated, we need to check if we can remove it.
|
||||
exprs = append(exprs, fmt.Sprintf("%s.`%s`::String", columnName, key.Name))
|
||||
existExprs = append(existExprs, fmt.Sprintf("%s.`%s` IS NOT NULL", columnName, key.Name))
|
||||
// once ClickHouse is updated, check whether this cast can be removed.
|
||||
exprs = append(exprs, fmt.Sprintf("%s.%s::String", columnName, querybuilder.ClickHouseIdentifier(key.Name)))
|
||||
existExprs = append(existExprs, fmt.Sprintf("%s.%s IS NOT NULL", columnName, querybuilder.ClickHouseIdentifier(key.Name)))
|
||||
case schema.ColumnTypeEnumString,
|
||||
schema.ColumnTypeEnumUInt64,
|
||||
schema.ColumnTypeEnumUInt32,
|
||||
@@ -319,13 +458,13 @@ func (m *fieldMapper) resolveColumnExprs(
|
||||
|
||||
switch valueType := column.Type.(schema.MapColumnType).ValueType; valueType.GetType() {
|
||||
case schema.ColumnTypeEnumString, schema.ColumnTypeEnumFloat64, schema.ColumnTypeEnumBool:
|
||||
// a key could have been materialized, if so return the materialized column name
|
||||
if key.Materialized {
|
||||
// a key could have been materialized, if so return the materialized column name
|
||||
exprs = append(exprs, telemetrytypes.FieldKeyToMaterializedColumnName(key))
|
||||
existExprs = append(existExprs, telemetrytypes.FieldKeyToMaterializedColumnNameForExists(key))
|
||||
} else {
|
||||
exprs = append(exprs, fmt.Sprintf("%s['%s']", columnName, key.Name))
|
||||
existExprs = append(existExprs, fmt.Sprintf("mapContains(%s, '%s')", columnName, key.Name))
|
||||
exprs = append(exprs, fmt.Sprintf("%s[%s]", columnName, querybuilder.ClickHouseStringLiteral(key.Name)))
|
||||
existExprs = append(existExprs, fmt.Sprintf("mapContains(%s, %s)", columnName, querybuilder.ClickHouseStringLiteral(key.Name)))
|
||||
}
|
||||
default:
|
||||
return nil, nil, nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "value type %s is not supported for map column type %s", valueType, column.Type)
|
||||
@@ -348,18 +487,23 @@ func (m *fieldMapper) ColumnExpressionFor(
|
||||
keys map[string][]*telemetrytypes.TelemetryFieldKey,
|
||||
) (string, error) {
|
||||
|
||||
// Resolve the candidate column(s).
|
||||
var candidates []*telemetrytypes.TelemetryFieldKey
|
||||
// Resolve the candidate logical field(s).
|
||||
var candidates []*telemetrytypes.LogicalField
|
||||
switch _, err := m.FieldFor(ctx, orgID, startNs, endNs, field); {
|
||||
case err == nil:
|
||||
candidates = []*telemetrytypes.TelemetryFieldKey{field}
|
||||
// A directly-resolvable key upgrades to its family when the metadata
|
||||
// map proves membership; otherwise it stays single-member.
|
||||
candidates = []*telemetrytypes.LogicalField{logicalForResolvedColumn(field, keys)}
|
||||
case errors.Is(err, qbtypes.ErrColumnNotFound):
|
||||
// column (when the bare name is one) plus metadata matches, else synthesized
|
||||
// type-variant keys.
|
||||
candidates = m.CandidateKeys(ctx, orgID, field, nil, keys)
|
||||
if len(candidates) == 0 {
|
||||
// The legacy candidate flow, unchanged: column (when the bare name is
|
||||
// one) plus metadata matches, else synthesized type-variant keys. The
|
||||
// family step below only swaps candidates for their family; it never
|
||||
// changes candidate order or non-family behavior.
|
||||
raw := m.CandidateKeys(ctx, orgID, field, nil, keys)
|
||||
if len(raw) == 0 {
|
||||
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys))...)
|
||||
}
|
||||
candidates = upgradeToFamilies(field, querybuilder.WrapAsLogicalFields(field.Name, raw), keys)
|
||||
default:
|
||||
return "", err
|
||||
}
|
||||
@@ -373,21 +517,21 @@ func (m *fieldMapper) ColumnExpressionFor(
|
||||
dummyValue = 0.0
|
||||
}
|
||||
stmts := make([]string, 0, len(candidates)*2)
|
||||
for _, key := range candidates {
|
||||
value, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
|
||||
for _, logical := range candidates {
|
||||
value, err := m.FieldForLogical(ctx, orgID, startNs, endNs, logical)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
guard, err := m.existsExpressionFor(ctx, orgID, startNs, endNs, key, true)
|
||||
guard, err := m.ExistsForLogical(ctx, orgID, startNs, endNs, logical, true)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
coerced := value
|
||||
// a time column keeps its native type; coercing it would yield seconds
|
||||
if temporal, err := m.columnIsTemporal(ctx, startNs, endNs, key); err != nil {
|
||||
if temporal, err := m.logicalIsTemporal(ctx, startNs, endNs, logical); err != nil {
|
||||
return "", err
|
||||
} else if !temporal {
|
||||
coerced, _ = querybuilder.DataTypeCollisionHandledFieldName(key, dummyValue, value, qbtypes.FilterOperatorUnknown)
|
||||
coerced, _ = querybuilder.DataTypeCollisionHandledFieldName(logical.Single(), dummyValue, value, qbtypes.FilterOperatorUnknown)
|
||||
}
|
||||
stmts = append(stmts, guard, coerced)
|
||||
}
|
||||
@@ -395,13 +539,14 @@ func (m *fieldMapper) ColumnExpressionFor(
|
||||
}
|
||||
|
||||
if len(candidates) == 1 {
|
||||
value, err := m.FieldFor(ctx, orgID, startNs, endNs, candidates[0])
|
||||
logical := candidates[0]
|
||||
value, err := m.FieldForLogical(ctx, orgID, startNs, endNs, logical)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
exprs, existExprs, _, _ := m.resolveColumnExprs(ctx, startNs, endNs, candidates[0])
|
||||
if len(exprs) == 1 && len(existExprs) == 1 {
|
||||
guard, err := m.existsExpressionFor(ctx, orgID, startNs, endNs, candidates[0], true)
|
||||
exprs, existExprs, _, _ := m.resolveColumnExprs(ctx, startNs, endNs, logical.Single())
|
||||
if !logical.IsFamily() && len(exprs) == 1 && len(existExprs) == 1 {
|
||||
guard, err := m.ExistsForLogical(ctx, orgID, startNs, endNs, logical, true)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
@@ -413,12 +558,12 @@ func (m *fieldMapper) ColumnExpressionFor(
|
||||
// Multiple candidates (collision / synth): multiIf picks the first that exists,
|
||||
// stringified so branches share a type.
|
||||
args := make([]string, 0, len(candidates))
|
||||
for _, key := range candidates {
|
||||
value, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
|
||||
for _, logical := range candidates {
|
||||
value, err := m.FieldForLogical(ctx, orgID, startNs, endNs, logical)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
guard, err := m.existsExpressionFor(ctx, orgID, startNs, endNs, key, true)
|
||||
guard, err := m.ExistsForLogical(ctx, orgID, startNs, endNs, logical, true)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
@@ -427,6 +572,15 @@ func (m *fieldMapper) ColumnExpressionFor(
|
||||
return fmt.Sprintf("multiIf(%s, NULL)", strings.Join(args, ", ")), nil
|
||||
}
|
||||
|
||||
// logicalIsTemporal reports whether the logical field resolves to a single time
|
||||
// column. A family is attribute-backed and never temporal.
|
||||
func (m *fieldMapper) logicalIsTemporal(ctx context.Context, startNs, endNs uint64, logical *telemetrytypes.LogicalField) (bool, error) {
|
||||
if logical.IsFamily() {
|
||||
return false, nil
|
||||
}
|
||||
return m.columnIsTemporal(ctx, startNs, endNs, logical.Single())
|
||||
}
|
||||
|
||||
// columnIsTemporal reports whether key resolves to a single time column, after evolution
|
||||
// selection. Multiple columns mean an attribute-map union, which is never temporal.
|
||||
func (m *fieldMapper) columnIsTemporal(ctx context.Context, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (bool, error) {
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -12,6 +13,20 @@ import (
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestFieldForQuotesRequestKeyNames(t *testing.T) {
|
||||
key := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "name'`\\); SELECT 1 --",
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
|
||||
expression, err := NewFieldMapper().FieldFor(context.Background(), valuer.UUID{}, 0, 0, key)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Contains(t, expression, "resource."+querybuilder.ClickHouseIdentifier(key.Name))
|
||||
assert.Contains(t, expression, "mapContains(resources_string, "+querybuilder.ClickHouseStringLiteral(key.Name)+")")
|
||||
}
|
||||
|
||||
func TestGetFieldKeyName(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
@@ -120,6 +135,118 @@ func TestGetFieldKeyName(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// FieldFor is a per-physical-key primitive: it never consults the family
|
||||
// table. Family composition is FieldForLogical's job.
|
||||
func TestFieldForIsPerKey(t *testing.T) {
|
||||
key := telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
|
||||
expression, err := NewFieldMapper().FieldFor(context.Background(), valuer.UUID{}, 0, 0, &key)
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "attributes_string['deployment.environment.name']", expression)
|
||||
}
|
||||
|
||||
func traceFamilyLogicalField(t *testing.T, requestedName string, members ...*telemetrytypes.TelemetryFieldKey) *telemetrytypes.LogicalField {
|
||||
t.Helper()
|
||||
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{}
|
||||
for _, member := range members {
|
||||
fieldKeys[member.Name] = []*telemetrytypes.TelemetryFieldKey{member}
|
||||
}
|
||||
requested := telemetrytypes.NewTelemetryFieldKey(requestedName, members[0].FieldContext, members[0].FieldDataType)
|
||||
matches := querybuilder.MatchingLogicalFields(requested, fieldKeys)
|
||||
require.Len(t, matches, 1)
|
||||
return matches[0]
|
||||
}
|
||||
|
||||
func TestFieldForLogicalMergesFamilyMembersCurrentFirst(t *testing.T) {
|
||||
current := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
old := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
|
||||
for _, requestedName := range []string{current.Name, old.Name} {
|
||||
logical := traceFamilyLogicalField(t, requestedName, current, old)
|
||||
expression, err := NewFieldMapper().FieldForLogical(context.Background(), valuer.UUID{}, 0, 0, logical)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
"COALESCE(NULLIF(attributes_string['deployment.environment.name'], ''), NULLIF(attributes_string['deployment.environment'], ''), '')",
|
||||
expression,
|
||||
"both request spellings address one logical field",
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// The old-name request with only the current spelling in metadata prunes to a
|
||||
// single member: plain single-key SQL, no coalesce.
|
||||
func TestFieldForLogicalPrunesToPresentMembers(t *testing.T) {
|
||||
current := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextAttribute,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
|
||||
logical := traceFamilyLogicalField(t, "deployment.environment", current)
|
||||
expression, err := NewFieldMapper().FieldForLogical(context.Background(), valuer.UUID{}, 0, 0, logical)
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "attributes_string['deployment.environment.name']", expression)
|
||||
}
|
||||
|
||||
// Every member brings its own storage state: the merge composes each member's
|
||||
// FieldFor output, so a materialized-with-evolutions member keeps its column
|
||||
// history inside the family expression with no sibling bookkeeping anywhere.
|
||||
func TestFieldForLogicalComposesResourceMembersFromTheirOwnStorage(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
fm := NewFieldMapper()
|
||||
start := uint64(time.Date(2024, 6, 1, 0, 0, 0, 0, time.UTC).UnixNano())
|
||||
end := uint64(time.Date(2024, 6, 5, 0, 0, 0, 0, time.UTC).UnixNano())
|
||||
|
||||
current := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment.name",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
}
|
||||
old := &telemetrytypes.TelemetryFieldKey{
|
||||
Name: "deployment.environment",
|
||||
Signal: telemetrytypes.SignalTraces,
|
||||
FieldContext: telemetrytypes.FieldContextResource,
|
||||
FieldDataType: telemetrytypes.FieldDataTypeString,
|
||||
Materialized: true,
|
||||
Evolutions: MockEvolutionData(time.Date(2024, 6, 2, 0, 0, 0, 0, time.UTC)),
|
||||
}
|
||||
|
||||
currentExpr, err := fm.FieldFor(ctx, valuer.UUID{}, start, end, current)
|
||||
require.NoError(t, err)
|
||||
oldExpr, err := fm.FieldFor(ctx, valuer.UUID{}, start, end, old)
|
||||
require.NoError(t, err)
|
||||
|
||||
logical := traceFamilyLogicalField(t, current.Name, current, old)
|
||||
expression, err := fm.FieldForLogical(ctx, valuer.UUID{}, start, end, logical)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t,
|
||||
"COALESCE(NULLIF("+currentExpr+", ''), NULLIF("+oldExpr+", ''), '')",
|
||||
expression,
|
||||
"the family merge is exactly the members' own expressions, current-first, with the keyless tail",
|
||||
)
|
||||
assert.Contains(t, expression, "`resource_string_deployment$$environment`",
|
||||
"the promoted member keeps its materialized column")
|
||||
}
|
||||
|
||||
func TestFieldForResourceWithEvolution(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
releaseTime := time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||
|
||||
@@ -65,7 +65,7 @@ var (
|
||||
ResourceMetaResourceTTLSetting = NewResourceMetaResource(KindTTLSetting)
|
||||
ResourceMetaResourceRule = NewResourceMetaResource(KindRule)
|
||||
ResourceMetaResourcePlannedMaintenance = NewResourceMetaResource(KindPlannedMaintenance)
|
||||
ResourceMetaResourceSavedView = NewResourceMetaResource(KindSavedView)
|
||||
ResourceMetaResourceSavedView = NewResourceMetaResource(KindSavedView, VerbCreate, VerbList, VerbRead, VerbUpdate, VerbDelete)
|
||||
ResourceMetaResourceTraceFunnel = NewResourceMetaResource(KindTraceFunnel)
|
||||
ResourceMetaResourceFactorPassword = NewResourceMetaResource(KindFactorPassword)
|
||||
ResourceMetaResourceFactorAPIKey = NewResourceMetaResource(KindFactorAPIKey, VerbCreate, VerbList, VerbRead, VerbUpdate, VerbDelete)
|
||||
|
||||
@@ -155,6 +155,8 @@ var operatorInverseMapping = map[FilterOperator]FilterOperator{
|
||||
// doesn't have value "redis"
|
||||
// Since we don't know the intent, we don't add the exists filter. They are expected
|
||||
// to add exists filter themselves if exclusion is desired.
|
||||
// Negative predicates therefore include rows where the key is absent; value
|
||||
// expressions must preserve the storage column's absent-key default.
|
||||
//
|
||||
// For the positive predicates, the key existence is implied.
|
||||
func (f FilterOperator) AddDefaultExistsFilter() bool {
|
||||
|
||||
@@ -1,31 +1,156 @@
|
||||
package savedviewtypes
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/types"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/uptrace/bun"
|
||||
"k8s.io/apimachinery/pkg/util/validation"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrCodeSavedViewInvalidInput = errors.MustNewCode("saved_view_invalid_input")
|
||||
ErrCodeSavedViewNotFound = errors.MustNewCode("saved_view_not_found")
|
||||
)
|
||||
|
||||
// savedViewNameSuffixLen mirrors dashboardtypes' generated-name logic.
|
||||
const savedViewNameSuffixLen = 8
|
||||
|
||||
var (
|
||||
SourceTraces = Source{valuer.NewString("traces")}
|
||||
SourceLogs = Source{valuer.NewString("logs")}
|
||||
SourceMetrics = Source{valuer.NewString("metrics")}
|
||||
SourceMeter = Source{valuer.NewString("meter")}
|
||||
)
|
||||
|
||||
type SavedView struct {
|
||||
bun.BaseModel `bun:"table:saved_views"`
|
||||
bun.BaseModel `bun:"table:saved_view"`
|
||||
|
||||
types.Identifiable
|
||||
types.TimeAuditable
|
||||
types.UserAuditable
|
||||
OrgID string `json:"orgId" bun:"org_id,notnull"`
|
||||
Name string `json:"name" bun:"name,type:text,notnull"`
|
||||
Category string `json:"category" bun:"category,type:text,notnull"`
|
||||
SourcePage string `json:"sourcePage" bun:"source_page,type:text,notnull"`
|
||||
Tags string `json:"tags" bun:"tags,type:text"`
|
||||
Data string `json:"data" bun:"data,type:text,notnull"`
|
||||
ExtraData string `json:"extraData" bun:"extra_data,type:text"`
|
||||
OrgID string `json:"-" bun:"org_id,notnull"`
|
||||
Name string `json:"name" bun:"name,type:text,notnull"`
|
||||
Source Source `json:"source" bun:"source,type:text,notnull"`
|
||||
Data SavedViewData `json:"data" bun:"data,type:text,notnull"`
|
||||
}
|
||||
|
||||
type PostableSavedView struct {
|
||||
Name string `json:"name"`
|
||||
GenerateName bool `json:"generateName"`
|
||||
Source Source `json:"source" required:"true"`
|
||||
Data SavedViewData `json:"data" required:"true"`
|
||||
}
|
||||
|
||||
type UpdatableSavedView struct {
|
||||
Source Source `json:"source" required:"true"`
|
||||
Data SavedViewData `json:"data" required:"true"`
|
||||
}
|
||||
|
||||
type ListSavedViewsParams struct {
|
||||
Source Source `query:"source"`
|
||||
Name string `query:"name"`
|
||||
}
|
||||
|
||||
type Source struct {
|
||||
valuer.String
|
||||
}
|
||||
|
||||
func (Source) Enum() []any {
|
||||
return []any{
|
||||
SourceTraces,
|
||||
SourceLogs,
|
||||
SourceMetrics,
|
||||
SourceMeter,
|
||||
}
|
||||
}
|
||||
|
||||
func (s Source) Validate() error {
|
||||
switch s {
|
||||
case SourceTraces, SourceLogs, SourceMetrics, SourceMeter:
|
||||
return nil
|
||||
default:
|
||||
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "invalid source: %s", s.StringValue())
|
||||
}
|
||||
}
|
||||
|
||||
func (postable PostableSavedView) ToSavedView(orgID string, createdBy string) *SavedView {
|
||||
now := time.Now()
|
||||
|
||||
name := postable.Name
|
||||
if postable.GenerateName {
|
||||
name = generateSavedViewName(postable.Data.Spec.DisplayName)
|
||||
}
|
||||
|
||||
return &SavedView{
|
||||
Identifiable: types.Identifiable{ID: valuer.GenerateUUID()},
|
||||
TimeAuditable: types.TimeAuditable{CreatedAt: now, UpdatedAt: now},
|
||||
UserAuditable: types.UserAuditable{CreatedBy: createdBy, UpdatedBy: createdBy},
|
||||
OrgID: orgID,
|
||||
Name: name,
|
||||
Source: postable.Source,
|
||||
Data: postable.Data,
|
||||
}
|
||||
}
|
||||
|
||||
// ToSavedView builds the row to write for an update. Name is immutable and
|
||||
// deliberately absent -- the caller identifies the row by id/orgID alone.
|
||||
func (updatable UpdatableSavedView) ToSavedView(id valuer.UUID, orgID string, updatedBy string) *SavedView {
|
||||
return &SavedView{
|
||||
Identifiable: types.Identifiable{ID: id},
|
||||
TimeAuditable: types.TimeAuditable{UpdatedAt: time.Now()},
|
||||
UserAuditable: types.UserAuditable{UpdatedBy: updatedBy},
|
||||
OrgID: orgID,
|
||||
Source: updatable.Source,
|
||||
Data: updatable.Data,
|
||||
}
|
||||
}
|
||||
|
||||
func (p *PostableSavedView) Validate() error {
|
||||
if err := p.validateName(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := p.Source.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return p.Data.Validate()
|
||||
}
|
||||
|
||||
func (p *PostableSavedView) validateName() error {
|
||||
if !p.GenerateName {
|
||||
return validateSavedViewName(p.Name)
|
||||
}
|
||||
if p.Name != "" {
|
||||
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "name must be empty when generateName is true, got %q", p.Name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (u *UpdatableSavedView) Validate() error {
|
||||
if err := u.Source.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return u.Data.Validate()
|
||||
}
|
||||
|
||||
func (p *ListSavedViewsParams) Validate() error {
|
||||
if p.Source.IsZero() {
|
||||
return nil
|
||||
}
|
||||
|
||||
return p.Source.Validate()
|
||||
}
|
||||
|
||||
func NewStatsFromSavedViews(savedViews []*SavedView) map[string]any {
|
||||
stats := make(map[string]any)
|
||||
for _, savedView := range savedViews {
|
||||
key := "savedview.source." + strings.ToLower(string(savedView.SourcePage)) + ".count"
|
||||
key := "savedview.source." + strings.ToLower(savedView.Source.StringValue()) + ".count"
|
||||
if _, ok := stats[key]; !ok {
|
||||
stats[key] = int64(1)
|
||||
} else {
|
||||
@@ -36,3 +161,54 @@ func NewStatsFromSavedViews(savedViews []*SavedView) map[string]any {
|
||||
stats["savedview.count"] = int64(len(savedViews))
|
||||
return stats
|
||||
}
|
||||
|
||||
// Matches https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-label-names.
|
||||
func validateSavedViewName(name string) error {
|
||||
if name == "" {
|
||||
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "name is required")
|
||||
}
|
||||
if errs := validation.IsDNS1123Label(name); len(errs) > 0 {
|
||||
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "name %q is invalid: %s", name, strings.Join(errs, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// generateSavedViewName is a copy of dashboardtypes.generateDashboardName: slugify
|
||||
// the display name and append a random suffix for practical collision avoidance
|
||||
// (the DB unique index on (org_id, name) is what actually guarantees uniqueness).
|
||||
func generateSavedViewName(displayName string) string {
|
||||
const dns1123LabelMaxLen = 63
|
||||
suffixAlphabet := []byte("abcdefghijklmnopqrstuvwxyz0123456789")
|
||||
|
||||
var b strings.Builder
|
||||
b.Grow(len(displayName))
|
||||
prevHyphen := false
|
||||
for _, r := range strings.ToLower(displayName) {
|
||||
switch {
|
||||
case (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9'):
|
||||
b.WriteRune(r)
|
||||
prevHyphen = false
|
||||
case b.Len() > 0 && !prevHyphen:
|
||||
b.WriteByte('-')
|
||||
prevHyphen = true
|
||||
}
|
||||
}
|
||||
prefix := strings.TrimRight(b.String(), "-")
|
||||
|
||||
suffix := make([]byte, savedViewNameSuffixLen)
|
||||
if _, err := rand.Read(suffix); err != nil {
|
||||
panic(errors.WrapInternalf(err, errors.CodeInternal, "read random for saved view name suffix"))
|
||||
}
|
||||
for i := range suffix {
|
||||
suffix[i] = suffixAlphabet[int(suffix[i])%len(suffixAlphabet)]
|
||||
}
|
||||
|
||||
maxPrefix := dns1123LabelMaxLen - 1 - savedViewNameSuffixLen
|
||||
if len(prefix) > maxPrefix {
|
||||
prefix = strings.TrimRight(prefix[:maxPrefix], "-")
|
||||
}
|
||||
if prefix == "" {
|
||||
return string(suffix)
|
||||
}
|
||||
return prefix + "-" + string(suffix)
|
||||
}
|
||||
|
||||
228
pkg/types/savedviewtypes/savedview_test.go
Normal file
228
pkg/types/savedviewtypes/savedview_test.go
Normal file
@@ -0,0 +1,228 @@
|
||||
package savedviewtypes
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"k8s.io/apimachinery/pkg/util/validation"
|
||||
)
|
||||
|
||||
func validPostableSavedView() PostableSavedView {
|
||||
return PostableSavedView{
|
||||
Name: "my-view",
|
||||
Source: SourceLogs,
|
||||
Data: SavedViewData{
|
||||
SchemaVersion: SavedViewSchemaVersion,
|
||||
Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func validUpdatableSavedView() UpdatableSavedView {
|
||||
return UpdatableSavedView{
|
||||
Source: SourceLogs,
|
||||
Data: SavedViewData{
|
||||
SchemaVersion: SavedViewSchemaVersion,
|
||||
Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceValidate(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
source Source
|
||||
expectError bool
|
||||
}{
|
||||
{name: "traces", source: SourceTraces},
|
||||
{name: "logs", source: SourceLogs},
|
||||
{name: "metrics", source: SourceMetrics},
|
||||
{name: "meter", source: SourceMeter},
|
||||
{name: "unknown is rejected", source: Source{valuer.NewString("bogus")}, expectError: true},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
err := c.source.Validate()
|
||||
if c.expectError {
|
||||
assert.Error(t, err)
|
||||
} else {
|
||||
assert.NoError(t, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPostableSavedViewValidate(t *testing.T) {
|
||||
t.Run("valid view", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
assert.NoError(t, view.Validate())
|
||||
})
|
||||
|
||||
t.Run("invalid source is rejected", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
view.Source = Source{valuer.NewString("bogus")}
|
||||
assert.Error(t, view.Validate())
|
||||
})
|
||||
|
||||
t.Run("invalid saved view data is rejected", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
view.Data.SchemaVersion = "v1"
|
||||
assert.Error(t, view.Validate())
|
||||
})
|
||||
|
||||
t.Run("invalid name is rejected", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
view.Name = "My View"
|
||||
assert.Error(t, view.Validate())
|
||||
})
|
||||
|
||||
t.Run("empty name without generateName is rejected", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
view.Name = ""
|
||||
assert.ErrorContains(t, view.Validate(), "name is required")
|
||||
})
|
||||
|
||||
t.Run("generateName true with empty name is allowed -- generated at ToSavedView time", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
view.Name = ""
|
||||
view.GenerateName = true
|
||||
assert.NoError(t, view.Validate())
|
||||
})
|
||||
|
||||
t.Run("generateName true with a non-empty name is rejected", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
view.GenerateName = true
|
||||
assert.ErrorContains(t, view.Validate(), "name must be empty when generateName is true")
|
||||
})
|
||||
|
||||
t.Run("empty displayName is rejected", func(t *testing.T) {
|
||||
view := validPostableSavedView()
|
||||
view.Data.Spec.DisplayName = ""
|
||||
assert.ErrorContains(t, view.Validate(), "displayName is required")
|
||||
})
|
||||
}
|
||||
|
||||
func TestUpdatableSavedViewValidate(t *testing.T) {
|
||||
t.Run("valid view", func(t *testing.T) {
|
||||
view := validUpdatableSavedView()
|
||||
assert.NoError(t, view.Validate())
|
||||
})
|
||||
|
||||
t.Run("invalid source is rejected", func(t *testing.T) {
|
||||
view := validUpdatableSavedView()
|
||||
view.Source = Source{valuer.NewString("bogus")}
|
||||
assert.Error(t, view.Validate())
|
||||
})
|
||||
|
||||
t.Run("empty displayName is rejected", func(t *testing.T) {
|
||||
view := validUpdatableSavedView()
|
||||
view.Data.Spec.DisplayName = ""
|
||||
assert.ErrorContains(t, view.Validate(), "displayName is required")
|
||||
})
|
||||
}
|
||||
|
||||
func TestListSavedViewsParamsValidate(t *testing.T) {
|
||||
t.Run("zero source is allowed", func(t *testing.T) {
|
||||
params := ListSavedViewsParams{}
|
||||
assert.NoError(t, params.Validate())
|
||||
})
|
||||
|
||||
t.Run("valid source is allowed", func(t *testing.T) {
|
||||
params := ListSavedViewsParams{Source: SourceLogs}
|
||||
assert.NoError(t, params.Validate())
|
||||
})
|
||||
|
||||
t.Run("invalid source is rejected", func(t *testing.T) {
|
||||
params := ListSavedViewsParams{Source: Source{valuer.NewString("bogus")}}
|
||||
assert.Error(t, params.Validate())
|
||||
})
|
||||
}
|
||||
|
||||
func TestNewSavedView(t *testing.T) {
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
view := validPostableSavedView()
|
||||
|
||||
savedView := view.ToSavedView(orgID, "creator@signoz.io")
|
||||
|
||||
assert.False(t, savedView.ID.IsZero())
|
||||
assert.Equal(t, orgID, savedView.OrgID)
|
||||
assert.Equal(t, "creator@signoz.io", savedView.CreatedBy)
|
||||
assert.Equal(t, "creator@signoz.io", savedView.UpdatedBy)
|
||||
assert.Equal(t, view.Name, savedView.Name)
|
||||
assert.Equal(t, view.Source, savedView.Source)
|
||||
assert.Equal(t, view.Data, savedView.Data)
|
||||
assert.False(t, savedView.CreatedAt.IsZero())
|
||||
assert.Equal(t, savedView.CreatedAt, savedView.UpdatedAt)
|
||||
}
|
||||
|
||||
func TestNewSavedView_GeneratesNameWhenEmpty(t *testing.T) {
|
||||
orgID := valuer.GenerateUUID().StringValue()
|
||||
view := validPostableSavedView()
|
||||
view.Name = ""
|
||||
view.GenerateName = true
|
||||
view.Data.Spec.DisplayName = "My View!"
|
||||
|
||||
savedView := view.ToSavedView(orgID, "creator@signoz.io")
|
||||
|
||||
assert.NotEmpty(t, savedView.Name)
|
||||
assert.Empty(t, validation.IsDNS1123Label(savedView.Name), "generated name must be a valid DNS-1123 label")
|
||||
assert.True(t, strings.HasPrefix(savedView.Name, "my-view-"))
|
||||
assert.Equal(t, "My View!", savedView.Data.Spec.DisplayName)
|
||||
}
|
||||
|
||||
func TestGenerateSavedViewName(t *testing.T) {
|
||||
tests := []struct {
|
||||
scenario string
|
||||
input string
|
||||
wantPrefix string
|
||||
}{
|
||||
{scenario: "simple words with spaces", input: "My View", wantPrefix: "my-view"},
|
||||
{scenario: "punctuation collapses", input: "Hello, World!", wantPrefix: "hello-world"},
|
||||
{scenario: "leading and trailing whitespace", input: " hello ", wantPrefix: "hello"},
|
||||
{scenario: "leading and trailing hyphens", input: "---abc---", wantPrefix: "abc"},
|
||||
{scenario: "consecutive non-alphanumerics collapse", input: "a___b...c", wantPrefix: "a-b-c"},
|
||||
{scenario: "digits are preserved", input: "Region us-east-1", wantPrefix: "region-us-east-1"},
|
||||
{scenario: "no alphanumerics drops prefix and returns suffix only", input: "!!! ???", wantPrefix: ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.scenario, func(t *testing.T) {
|
||||
got := generateSavedViewName(tt.input)
|
||||
assert.NotEmpty(t, got)
|
||||
assert.LessOrEqual(t, len(got), 63)
|
||||
assert.Empty(t, validation.IsDNS1123Label(got), "result must be a valid DNS-1123 label")
|
||||
|
||||
if tt.wantPrefix == "" {
|
||||
assert.Len(t, got, savedViewNameSuffixLen, "expected the bare random suffix")
|
||||
return
|
||||
}
|
||||
expectedPrefix := tt.wantPrefix + "-"
|
||||
assert.True(t, strings.HasPrefix(got, expectedPrefix), "expected prefix %q, got %q", expectedPrefix, got)
|
||||
assert.Len(t, got, len(expectedPrefix)+savedViewNameSuffixLen)
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("suffix differs across calls", func(t *testing.T) {
|
||||
first := generateSavedViewName("collision-test")
|
||||
second := generateSavedViewName("collision-test")
|
||||
assert.NotEqual(t, first, second, "expected the random suffix to differ across calls")
|
||||
})
|
||||
}
|
||||
|
||||
func TestNewStatsFromSavedViews(t *testing.T) {
|
||||
views := []*SavedView{
|
||||
{Source: SourceLogs},
|
||||
{Source: SourceLogs},
|
||||
{Source: SourceTraces},
|
||||
}
|
||||
|
||||
stats := NewStatsFromSavedViews(views)
|
||||
|
||||
assert.Equal(t, int64(3), stats["savedview.count"])
|
||||
assert.Equal(t, int64(2), stats["savedview.source.logs.count"])
|
||||
assert.Equal(t, int64(1), stats["savedview.source.traces.count"])
|
||||
assert.NotContains(t, stats, "savedview.source.metrics.count")
|
||||
}
|
||||
88
pkg/types/savedviewtypes/savedviewtypestest/store.go
Normal file
88
pkg/types/savedviewtypes/savedviewtypestest/store.go
Normal file
@@ -0,0 +1,88 @@
|
||||
package savedviewtypestest
|
||||
|
||||
import (
|
||||
"database/sql/driver"
|
||||
"encoding/json"
|
||||
"regexp"
|
||||
|
||||
"github.com/DATA-DOG/go-sqlmock"
|
||||
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
var savedViewColumns = []string{"id", "created_at", "updated_at", "created_by", "updated_by", "org_id", "name", "source", "data"}
|
||||
|
||||
type StoreTest struct {
|
||||
store savedviewtypes.Store
|
||||
mock sqlmock.Sqlmock
|
||||
}
|
||||
|
||||
func New(store savedviewtypes.Store, mock sqlmock.Sqlmock) *StoreTest {
|
||||
return &StoreTest{store: store, mock: mock}
|
||||
}
|
||||
|
||||
// Store returns the savedviewtypes.Store for calling methods under test.
|
||||
func (t *StoreTest) Store() savedviewtypes.Store { return t.store }
|
||||
|
||||
// Mock returns the sqlmock handle for setting query expectations.
|
||||
func (t *StoreTest) Mock() sqlmock.Sqlmock { return t.mock }
|
||||
|
||||
func savedViewRow(view *savedviewtypes.SavedView) []driver.Value {
|
||||
data, _ := json.Marshal(view.Data)
|
||||
return []driver.Value{
|
||||
view.ID.StringValue(),
|
||||
view.CreatedAt,
|
||||
view.UpdatedAt,
|
||||
view.CreatedBy,
|
||||
view.UpdatedBy,
|
||||
view.OrgID,
|
||||
view.Name,
|
||||
view.Source.StringValue(),
|
||||
string(data),
|
||||
}
|
||||
}
|
||||
|
||||
// ExpectCreate sets up the SQL expectation for a Create call.
|
||||
func (t *StoreTest) ExpectCreate() {
|
||||
t.mock.ExpectExec(`INSERT INTO "saved_view"`).WillReturnResult(sqlmock.NewResult(1, 1))
|
||||
}
|
||||
|
||||
// ExpectGet sets up the SQL expectation for a Get call. Pass view = nil to
|
||||
// simulate a not-found row.
|
||||
func (t *StoreTest) ExpectGet(orgID string, id valuer.UUID, view *savedviewtypes.SavedView) {
|
||||
rows := sqlmock.NewRows(savedViewColumns)
|
||||
if view != nil {
|
||||
rows.AddRow(savedViewRow(view)...)
|
||||
}
|
||||
|
||||
t.mock.ExpectQuery(`SELECT (.+) FROM "saved_view".+WHERE \(org_id = '` + regexp.QuoteMeta(orgID) + `' AND id = '` + regexp.QuoteMeta(id.StringValue()) + `'\)`).
|
||||
WillReturnRows(rows)
|
||||
}
|
||||
|
||||
// ExpectUpdate sets up the SQL expectation for an Update call scoped to
|
||||
// orgID/id. rowsAffected = 0 simulates a not-found target row.
|
||||
func (t *StoreTest) ExpectUpdate(orgID string, id valuer.UUID, rowsAffected int64) {
|
||||
t.mock.ExpectExec(`UPDATE "saved_view".+WHERE \(id = '` + regexp.QuoteMeta(id.StringValue()) + `'\) AND \(org_id = '` + regexp.QuoteMeta(orgID) + `'\)`).
|
||||
WillReturnResult(sqlmock.NewResult(0, rowsAffected))
|
||||
}
|
||||
|
||||
// ExpectDelete sets up the SQL expectation for a Delete call scoped to
|
||||
// orgID/id. rowsAffected = 0 simulates a not-found target row.
|
||||
func (t *StoreTest) ExpectDelete(orgID string, id valuer.UUID, rowsAffected int64) {
|
||||
t.mock.ExpectExec(`DELETE FROM "saved_view".+WHERE \(id = '` + regexp.QuoteMeta(id.StringValue()) + `'\) AND \(org_id = '` + regexp.QuoteMeta(orgID) + `'\)`).
|
||||
WillReturnResult(sqlmock.NewResult(0, rowsAffected))
|
||||
}
|
||||
|
||||
// ExpectList sets up the SQL expectation for a List call scoped to orgID.
|
||||
func (t *StoreTest) ExpectList(orgID string, views []*savedviewtypes.SavedView) {
|
||||
rows := sqlmock.NewRows(savedViewColumns)
|
||||
for _, view := range views {
|
||||
rows.AddRow(savedViewRow(view)...)
|
||||
}
|
||||
|
||||
t.mock.ExpectQuery(`SELECT (.+) FROM "saved_view".+WHERE \(org_id = '` + regexp.QuoteMeta(orgID) + `'\)`).WillReturnRows(rows)
|
||||
}
|
||||
|
||||
func (t *StoreTest) AssertExpectations() error {
|
||||
return t.mock.ExpectationsWereMet()
|
||||
}
|
||||
85
pkg/types/savedviewtypes/spec.go
Normal file
85
pkg/types/savedviewtypes/spec.go
Normal file
@@ -0,0 +1,85 @@
|
||||
package savedviewtypes
|
||||
|
||||
import (
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
// SavedViewSchemaVersion is the only schemaVersion currently.
|
||||
const SavedViewSchemaVersion = "v2"
|
||||
|
||||
var (
|
||||
PanelTypeValue = PanelType{valuer.NewString("value")}
|
||||
PanelTypeGraph = PanelType{valuer.NewString("graph")}
|
||||
PanelTypeTable = PanelType{valuer.NewString("table")}
|
||||
PanelTypeList = PanelType{valuer.NewString("list")}
|
||||
PanelTypeTrace = PanelType{valuer.NewString("trace")}
|
||||
)
|
||||
|
||||
// Display holds view-rendering preferences.
|
||||
type Display struct {
|
||||
MaxLines int `json:"maxLines"`
|
||||
FontSize string `json:"fontSize"`
|
||||
Format string `json:"format"`
|
||||
Color string `json:"color"`
|
||||
}
|
||||
|
||||
// SavedViewSpec is the typed content of a saved view.
|
||||
type SavedViewSpec struct {
|
||||
DisplayName string `json:"displayName" required:"true"`
|
||||
PanelType PanelType `json:"panelType" required:"true"`
|
||||
Queries []qbtypes.QueryEnvelope `json:"queries" required:"true" nullable:"false"`
|
||||
SelectedFields []telemetrytypes.TelemetryFieldKey `json:"selectedFields" required:"true" nullable:"false"`
|
||||
Display Display `json:"display" required:"true"`
|
||||
}
|
||||
|
||||
// SavedViewData is what's persisted as saved view data.
|
||||
type SavedViewData struct {
|
||||
SchemaVersion string `json:"schemaVersion" required:"true"`
|
||||
Spec SavedViewSpec `json:"spec" required:"true"`
|
||||
}
|
||||
|
||||
// PanelType is the explore-page panel a saved view renders as.
|
||||
type PanelType struct {
|
||||
valuer.String
|
||||
}
|
||||
|
||||
func (PanelType) Enum() []any {
|
||||
return []any{
|
||||
PanelTypeValue,
|
||||
PanelTypeGraph,
|
||||
PanelTypeTable,
|
||||
PanelTypeList,
|
||||
PanelTypeTrace,
|
||||
}
|
||||
}
|
||||
|
||||
func (p PanelType) Validate() error {
|
||||
switch p {
|
||||
case PanelTypeValue, PanelTypeGraph, PanelTypeTable, PanelTypeList, PanelTypeTrace:
|
||||
return nil
|
||||
default:
|
||||
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "invalid panel type: %s", p.StringValue())
|
||||
}
|
||||
}
|
||||
|
||||
func (s *SavedViewSpec) Validate() error {
|
||||
if s.DisplayName == "" {
|
||||
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "displayName is required")
|
||||
}
|
||||
if err := s.PanelType.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return (&qbtypes.CompositeQuery{Queries: s.Queries}).Validate()
|
||||
}
|
||||
|
||||
func (d *SavedViewData) Validate() error {
|
||||
if d.SchemaVersion != SavedViewSchemaVersion {
|
||||
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "schemaVersion must be %q, got %q", SavedViewSchemaVersion, d.SchemaVersion)
|
||||
}
|
||||
|
||||
return d.Spec.Validate()
|
||||
}
|
||||
140
pkg/types/savedviewtypes/spec_test.go
Normal file
140
pkg/types/savedviewtypes/spec_test.go
Normal file
@@ -0,0 +1,140 @@
|
||||
package savedviewtypes
|
||||
|
||||
import (
|
||||
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
|
||||
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func validQueries() []qbtypes.QueryEnvelope {
|
||||
return []qbtypes.QueryEnvelope{
|
||||
{
|
||||
Type: qbtypes.QueryTypeBuilder,
|
||||
Spec: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
|
||||
Signal: telemetrytypes.SignalLogs,
|
||||
Aggregations: []qbtypes.LogAggregation{{Expression: "count()"}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func TestPanelTypeValidate(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
panelType PanelType
|
||||
expectError bool
|
||||
}{
|
||||
{name: "value", panelType: PanelTypeValue},
|
||||
{name: "graph", panelType: PanelTypeGraph},
|
||||
{name: "table", panelType: PanelTypeTable},
|
||||
{name: "list", panelType: PanelTypeList},
|
||||
{name: "trace", panelType: PanelTypeTrace},
|
||||
{name: "unknown is rejected", panelType: PanelType{valuer.NewString("bogus")}, expectError: true},
|
||||
{name: "empty is rejected", panelType: PanelType{}, expectError: true},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
err := c.panelType.Validate()
|
||||
if c.expectError {
|
||||
assert.Error(t, err)
|
||||
} else {
|
||||
assert.NoError(t, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSavedViewSpecValidate(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
spec SavedViewSpec
|
||||
expectError bool
|
||||
}{
|
||||
{
|
||||
name: "valid spec",
|
||||
spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()},
|
||||
expectError: false,
|
||||
},
|
||||
{
|
||||
name: "empty display name is rejected",
|
||||
spec: SavedViewSpec{PanelType: PanelTypeGraph, Queries: validQueries()},
|
||||
expectError: true,
|
||||
},
|
||||
{
|
||||
name: "invalid panel type is rejected before queries are checked",
|
||||
spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelType{valuer.NewString("bogus")}, Queries: validQueries()},
|
||||
expectError: true,
|
||||
},
|
||||
{
|
||||
name: "no queries is rejected",
|
||||
spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph},
|
||||
expectError: true,
|
||||
},
|
||||
{
|
||||
name: "selected fields and display are not required",
|
||||
spec: SavedViewSpec{
|
||||
DisplayName: "My View",
|
||||
PanelType: PanelTypeTable,
|
||||
Queries: validQueries(),
|
||||
SelectedFields: []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}},
|
||||
Display: Display{MaxLines: 3, FontSize: "small", Format: "table", Color: "blue"},
|
||||
},
|
||||
expectError: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
err := c.spec.Validate()
|
||||
if c.expectError {
|
||||
assert.Error(t, err)
|
||||
} else {
|
||||
assert.NoError(t, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSavedViewDataValidate(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
data SavedViewData
|
||||
expectError bool
|
||||
}{
|
||||
{
|
||||
name: "valid data",
|
||||
data: SavedViewData{SchemaVersion: SavedViewSchemaVersion, Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()}},
|
||||
expectError: false,
|
||||
},
|
||||
{
|
||||
name: "wrong schema version is rejected",
|
||||
data: SavedViewData{SchemaVersion: "v1", Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()}},
|
||||
expectError: true,
|
||||
},
|
||||
{
|
||||
name: "empty schema version is rejected",
|
||||
data: SavedViewData{Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()}},
|
||||
expectError: true,
|
||||
},
|
||||
{
|
||||
name: "invalid spec is rejected",
|
||||
data: SavedViewData{SchemaVersion: SavedViewSchemaVersion, Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph}},
|
||||
expectError: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
err := c.data.Validate()
|
||||
if c.expectError {
|
||||
assert.Error(t, err)
|
||||
} else {
|
||||
assert.NoError(t, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
15
pkg/types/savedviewtypes/store.go
Normal file
15
pkg/types/savedviewtypes/store.go
Normal file
@@ -0,0 +1,15 @@
|
||||
package savedviewtypes
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
)
|
||||
|
||||
type Store interface {
|
||||
Create(ctx context.Context, view *SavedView) error
|
||||
Get(ctx context.Context, orgID string, id valuer.UUID) (*SavedView, error)
|
||||
Update(ctx context.Context, view *SavedView) error
|
||||
Delete(ctx context.Context, orgID string, id valuer.UUID) error
|
||||
List(ctx context.Context, orgID string, source Source, name string) ([]*SavedView, error)
|
||||
}
|
||||
@@ -2,6 +2,7 @@ package telemetrytypes
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/SigNoz/signoz/pkg/valuer"
|
||||
@@ -50,6 +51,74 @@ type TelemetryFieldKey struct {
|
||||
Evolutions []*EvolutionEntry `json:"-"`
|
||||
}
|
||||
|
||||
// Copy returns an independent copy of f.
|
||||
func (f *TelemetryFieldKey) Copy() *TelemetryFieldKey {
|
||||
if f == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
copied := *f
|
||||
copied.Indexes = slices.Clone(f.Indexes)
|
||||
if f.Evolutions != nil {
|
||||
copied.Evolutions = make([]*EvolutionEntry, len(f.Evolutions))
|
||||
for index, evolution := range f.Evolutions {
|
||||
if evolution != nil {
|
||||
copiedEvolution := *evolution
|
||||
copied.Evolutions[index] = &copiedEvolution
|
||||
}
|
||||
}
|
||||
}
|
||||
copied.JSONPlan = copyJSONAccessPlan(f.JSONPlan, f, &copied)
|
||||
|
||||
return &copied
|
||||
}
|
||||
|
||||
func copyJSONAccessPlan(plan JSONAccessPlan, sourceKey, copiedKey *TelemetryFieldKey) JSONAccessPlan {
|
||||
if plan == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
nodes := make(map[*JSONAccessNode]*JSONAccessNode)
|
||||
var copyNode func(*JSONAccessNode) *JSONAccessNode
|
||||
copyNode = func(node *JSONAccessNode) *JSONAccessNode {
|
||||
if node == nil {
|
||||
return nil
|
||||
}
|
||||
if copiedNode, ok := nodes[node]; ok {
|
||||
return copiedNode
|
||||
}
|
||||
|
||||
copiedNode := *node
|
||||
nodes[node] = &copiedNode
|
||||
copiedNode.Parent = copyNode(node.Parent)
|
||||
if node.Branches != nil {
|
||||
copiedNode.Branches = make(map[JSONAccessBranchType]*JSONAccessNode, len(node.Branches))
|
||||
for branchType, branch := range node.Branches {
|
||||
copiedNode.Branches[branchType] = copyNode(branch)
|
||||
}
|
||||
}
|
||||
if node.TerminalConfig != nil {
|
||||
copiedTerminal := *node.TerminalConfig
|
||||
switch node.TerminalConfig.Key {
|
||||
case nil:
|
||||
case sourceKey:
|
||||
copiedTerminal.Key = copiedKey
|
||||
default:
|
||||
copiedTerminal.Key = node.TerminalConfig.Key.Copy()
|
||||
}
|
||||
copiedNode.TerminalConfig = &copiedTerminal
|
||||
}
|
||||
|
||||
return &copiedNode
|
||||
}
|
||||
|
||||
copied := make(JSONAccessPlan, len(plan))
|
||||
for index, node := range plan {
|
||||
copied[index] = copyNode(node)
|
||||
}
|
||||
return copied
|
||||
}
|
||||
|
||||
func (f *TelemetryFieldKey) KeyNameContainsArray() bool {
|
||||
return strings.Contains(f.Name, ArraySep) || strings.Contains(f.Name, ArrayAnyIndex)
|
||||
}
|
||||
@@ -233,6 +302,15 @@ type MetricContext struct {
|
||||
MetricNamespace string `json:"metricNamespace,omitempty"`
|
||||
}
|
||||
|
||||
// Copy returns an independent copy of m.
|
||||
func (m *MetricContext) Copy() *MetricContext {
|
||||
if m == nil {
|
||||
return nil
|
||||
}
|
||||
copied := *m
|
||||
return &copied
|
||||
}
|
||||
|
||||
type FieldKeySelector struct {
|
||||
StartUnixMilli int64 `json:"startUnixMilli"`
|
||||
EndUnixMilli int64 `json:"endUnixMilli"`
|
||||
@@ -246,6 +324,16 @@ type FieldKeySelector struct {
|
||||
MetricContext *MetricContext `json:"metricContext,omitempty"`
|
||||
}
|
||||
|
||||
// Copy returns an independent copy of s.
|
||||
func (s *FieldKeySelector) Copy() *FieldKeySelector {
|
||||
if s == nil {
|
||||
return nil
|
||||
}
|
||||
copied := *s
|
||||
copied.MetricContext = s.MetricContext.Copy()
|
||||
return &copied
|
||||
}
|
||||
|
||||
type FieldValueSelector struct {
|
||||
*FieldKeySelector
|
||||
ExistingQuery string `json:"existingQuery"`
|
||||
|
||||
75
pkg/types/telemetrytypes/field_copy_test.go
Normal file
75
pkg/types/telemetrytypes/field_copy_test.go
Normal file
@@ -0,0 +1,75 @@
|
||||
package telemetrytypes
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestTelemetryFieldKeyCopyOwnsMutableState(t *testing.T) {
|
||||
original := &TelemetryFieldKey{
|
||||
Name: "items.name",
|
||||
FieldContext: FieldContextBody,
|
||||
FieldDataType: FieldDataTypeString,
|
||||
Indexes: []TelemetryFieldKeySkipIndex{
|
||||
{Name: "items.name"},
|
||||
},
|
||||
Evolutions: []*EvolutionEntry{
|
||||
{FieldName: "items.name"},
|
||||
},
|
||||
}
|
||||
require.NoError(t, original.SetJSONAccessPlan(JSONColumnMetadata{BaseColumn: "body_v2"}, nil))
|
||||
require.Len(t, original.JSONPlan, 1)
|
||||
require.NotNil(t, original.JSONPlan[0].TerminalConfig)
|
||||
|
||||
copied := original.Copy()
|
||||
require.NotNil(t, copied)
|
||||
require.NotSame(t, original, copied)
|
||||
require.Len(t, copied.JSONPlan, 1)
|
||||
require.NotNil(t, copied.JSONPlan[0].TerminalConfig)
|
||||
|
||||
assert.NotSame(t, original.JSONPlan[0], copied.JSONPlan[0])
|
||||
assert.NotSame(t, original.JSONPlan[0].Parent, copied.JSONPlan[0].Parent)
|
||||
assert.Same(t, copied, copied.JSONPlan[0].TerminalConfig.Key)
|
||||
assert.Equal(t, original.JSONPlan[0].Alias(), copied.JSONPlan[0].Alias())
|
||||
|
||||
copied.Name = "changed"
|
||||
copied.Indexes[0].Name = "changed"
|
||||
copied.Evolutions[0].FieldName = "changed"
|
||||
copied.JSONPlan[0].Name = "changed"
|
||||
copied.JSONPlan[0].Parent.Name = "changed"
|
||||
|
||||
assert.Equal(t, "items.name", original.Name)
|
||||
assert.Equal(t, "items.name", original.Indexes[0].Name)
|
||||
assert.Equal(t, "items.name", original.Evolutions[0].FieldName)
|
||||
assert.Equal(t, "items.name", original.JSONPlan[0].Name)
|
||||
assert.Equal(t, "body_v2", original.JSONPlan[0].Parent.Name)
|
||||
}
|
||||
|
||||
func TestFieldKeySelectorCopyOwnsMetricContext(t *testing.T) {
|
||||
original := &FieldKeySelector{
|
||||
Name: "state",
|
||||
MetricContext: &MetricContext{
|
||||
MetricName: "system.cpu.time",
|
||||
MetricNamespace: "system",
|
||||
},
|
||||
}
|
||||
|
||||
copied := original.Copy()
|
||||
require.NotNil(t, copied)
|
||||
require.NotNil(t, copied.MetricContext)
|
||||
assert.NotSame(t, original.MetricContext, copied.MetricContext)
|
||||
|
||||
copied.Name = "changed"
|
||||
copied.MetricContext.MetricName = "changed"
|
||||
|
||||
assert.Equal(t, "state", original.Name)
|
||||
assert.Equal(t, "system.cpu.time", original.MetricContext.MetricName)
|
||||
}
|
||||
|
||||
func TestNilFieldCopies(t *testing.T) {
|
||||
assert.Nil(t, (*TelemetryFieldKey)(nil).Copy())
|
||||
assert.Nil(t, (*FieldKeySelector)(nil).Copy())
|
||||
assert.Nil(t, (*MetricContext)(nil).Copy())
|
||||
}
|
||||
78
pkg/types/telemetrytypes/logical_field.go
Normal file
78
pkg/types/telemetrytypes/logical_field.go
Normal file
@@ -0,0 +1,78 @@
|
||||
package telemetrytypes
|
||||
|
||||
// LogicalField is resolution output: one queryable field, addressed by the
|
||||
// spelling the request used, backed by the physical member keys that store it.
|
||||
//
|
||||
// The resolver expresses ambiguity ("possibly different fields sharing a
|
||||
// name") as a []*LogicalField — never inside one LogicalField. Within one
|
||||
// LogicalField, members are alternate physical spellings of the same field
|
||||
// (a semantic-convention family), ordered current-first; compilers merge
|
||||
// them into one expression with current-wins precedence. Across the slice,
|
||||
// compilers build one condition per LogicalField and combine per the
|
||||
// operator, exactly as they previously combined ambiguous keys.
|
||||
//
|
||||
// Members always has at least one entry. A non-family field has exactly
|
||||
// one. Members alias the metadata map entries and must not be mutated.
|
||||
type LogicalField struct {
|
||||
// Name is the requested spelling. It is the response identity: aliases,
|
||||
// series labels, and warnings use it, so responses echo the request.
|
||||
Name string
|
||||
|
||||
// The physical identity every member shares. Members with a different
|
||||
// signal, field context, or data type belong to different logical
|
||||
// fields by definition.
|
||||
Signal Signal
|
||||
FieldContext FieldContext
|
||||
FieldDataType FieldDataType
|
||||
|
||||
// Members are the physical keys that store this field, ordered
|
||||
// current-first. Each member carries its own physical facts
|
||||
// (Materialized, Evolutions, JSONPlan, ...), so per-member accessors
|
||||
// need no sibling information.
|
||||
Members []*TelemetryFieldKey
|
||||
}
|
||||
|
||||
// SingleLogicalField wraps one physical key as its own logical field.
|
||||
func SingleLogicalField(name string, key *TelemetryFieldKey) *LogicalField {
|
||||
return &LogicalField{
|
||||
Name: name,
|
||||
Signal: key.Signal,
|
||||
FieldContext: key.FieldContext,
|
||||
FieldDataType: key.FieldDataType,
|
||||
Members: []*TelemetryFieldKey{key},
|
||||
}
|
||||
}
|
||||
|
||||
// Single returns the only member. It is the accessor for signals whose
|
||||
// logical fields are always single-member (everything except traces today).
|
||||
func (l *LogicalField) Single() *TelemetryFieldKey {
|
||||
return l.Members[0]
|
||||
}
|
||||
|
||||
// IsFamily reports whether the field has more than one physical member.
|
||||
func (l *LogicalField) IsFamily() bool {
|
||||
return len(l.Members) > 1
|
||||
}
|
||||
|
||||
// String implements fmt.Stringer for warning messages.
|
||||
func (l *LogicalField) String() string {
|
||||
if len(l.Members) == 1 {
|
||||
return l.Members[0].String()
|
||||
}
|
||||
names := make([]string, 0, len(l.Members))
|
||||
for _, member := range l.Members {
|
||||
names = append(names, member.Name)
|
||||
}
|
||||
return l.Name + "(" + l.FieldContext.StringValue() + ", " + l.FieldDataType.StringValue() + ", members: " + joinNames(names) + ")"
|
||||
}
|
||||
|
||||
func joinNames(names []string) string {
|
||||
out := ""
|
||||
for i, name := range names {
|
||||
if i > 0 {
|
||||
out += ", "
|
||||
}
|
||||
out += name
|
||||
}
|
||||
return out
|
||||
}
|
||||
721
scripts/semconv/generate.go
Normal file
721
scripts/semconv/generate.go
Normal file
@@ -0,0 +1,721 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
const (
|
||||
kindAttribute = "attribute"
|
||||
kindMetric = "metric"
|
||||
)
|
||||
|
||||
type stringListFlag []string
|
||||
|
||||
func (f *stringListFlag) String() string { return strings.Join(*f, ",") }
|
||||
func (f *stringListFlag) Set(value string) error {
|
||||
*f = append(*f, value)
|
||||
return nil
|
||||
}
|
||||
|
||||
type schemaFile struct {
|
||||
FileFormat string `yaml:"file_format"`
|
||||
SchemaURL string `yaml:"schema_url"`
|
||||
Versions map[string]schemaVersion `yaml:"versions"`
|
||||
}
|
||||
|
||||
type schemaVersion struct {
|
||||
All changeSection `yaml:"all"`
|
||||
Resources changeSection `yaml:"resources"`
|
||||
Spans changeSection `yaml:"spans"`
|
||||
Logs changeSection `yaml:"logs"`
|
||||
Metrics changeSection `yaml:"metrics"`
|
||||
}
|
||||
|
||||
type changeSection struct {
|
||||
Changes []schemaChange `yaml:"changes"`
|
||||
}
|
||||
|
||||
type schemaChange struct {
|
||||
RenameAttributes *attributeRename `yaml:"rename_attributes"`
|
||||
RenameMetrics map[string]string `yaml:"rename_metrics"`
|
||||
}
|
||||
|
||||
type attributeRename struct {
|
||||
AttributeMap map[string]string `yaml:"attribute_map"`
|
||||
ApplyToMetrics []string `yaml:"apply_to_metrics"`
|
||||
}
|
||||
|
||||
type overlayFile struct {
|
||||
DefaultEnabled bool `yaml:"default_enabled"`
|
||||
// Families is keyed only by current name. One name cannot carry separate
|
||||
// policies for attribute and metric families; set kind explicitly whenever
|
||||
// a metric-name family is configured.
|
||||
Families map[string]overlayFamily `yaml:"families"`
|
||||
}
|
||||
|
||||
type overlayFamily struct {
|
||||
Enabled *bool `yaml:"enabled"`
|
||||
Kind string `yaml:"kind"`
|
||||
Old []string `yaml:"old"`
|
||||
AddOld []string `yaml:"add_old"`
|
||||
ExcludeOld []string `yaml:"exclude_old"`
|
||||
Contexts []string `yaml:"contexts"`
|
||||
Signals []string `yaml:"signals"`
|
||||
AddContexts []string `yaml:"add_contexts"`
|
||||
AddSignals []string `yaml:"add_signals"`
|
||||
ApplyToMetrics []string `yaml:"apply_to_metrics"`
|
||||
AddApplyToMetrics []string `yaml:"add_apply_to_metrics"`
|
||||
ValueMap map[string]string `yaml:"value_map"`
|
||||
}
|
||||
|
||||
type edge struct {
|
||||
old string
|
||||
current string
|
||||
kind string
|
||||
contexts []string
|
||||
signals []string
|
||||
allContexts bool
|
||||
allSignals bool
|
||||
applyToMetrics []string
|
||||
}
|
||||
|
||||
type graphKey struct{ kind, name string }
|
||||
|
||||
type generatedFamily struct {
|
||||
Current string
|
||||
Old []string
|
||||
Kind string
|
||||
Contexts []string
|
||||
Signals []string
|
||||
ApplyToMetrics []string
|
||||
ValueMap map[string]string
|
||||
}
|
||||
|
||||
func main() {
|
||||
root, err := findRepoRoot()
|
||||
if err != nil {
|
||||
fatal(err)
|
||||
}
|
||||
|
||||
var schemaPaths stringListFlag
|
||||
flag.Var(&schemaPaths, "schema", "schema source (repeatable)")
|
||||
overlayPath := flag.String("overlay", filepath.Join(root, "scripts/semconv/overlay.yaml"), "SigNoz overlay")
|
||||
goOutput := flag.String("go-out", filepath.Join(root, "pkg/semconv/families_gen.go"), "generated Go output")
|
||||
tsOutput := flag.String("ts-out", filepath.Join(root, "frontend/src/constants/generated/semconvFamilies.gen.ts"), "generated TypeScript output")
|
||||
check := flag.Bool("check", false, "fail if generated files are stale")
|
||||
flag.Parse()
|
||||
|
||||
if len(schemaPaths) == 0 {
|
||||
schemaPaths = append(schemaPaths, filepath.Join(root, "scripts/semconv/schema-1.42.0.yaml"))
|
||||
}
|
||||
|
||||
families, err := generate(schemaPaths, *overlayPath)
|
||||
if err != nil {
|
||||
fatal(err)
|
||||
}
|
||||
goBytes, err := renderGo(families)
|
||||
if err != nil {
|
||||
fatal(err)
|
||||
}
|
||||
tsBytes := renderTypeScript(families)
|
||||
|
||||
if *check {
|
||||
if err := checkFile(*goOutput, goBytes); err != nil {
|
||||
fatal(err)
|
||||
}
|
||||
if err := checkFile(*tsOutput, tsBytes); err != nil {
|
||||
fatal(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if err := os.WriteFile(*goOutput, goBytes, 0o644); err != nil {
|
||||
fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(*tsOutput, tsBytes, 0o644); err != nil {
|
||||
fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func fatal(err error) {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
func findRepoRoot() (string, error) {
|
||||
dir, err := os.Getwd()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
for {
|
||||
if _, err := os.Stat(filepath.Join(dir, "go.mod")); err == nil {
|
||||
return dir, nil
|
||||
}
|
||||
parent := filepath.Dir(dir)
|
||||
if parent == dir {
|
||||
return "", errors.New("could not find repository root")
|
||||
}
|
||||
dir = parent
|
||||
}
|
||||
}
|
||||
|
||||
func generate(schemaPaths []string, overlayPath string) ([]generatedFamily, error) {
|
||||
var schemas []schemaFile
|
||||
for _, path := range schemaPaths {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read schema %s: %w", path, err)
|
||||
}
|
||||
var schema schemaFile
|
||||
if err := decodeKnownFields(data, &schema); err != nil {
|
||||
return nil, fmt.Errorf("parse schema %s: %w", path, err)
|
||||
}
|
||||
schemas = append(schemas, schema)
|
||||
}
|
||||
|
||||
overlayData, err := os.ReadFile(overlayPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read overlay: %w", err)
|
||||
}
|
||||
var overlay overlayFile
|
||||
if err := decodeKnownFields(overlayData, &overlay); err != nil {
|
||||
return nil, fmt.Errorf("parse overlay: %w", err)
|
||||
}
|
||||
|
||||
return buildFamilies(schemas, overlay)
|
||||
}
|
||||
|
||||
func decodeKnownFields(data []byte, target any) error {
|
||||
decoder := yaml.NewDecoder(bytes.NewReader(data))
|
||||
decoder.KnownFields(true)
|
||||
return decoder.Decode(target)
|
||||
}
|
||||
|
||||
func collectEdges(schemas []schemaFile) ([]edge, error) {
|
||||
var edges []edge
|
||||
for _, schema := range schemas {
|
||||
versions := make([]string, 0, len(schema.Versions))
|
||||
versionParts := make(map[string][3]int, len(schema.Versions))
|
||||
for version := range schema.Versions {
|
||||
parts, err := parseSchemaVersion(version)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
versions = append(versions, version)
|
||||
versionParts[version] = parts
|
||||
}
|
||||
sort.Slice(versions, func(i, j int) bool {
|
||||
return compareVersionParts(versionParts[versions[i]], versionParts[versions[j]]) < 0
|
||||
})
|
||||
for _, versionName := range versions {
|
||||
version := schema.Versions[versionName]
|
||||
var versionEdges []edge
|
||||
sections := []struct {
|
||||
name string
|
||||
section changeSection
|
||||
}{
|
||||
{name: "all", section: version.All},
|
||||
{name: "resources", section: version.Resources},
|
||||
{name: "spans", section: version.Spans},
|
||||
{name: "logs", section: version.Logs},
|
||||
{name: "metrics", section: version.Metrics},
|
||||
}
|
||||
for _, scoped := range sections {
|
||||
contexts, signals, allContexts, allSignals, err := scopeForSection(scoped.name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, change := range scoped.section.Changes {
|
||||
if change.RenameAttributes != nil {
|
||||
for _, old := range sortedMapKeys(change.RenameAttributes.AttributeMap) {
|
||||
versionEdges = append(versionEdges, edge{
|
||||
old: old, current: change.RenameAttributes.AttributeMap[old], kind: kindAttribute,
|
||||
contexts: contexts, signals: signals,
|
||||
allContexts: allContexts, allSignals: allSignals,
|
||||
applyToMetrics: change.RenameAttributes.ApplyToMetrics,
|
||||
})
|
||||
}
|
||||
}
|
||||
for _, old := range sortedMapKeys(change.RenameMetrics) {
|
||||
versionEdges = append(versionEdges, edge{
|
||||
old: old, current: change.RenameMetrics[old], kind: kindMetric,
|
||||
contexts: []string{"metric"}, signals: []string{"metrics"},
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := rejectSameVersionChains(versionName, versionEdges); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
edges = append(edges, versionEdges...)
|
||||
}
|
||||
}
|
||||
return edges, nil
|
||||
}
|
||||
|
||||
func rejectSameVersionChains(version string, edges []edge) error {
|
||||
oldNames := make(map[graphKey]struct{}, len(edges))
|
||||
for _, item := range edges {
|
||||
oldNames[graphKey{kind: item.kind, name: item.old}] = struct{}{}
|
||||
}
|
||||
for _, item := range edges {
|
||||
if _, ok := oldNames[graphKey{kind: item.kind, name: item.current}]; ok {
|
||||
return fmt.Errorf(
|
||||
"schema version %q contains a same-version %s rename chain through %q",
|
||||
version,
|
||||
item.kind,
|
||||
item.current,
|
||||
)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func parseSchemaVersion(version string) ([3]int, error) {
|
||||
parts := strings.Split(version, ".")
|
||||
if len(parts) != 3 {
|
||||
return [3]int{}, fmt.Errorf("schema version %q must contain major, minor, and patch numbers", version)
|
||||
}
|
||||
|
||||
var parsed [3]int
|
||||
for i, part := range parts {
|
||||
value, err := strconv.Atoi(part)
|
||||
if err != nil || value < 0 {
|
||||
return [3]int{}, fmt.Errorf("schema version %q contains invalid numeric component %q", version, part)
|
||||
}
|
||||
parsed[i] = value
|
||||
}
|
||||
return parsed, nil
|
||||
}
|
||||
|
||||
func compareVersionParts(left, right [3]int) int {
|
||||
for i := range left {
|
||||
if left[i] < right[i] {
|
||||
return -1
|
||||
}
|
||||
if left[i] > right[i] {
|
||||
return 1
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func scopeForSection(section string) (contexts, signals []string, allContexts, allSignals bool, err error) {
|
||||
switch section {
|
||||
case "all":
|
||||
return nil, nil, true, true, nil
|
||||
case "resources":
|
||||
return []string{"resource"}, nil, false, true, nil
|
||||
case "spans":
|
||||
return []string{"attribute"}, []string{"traces"}, false, false, nil
|
||||
case "logs":
|
||||
return []string{"attribute"}, []string{"logs"}, false, false, nil
|
||||
case "metrics":
|
||||
return []string{"attribute"}, []string{"metrics"}, false, false, nil
|
||||
default:
|
||||
return nil, nil, false, false, fmt.Errorf("unsupported schema section %q", section)
|
||||
}
|
||||
}
|
||||
|
||||
func buildFamilies(schemas []schemaFile, overlay overlayFile) ([]generatedFamily, error) {
|
||||
edges, err := collectEdges(schemas)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
next := make(map[graphKey]string)
|
||||
for _, item := range edges {
|
||||
key := graphKey{kind: item.kind, name: item.old}
|
||||
if existing, ok := next[key]; ok && existing == item.current {
|
||||
// Repeated entries are common in chained schema histories. Treat an
|
||||
// identical edge as a no-op so it cannot sever a later edge in the
|
||||
// same chain (A -> B, B -> C, then a repeated A -> B).
|
||||
continue
|
||||
}
|
||||
// Schema history occasionally repeats an old name with a newer direct
|
||||
// destination or rolls a rename back. Edges are collected
|
||||
// oldest-to-newest, so the latest published current name must be a root.
|
||||
delete(next, graphKey{kind: item.kind, name: item.current})
|
||||
next[key] = item.current
|
||||
}
|
||||
|
||||
type familyState struct {
|
||||
family generatedFamily
|
||||
distance map[string]int
|
||||
allContexts bool
|
||||
allSignals bool
|
||||
}
|
||||
states := map[graphKey]*familyState{}
|
||||
for _, item := range edges {
|
||||
root, distance, err := rootFor(next, item.kind, item.old)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
key := graphKey{kind: item.kind, name: root}
|
||||
state := states[key]
|
||||
if state == nil {
|
||||
state = &familyState{
|
||||
family: generatedFamily{Current: root, Kind: item.kind},
|
||||
distance: map[string]int{},
|
||||
}
|
||||
states[key] = state
|
||||
}
|
||||
if prior, ok := state.distance[item.old]; !ok || distance < prior {
|
||||
state.distance[item.old] = distance
|
||||
}
|
||||
state.allContexts = state.allContexts || item.allContexts
|
||||
state.allSignals = state.allSignals || item.allSignals
|
||||
state.family.Contexts = appendUnique(state.family.Contexts, item.contexts...)
|
||||
state.family.Signals = appendUnique(state.family.Signals, item.signals...)
|
||||
state.family.ApplyToMetrics = appendUnique(state.family.ApplyToMetrics, item.applyToMetrics...)
|
||||
}
|
||||
|
||||
for _, state := range states {
|
||||
for old := range state.distance {
|
||||
if old != state.family.Current {
|
||||
state.family.Old = append(state.family.Old, old)
|
||||
}
|
||||
}
|
||||
sort.Slice(state.family.Old, func(i, j int) bool {
|
||||
left, right := state.family.Old[i], state.family.Old[j]
|
||||
if state.distance[left] != state.distance[right] {
|
||||
return state.distance[left] < state.distance[right]
|
||||
}
|
||||
return left < right
|
||||
})
|
||||
if state.allContexts {
|
||||
state.family.Contexts = nil
|
||||
} else {
|
||||
sort.Strings(state.family.Contexts)
|
||||
}
|
||||
if state.allSignals {
|
||||
state.family.Signals = nil
|
||||
} else {
|
||||
sort.Strings(state.family.Signals)
|
||||
}
|
||||
sort.Strings(state.family.ApplyToMetrics)
|
||||
}
|
||||
|
||||
for _, current := range sortedMapKeys(overlay.Families) {
|
||||
policy := overlay.Families[current]
|
||||
kind, err := normalizedOverlayKind(current, policy)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
policy.Kind = kind
|
||||
overlay.Families[current] = policy
|
||||
key := graphKey{kind: kind, name: current}
|
||||
state := states[key]
|
||||
if state == nil {
|
||||
if len(policy.Old) == 0 {
|
||||
return nil, fmt.Errorf(
|
||||
"overlay family %q with kind %q is absent from schemas and has no old members",
|
||||
current,
|
||||
kind,
|
||||
)
|
||||
}
|
||||
state = &familyState{
|
||||
family: generatedFamily{Current: current, Kind: kind, Old: append([]string(nil), policy.Old...)},
|
||||
distance: map[string]int{},
|
||||
}
|
||||
states[key] = state
|
||||
}
|
||||
applyOverlay(&state.family, policy)
|
||||
}
|
||||
|
||||
var result []generatedFamily
|
||||
for key, state := range states {
|
||||
policy, hasPolicy := overlay.Families[key.name]
|
||||
enabled := overlay.DefaultEnabled
|
||||
if hasPolicy && policy.Kind != key.kind {
|
||||
hasPolicy = false
|
||||
}
|
||||
if hasPolicy && policy.Enabled != nil {
|
||||
enabled = *policy.Enabled
|
||||
}
|
||||
if !enabled {
|
||||
continue
|
||||
}
|
||||
if len(state.family.Old) == 0 {
|
||||
return nil, fmt.Errorf(
|
||||
"enabled family %q with kind %q has no old members",
|
||||
state.family.Current,
|
||||
state.family.Kind,
|
||||
)
|
||||
}
|
||||
sort.Strings(state.family.Contexts)
|
||||
sort.Strings(state.family.Signals)
|
||||
sort.Strings(state.family.ApplyToMetrics)
|
||||
result = append(result, state.family)
|
||||
}
|
||||
|
||||
sort.Slice(result, func(i, j int) bool {
|
||||
if result[i].Current != result[j].Current {
|
||||
return result[i].Current < result[j].Current
|
||||
}
|
||||
return result[i].Kind < result[j].Kind
|
||||
})
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func rootFor(next map[graphKey]string, kind, name string) (string, int, error) {
|
||||
seen := map[string]bool{}
|
||||
distance := 0
|
||||
for {
|
||||
if seen[name] {
|
||||
return "", 0, fmt.Errorf("rename cycle for %s %q", kind, name)
|
||||
}
|
||||
seen[name] = true
|
||||
current, ok := next[graphKey{kind: kind, name: name}]
|
||||
if !ok {
|
||||
return name, distance, nil
|
||||
}
|
||||
name = current
|
||||
distance++
|
||||
}
|
||||
}
|
||||
|
||||
func normalizedOverlayKind(current string, policy overlayFamily) (string, error) {
|
||||
kind := policy.Kind
|
||||
if kind == "" {
|
||||
kind = kindAttribute
|
||||
}
|
||||
if kind != kindAttribute && kind != kindMetric {
|
||||
return "", fmt.Errorf("overlay family %q has unsupported kind %q", current, kind)
|
||||
}
|
||||
return kind, nil
|
||||
}
|
||||
|
||||
func applyOverlay(family *generatedFamily, policy overlayFamily) {
|
||||
if policy.Kind != "" {
|
||||
family.Kind = policy.Kind
|
||||
}
|
||||
if policy.Old != nil {
|
||||
family.Old = append([]string(nil), policy.Old...)
|
||||
}
|
||||
family.Old = appendUnique(family.Old, policy.AddOld...)
|
||||
if len(policy.ExcludeOld) > 0 {
|
||||
excluded := make(map[string]bool, len(policy.ExcludeOld))
|
||||
for _, old := range policy.ExcludeOld {
|
||||
excluded[old] = true
|
||||
}
|
||||
family.Old = deleteMatching(family.Old, excluded)
|
||||
}
|
||||
if policy.Contexts != nil {
|
||||
family.Contexts = append([]string(nil), policy.Contexts...)
|
||||
}
|
||||
if policy.Signals != nil {
|
||||
family.Signals = append([]string(nil), policy.Signals...)
|
||||
}
|
||||
family.Contexts = appendUnique(family.Contexts, policy.AddContexts...)
|
||||
family.Signals = appendUnique(family.Signals, policy.AddSignals...)
|
||||
if policy.ApplyToMetrics != nil {
|
||||
family.ApplyToMetrics = append([]string(nil), policy.ApplyToMetrics...)
|
||||
}
|
||||
family.ApplyToMetrics = appendUnique(family.ApplyToMetrics, policy.AddApplyToMetrics...)
|
||||
if policy.ValueMap != nil {
|
||||
family.ValueMap = make(map[string]string, len(policy.ValueMap))
|
||||
for old, current := range policy.ValueMap {
|
||||
family.ValueMap[old] = current
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func appendUnique(values []string, additions ...string) []string {
|
||||
seen := make(map[string]bool, len(values)+len(additions))
|
||||
for _, value := range values {
|
||||
seen[value] = true
|
||||
}
|
||||
for _, value := range additions {
|
||||
if value == "" || seen[value] {
|
||||
continue
|
||||
}
|
||||
seen[value] = true
|
||||
values = append(values, value)
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
func deleteMatching(values []string, excluded map[string]bool) []string {
|
||||
result := values[:0]
|
||||
for _, value := range values {
|
||||
if !excluded[value] {
|
||||
result = append(result, value)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func renderGo(families []generatedFamily) ([]byte, error) {
|
||||
var out bytes.Buffer
|
||||
out.WriteString("// Code generated by scripts/semconv. DO NOT EDIT.\n\n")
|
||||
out.WriteString("package semconv\n\n")
|
||||
needsTelemetryTypes := false
|
||||
for _, family := range families {
|
||||
if len(family.Contexts) > 0 || len(family.Signals) > 0 {
|
||||
needsTelemetryTypes = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if needsTelemetryTypes {
|
||||
out.WriteString("import \"github.com/SigNoz/signoz/pkg/types/telemetrytypes\"\n\n")
|
||||
}
|
||||
out.WriteString("var families = []Family{\n")
|
||||
for _, family := range families {
|
||||
contexts, err := goFieldContextSlice(family.Contexts)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("render family %q: %w", family.Current, err)
|
||||
}
|
||||
signals, err := goSignalSlice(family.Signals)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("render family %q: %w", family.Current, err)
|
||||
}
|
||||
out.WriteString("\t{\n")
|
||||
fmt.Fprintf(&out, "\t\tCurrent: %s,\n", strconv.Quote(family.Current))
|
||||
fmt.Fprintf(&out, "\t\tOld: %s,\n", goStringSlice(family.Old))
|
||||
if family.Kind == kindMetric {
|
||||
out.WriteString("\t\tKind: KindMetric,\n")
|
||||
} else {
|
||||
out.WriteString("\t\tKind: KindAttribute,\n")
|
||||
}
|
||||
fmt.Fprintf(&out, "\t\tContexts: %s,\n", contexts)
|
||||
fmt.Fprintf(&out, "\t\tSignals: %s,\n", signals)
|
||||
fmt.Fprintf(&out, "\t\tApplyToMetrics: %s,\n", goStringSlice(family.ApplyToMetrics))
|
||||
if len(family.ValueMap) > 0 {
|
||||
out.WriteString("\t\tValueMap: map[string]string{\n")
|
||||
keys := sortedMapKeys(family.ValueMap)
|
||||
for _, key := range keys {
|
||||
fmt.Fprintf(&out, "\t\t\t%s: %s,\n", strconv.Quote(key), strconv.Quote(family.ValueMap[key]))
|
||||
}
|
||||
out.WriteString("\t\t},\n")
|
||||
}
|
||||
out.WriteString("\t},\n")
|
||||
}
|
||||
out.WriteString("}\n")
|
||||
return format.Source(out.Bytes())
|
||||
}
|
||||
|
||||
func goStringSlice(values []string) string {
|
||||
if len(values) == 0 {
|
||||
return "nil"
|
||||
}
|
||||
quoted := make([]string, len(values))
|
||||
for i, value := range values {
|
||||
quoted[i] = strconv.Quote(value)
|
||||
}
|
||||
return "[]string{" + strings.Join(quoted, ", ") + "}"
|
||||
}
|
||||
|
||||
func goFieldContextSlice(values []string) (string, error) {
|
||||
if len(values) == 0 {
|
||||
return "nil", nil
|
||||
}
|
||||
constants := make([]string, len(values))
|
||||
for i, value := range values {
|
||||
switch value {
|
||||
case "metric":
|
||||
constants[i] = "telemetrytypes.FieldContextMetric"
|
||||
case "resource":
|
||||
constants[i] = "telemetrytypes.FieldContextResource"
|
||||
case "attribute":
|
||||
constants[i] = "telemetrytypes.FieldContextAttribute"
|
||||
default:
|
||||
return "", fmt.Errorf("unsupported field context %q", value)
|
||||
}
|
||||
}
|
||||
return "[]telemetrytypes.FieldContext{" + strings.Join(constants, ", ") + "}", nil
|
||||
}
|
||||
|
||||
func goSignalSlice(values []string) (string, error) {
|
||||
if len(values) == 0 {
|
||||
return "nil", nil
|
||||
}
|
||||
constants := make([]string, len(values))
|
||||
for i, value := range values {
|
||||
switch value {
|
||||
case "traces":
|
||||
constants[i] = "telemetrytypes.SignalTraces"
|
||||
case "logs":
|
||||
constants[i] = "telemetrytypes.SignalLogs"
|
||||
case "metrics":
|
||||
constants[i] = "telemetrytypes.SignalMetrics"
|
||||
default:
|
||||
return "", fmt.Errorf("unsupported signal %q", value)
|
||||
}
|
||||
}
|
||||
return "[]telemetrytypes.Signal{" + strings.Join(constants, ", ") + "}", nil
|
||||
}
|
||||
|
||||
func renderTypeScript(families []generatedFamily) []byte {
|
||||
var out bytes.Buffer
|
||||
out.WriteString("// Code generated by scripts/semconv. DO NOT EDIT.\n\n")
|
||||
out.WriteString("export type SemconvFamily = {\n")
|
||||
out.WriteString("\treadonly current: string;\n\treadonly old: readonly string[];\n")
|
||||
out.WriteString("\treadonly kind: 'attribute' | 'metric';\n")
|
||||
out.WriteString("\treadonly contexts: readonly string[];\n\treadonly signals: readonly string[];\n")
|
||||
out.WriteString("\treadonly applyToMetrics: readonly string[];\n")
|
||||
out.WriteString("\treadonly valueMap: Readonly<Record<string, string>>;\n};\n\n")
|
||||
out.WriteString("export const SEMCONV_FAMILIES: readonly SemconvFamily[] = [\n")
|
||||
for _, family := range families {
|
||||
out.WriteString("\t{\n")
|
||||
fmt.Fprintf(&out, "\t\tcurrent: %s,\n", tsString(family.Current))
|
||||
fmt.Fprintf(&out, "\t\told: %s,\n", tsStringSlice(family.Old))
|
||||
fmt.Fprintf(&out, "\t\tkind: %s,\n", tsString(family.Kind))
|
||||
fmt.Fprintf(&out, "\t\tcontexts: %s,\n", tsStringSlice(family.Contexts))
|
||||
fmt.Fprintf(&out, "\t\tsignals: %s,\n", tsStringSlice(family.Signals))
|
||||
fmt.Fprintf(&out, "\t\tapplyToMetrics: %s,\n", tsStringSlice(family.ApplyToMetrics))
|
||||
out.WriteString("\t\tvalueMap: {")
|
||||
keys := sortedMapKeys(family.ValueMap)
|
||||
for i, key := range keys {
|
||||
if i > 0 {
|
||||
out.WriteString(", ")
|
||||
}
|
||||
fmt.Fprintf(&out, "%s: %s", tsString(key), tsString(family.ValueMap[key]))
|
||||
}
|
||||
out.WriteString("},\n\t},\n")
|
||||
}
|
||||
out.WriteString("] as const;\n")
|
||||
return out.Bytes()
|
||||
}
|
||||
|
||||
func tsString(value string) string {
|
||||
quoted := strconv.Quote(value)
|
||||
return "'" + strings.ReplaceAll(quoted[1:len(quoted)-1], "'", `\'`) + "'"
|
||||
}
|
||||
func tsStringSlice(values []string) string {
|
||||
quoted := make([]string, len(values))
|
||||
for i, value := range values {
|
||||
quoted[i] = tsString(value)
|
||||
}
|
||||
return "[" + strings.Join(quoted, ", ") + "]"
|
||||
}
|
||||
|
||||
func sortedMapKeys[T any](values map[string]T) []string {
|
||||
keys := make([]string, 0, len(values))
|
||||
for key := range values {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
return keys
|
||||
}
|
||||
|
||||
func checkFile(path string, expected []byte) error {
|
||||
actual, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("generated file %s is missing: run go run ./scripts/semconv", path)
|
||||
}
|
||||
if !bytes.Equal(actual, expected) {
|
||||
return fmt.Errorf("generated file %s is stale: run go run ./scripts/semconv", path)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
376
scripts/semconv/generate_test.go
Normal file
376
scripts/semconv/generate_test.go
Normal file
@@ -0,0 +1,376 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestSchemaDecoderRejectsUnsupportedSection(t *testing.T) {
|
||||
var schema schemaFile
|
||||
err := decodeKnownFields([]byte(`
|
||||
versions:
|
||||
1.0.0:
|
||||
span_events:
|
||||
changes:
|
||||
- rename_events:
|
||||
event_map:
|
||||
old: current
|
||||
`), &schema)
|
||||
|
||||
assert.ErrorContains(t, err, "field span_events not found", "unsupported schema sections must fail generation")
|
||||
}
|
||||
|
||||
func TestBuildFamiliesRejectsMalformedSchemaVersion(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
latest:
|
||||
spans:
|
||||
changes: []
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
_, err := buildFamilies([]schemaFile{schema}, overlayFile{})
|
||||
assert.ErrorContains(t, err, `schema version "latest"`, "malformed versions must not be silently reordered")
|
||||
}
|
||||
|
||||
func TestBuildFamiliesResolvesRenameChain(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
4.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
a: b
|
||||
3.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
b: c
|
||||
x: c
|
||||
2.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
a: b
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
a: b
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
enabled := true
|
||||
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
|
||||
"c": {Enabled: &enabled},
|
||||
}})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []generatedFamily{{
|
||||
Current: "c",
|
||||
Old: []string{"b", "x", "a"},
|
||||
Kind: kindAttribute,
|
||||
Contexts: []string{"attribute"},
|
||||
Signals: []string{"traces"},
|
||||
}}, families, "predecessors should be ordered by distance and then name")
|
||||
}
|
||||
|
||||
func TestBuildFamiliesMapsSchemaSectionsToScopes(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
1.0.0:
|
||||
all:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
all.old: all.current
|
||||
resources:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
resource.old: resource.current
|
||||
logs:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
log.old: log.current
|
||||
metrics:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
state: cpu.mode
|
||||
apply_to_metrics: [system.cpu.time]
|
||||
- rename_metrics:
|
||||
old.metric: current.metric
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
enabled := true
|
||||
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
|
||||
"all.current": {Enabled: &enabled},
|
||||
"resource.current": {Enabled: &enabled},
|
||||
"log.current": {Enabled: &enabled},
|
||||
"cpu.mode": {Enabled: &enabled},
|
||||
"current.metric": {Enabled: &enabled, Kind: kindMetric},
|
||||
}})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []generatedFamily{
|
||||
{
|
||||
Current: "all.current", Old: []string{"all.old"}, Kind: kindAttribute,
|
||||
Contexts: nil, Signals: nil,
|
||||
},
|
||||
{
|
||||
Current: "cpu.mode", Old: []string{"state"}, Kind: kindAttribute,
|
||||
Contexts: []string{"attribute"}, Signals: []string{"metrics"},
|
||||
ApplyToMetrics: []string{"system.cpu.time"},
|
||||
},
|
||||
{
|
||||
Current: "current.metric", Old: []string{"old.metric"}, Kind: kindMetric,
|
||||
Contexts: []string{"metric"}, Signals: []string{"metrics"},
|
||||
},
|
||||
{
|
||||
Current: "log.current", Old: []string{"log.old"}, Kind: kindAttribute,
|
||||
Contexts: []string{"attribute"}, Signals: []string{"logs"},
|
||||
},
|
||||
{
|
||||
Current: "resource.current", Old: []string{"resource.old"}, Kind: kindAttribute,
|
||||
Contexts: []string{"resource"},
|
||||
},
|
||||
}, families, "schema sections should produce their documented signal and context scopes")
|
||||
}
|
||||
|
||||
func TestOverlayAddsFamilyWithoutSchemaHistory(t *testing.T) {
|
||||
enabled := true
|
||||
families, err := buildFamilies(nil, overlayFile{Families: map[string]overlayFamily{
|
||||
"added.current": {
|
||||
Enabled: &enabled,
|
||||
Old: []string{"added.old"},
|
||||
Contexts: []string{"resource"},
|
||||
Signals: []string{"traces"},
|
||||
},
|
||||
}})
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []generatedFamily{{
|
||||
Current: "added.current",
|
||||
Old: []string{"added.old"},
|
||||
Kind: kindAttribute,
|
||||
Contexts: []string{"resource"},
|
||||
Signals: []string{"traces"},
|
||||
}}, families, "an explicit overlay family should not require schema history")
|
||||
}
|
||||
|
||||
func TestOverlayOverridesGeneratedFamily(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
1.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
old: current
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
enabled := true
|
||||
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
|
||||
"current": {
|
||||
Enabled: &enabled,
|
||||
AddOld: []string{"older"},
|
||||
ExcludeOld: []string{"old"},
|
||||
AddContexts: []string{"resource"},
|
||||
AddSignals: []string{"logs"},
|
||||
ValueMap: map[string]string{"legacy": "current"},
|
||||
},
|
||||
}})
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []generatedFamily{{
|
||||
Current: "current",
|
||||
Old: []string{"older"},
|
||||
Kind: kindAttribute,
|
||||
Contexts: []string{"attribute", "resource"},
|
||||
Signals: []string{"logs", "traces"},
|
||||
ValueMap: map[string]string{"legacy": "current"},
|
||||
}}, families, "overlay additions and exclusions should be applied to the generated family")
|
||||
}
|
||||
|
||||
func TestOverlayDisablesFamilyWhenDefaultIsEnabled(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
1.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
old: current
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
disabled := false
|
||||
families, err := buildFamilies([]schemaFile{schema}, overlayFile{
|
||||
DefaultEnabled: true,
|
||||
Families: map[string]overlayFamily{
|
||||
"current": {Enabled: &disabled},
|
||||
},
|
||||
})
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Empty(t, families, "an explicitly disabled family must override default_enabled")
|
||||
}
|
||||
|
||||
func TestRenderGoIsDeterministic(t *testing.T) {
|
||||
families := []generatedFamily{{
|
||||
Current: "current", Old: []string{"old"}, Kind: kindAttribute,
|
||||
ValueMap: map[string]string{"b": "2", "a": "1"},
|
||||
}}
|
||||
|
||||
first, err := renderGo(families)
|
||||
require.NoError(t, err)
|
||||
second, err := renderGo(families)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, first, second, "Go generation must not depend on map iteration order")
|
||||
}
|
||||
|
||||
func TestRenderGoUsesCanonicalTelemetryTypes(t *testing.T) {
|
||||
families := []generatedFamily{{
|
||||
Current: "current", Old: []string{"old"}, Kind: kindAttribute,
|
||||
Contexts: []string{"resource"}, Signals: []string{"traces"},
|
||||
}}
|
||||
|
||||
output, err := renderGo(families)
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(output), "telemetrytypes.FieldContextResource", "generated contexts should use telemetrytypes")
|
||||
assert.Contains(t, string(output), "telemetrytypes.SignalTraces", "generated signals should use telemetrytypes")
|
||||
}
|
||||
|
||||
func TestRenderTypeScriptIsDeterministic(t *testing.T) {
|
||||
families := []generatedFamily{{
|
||||
Current: "current", Old: []string{"old"}, Kind: kindAttribute,
|
||||
ValueMap: map[string]string{"b": "2", "a": "1"},
|
||||
}}
|
||||
|
||||
assert.Equal(t, renderTypeScript(families), renderTypeScript(families), "TypeScript generation must not depend on map iteration order")
|
||||
}
|
||||
|
||||
func TestBuildFamiliesHandlesRenameRollback(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
2.0.0:
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
temporary: original
|
||||
1.0.0:
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
original: temporary
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
enabled := true
|
||||
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
|
||||
"original": {Enabled: &enabled, Kind: kindMetric},
|
||||
}})
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []generatedFamily{{
|
||||
Current: "original",
|
||||
Old: []string{"temporary"},
|
||||
Kind: kindMetric,
|
||||
Contexts: []string{"metric"},
|
||||
Signals: []string{"metrics"},
|
||||
}}, families, "the latest rollback destination should remain the family root")
|
||||
}
|
||||
|
||||
func TestBuildFamiliesRejectsSameVersionRenameChain(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
1.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
x: y
|
||||
y: z
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
_, err := buildFamilies([]schemaFile{schema}, overlayFile{})
|
||||
assert.ErrorContains(t, err, `same-version attribute rename chain through "y"`, "order-sensitive same-version chains must be rejected")
|
||||
}
|
||||
|
||||
func TestBuildFamiliesRejectsOverlayFamilyWithoutHistory(t *testing.T) {
|
||||
enabled := true
|
||||
_, err := buildFamilies(nil, overlayFile{Families: map[string]overlayFamily{
|
||||
"missing": {Enabled: &enabled},
|
||||
}})
|
||||
|
||||
assert.ErrorContains(t, err, `overlay family "missing" with kind "attribute" is absent`, "an overlay cannot invent a family without old members")
|
||||
}
|
||||
|
||||
func TestBuildFamiliesRejectsEnabledFamilyWithoutOldMembers(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
1.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
old: current
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
enabled := true
|
||||
_, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
|
||||
"current": {Enabled: &enabled, ExcludeOld: []string{"old"}},
|
||||
}})
|
||||
assert.ErrorContains(t, err, `enabled family "current" with kind "attribute" has no old members`, "exclude_old cannot empty an enabled family")
|
||||
}
|
||||
|
||||
func TestOverlayKindDefaultsToAttribute(t *testing.T) {
|
||||
var schema schemaFile
|
||||
require.NoError(t, decodeKnownFields([]byte(`
|
||||
versions:
|
||||
1.0.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
attribute.old: shared.current
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
metric.old: shared.current
|
||||
`), &schema), "test schema must decode")
|
||||
|
||||
enabled := true
|
||||
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
|
||||
"shared.current": {Enabled: &enabled},
|
||||
}})
|
||||
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []generatedFamily{{
|
||||
Current: "shared.current",
|
||||
Old: []string{"attribute.old"},
|
||||
Kind: kindAttribute,
|
||||
Contexts: []string{"attribute"},
|
||||
Signals: []string{"traces"},
|
||||
}}, families, "a kind-less overlay policy should affect only the attribute family")
|
||||
}
|
||||
|
||||
func TestCheckFileReportsStaleOutput(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "generated.go")
|
||||
require.NoError(t, os.WriteFile(path, []byte("old"), 0o600), "test output must be writable")
|
||||
|
||||
assert.ErrorContains(t, checkFile(path, []byte("new")), "is stale", "check mode must reject stale generated output")
|
||||
}
|
||||
|
||||
func TestTypeScriptStringEscapesControlCharacters(t *testing.T) {
|
||||
assert.Equal(t, `'line\n\t\x01\'\\end'`, tsString("line\n\t\x01'\\end"), "generated TypeScript strings must remain valid literals")
|
||||
}
|
||||
11
scripts/semconv/overlay.yaml
Normal file
11
scripts/semconv/overlay.yaml
Normal file
@@ -0,0 +1,11 @@
|
||||
# SigNoz semantic-convention rollout policy.
|
||||
#
|
||||
# Families are keyed by their current OpenTelemetry name. Schema-derived
|
||||
# families are disabled by default so rollout remains explicit and reversible.
|
||||
default_enabled: false
|
||||
|
||||
families:
|
||||
deployment.environment.name:
|
||||
enabled: true
|
||||
db.system.name:
|
||||
enabled: true
|
||||
760
scripts/semconv/schema-1.42.0.yaml
Normal file
760
scripts/semconv/schema-1.42.0.yaml
Normal file
@@ -0,0 +1,760 @@
|
||||
|
||||
|
||||
file_format: 1.1.0
|
||||
schema_url: https://opentelemetry.io/schemas/1.42.0
|
||||
versions:
|
||||
1.42.0:
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
v8js.memory.heap.limit: v8js.memory.heap.space.size
|
||||
1.41.1:
|
||||
1.41.0:
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
k8s.container.cpu.limit: k8s.container.cpu.limit.desired
|
||||
k8s.container.cpu.limit_utilization: k8s.container.cpu.limit.utilization
|
||||
k8s.container.cpu.request: k8s.container.cpu.request.desired
|
||||
k8s.container.cpu.request_utilization: k8s.container.cpu.request.utilization
|
||||
k8s.container.memory.limit: k8s.container.memory.limit.desired
|
||||
k8s.container.memory.request: k8s.container.memory.request.desired
|
||||
1.40.0:
|
||||
all:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
feature_flag.evaluation.error.message: feature_flag.error.message
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
system.memory.shared: system.memory.linux.shared
|
||||
1.39.0:
|
||||
all:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
linux.memory.slab.state: system.memory.linux.slab.state
|
||||
peer.service: service.peer.name
|
||||
rpc.connect_rpc.error_code: rpc.response.status_code
|
||||
rpc.connect_rpc.request.metadata: rpc.request.metadata
|
||||
rpc.connect_rpc.response.metadata: rpc.response.metadata
|
||||
rpc.grpc.request.metadata: rpc.request.metadata
|
||||
rpc.grpc.response.metadata: rpc.response.metadata
|
||||
rpc.jsonrpc.request_id: jsonrpc.request.id
|
||||
rpc.jsonrpc.version: jsonrpc.protocol.version
|
||||
rpc.system: rpc.system.name
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
process.open_file_descriptor.count: process.unix.file_descriptor.count
|
||||
system.linux.memory.available: system.memory.linux.available
|
||||
system.linux.memory.slab.usage: system.memory.linux.slab.usage
|
||||
1.38.0:
|
||||
all:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
process.context_switch_type: process.context_switch.type
|
||||
process.paging.fault_type: system.paging.fault.type
|
||||
system.cpu.logical_number: cpu.logical_number
|
||||
system.paging.type: system.paging.fault.type
|
||||
system.process.status: process.state
|
||||
system.processes.status: process.state
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
k8s.cronjob.active_jobs: k8s.cronjob.job.active
|
||||
k8s.daemonset.current_scheduled_nodes: k8s.daemonset.node.current_scheduled
|
||||
k8s.daemonset.desired_scheduled_nodes: k8s.daemonset.node.desired_scheduled
|
||||
k8s.daemonset.misscheduled_nodes: k8s.daemonset.node.misscheduled
|
||||
k8s.daemonset.ready_nodes: k8s.daemonset.node.ready
|
||||
k8s.deployment.available_pods: k8s.deployment.pod.available
|
||||
k8s.deployment.desired_pods: k8s.deployment.pod.desired
|
||||
k8s.hpa.current_pods: k8s.hpa.pod.current
|
||||
k8s.hpa.desired_pods: k8s.hpa.pod.desired
|
||||
k8s.hpa.max_pods: k8s.hpa.pod.max
|
||||
k8s.hpa.min_pods: k8s.hpa.pod.min
|
||||
k8s.job.active_pods: k8s.job.pod.active
|
||||
k8s.job.desired_successful_pods: k8s.job.pod.desired_successful
|
||||
k8s.job.failed_pods: k8s.job.pod.failed
|
||||
k8s.job.max_parallel_pods: k8s.job.pod.max_parallel
|
||||
k8s.job.successful_pods: k8s.job.pod.successful
|
||||
k8s.node.allocatable.cpu: k8s.node.cpu.allocatable
|
||||
k8s.node.allocatable.ephemeral_storage: k8s.node.ephemeral_storage.allocatable
|
||||
k8s.node.allocatable.memory: k8s.node.memory.allocatable
|
||||
k8s.node.allocatable.pods: k8s.node.pod.allocatable
|
||||
k8s.replicaset.available_pods: k8s.replicaset.pod.available
|
||||
k8s.replicaset.desired_pods: k8s.replicaset.pod.desired
|
||||
k8s.replication_controller.available_pods: k8s.replicationcontroller.pod.available
|
||||
k8s.replication_controller.desired_pods: k8s.replicationcontroller.pod.desired
|
||||
k8s.replicationcontroller.available_pods: k8s.replicationcontroller.pod.available
|
||||
k8s.replicationcontroller.desired_pods: k8s.replicationcontroller.pod.desired
|
||||
k8s.statefulset.current_pods: k8s.statefulset.pod.current
|
||||
k8s.statefulset.desired_pods: k8s.statefulset.pod.desired
|
||||
k8s.statefulset.ready_pods: k8s.statefulset.pod.ready
|
||||
k8s.statefulset.updated_pods: k8s.statefulset.pod.updated
|
||||
v8js.heap.space.available_size: v8js.memory.heap.space.available_size
|
||||
v8js.heap.space.physical_size: v8js.memory.heap.space.physical_size
|
||||
1.37.0:
|
||||
all:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
android.state: android.app.state
|
||||
container.runtime: container.runtime.name
|
||||
enduser.role: user.roles
|
||||
gen_ai.openai.request.service_tier: openai.request.service_tier
|
||||
gen_ai.openai.response.service_tier: openai.response.service_tier
|
||||
gen_ai.openai.response.system_fingerprint: openai.response.system_fingerprint
|
||||
gen_ai.system: gen_ai.provider.name
|
||||
ios.state: ios.app.state
|
||||
1.36.0:
|
||||
1.35.0:
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1698
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
az.namespace: azure.resource_provider.namespace
|
||||
az.service_request_id: azure.service.request.id
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/issues/1800
|
||||
- rename_metrics:
|
||||
system.network.connections: system.network.connection.count
|
||||
1.34.0:
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/2295
|
||||
- rename_metrics:
|
||||
cpu.time: system.cpu.time
|
||||
cpu.utilization: system.cpu.utilization
|
||||
cpu.frequency: system.cpu.frequency
|
||||
1.33.0:
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1982
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
feature_flag.provider_name: feature_flag.provider.name
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1994
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
feature_flag.evaluation.error.message: error.message
|
||||
1.32.0:
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1989
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
feature_flag.evaluation.reason: feature_flag.result.reason
|
||||
feature_flag.variant: feature_flag.result.variant
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/2042
|
||||
- rename_metrics:
|
||||
otel.sdk.span.live.count: otel.sdk.span.live
|
||||
otel.sdk.span.ended.count: otel.sdk.span.ended
|
||||
otel.sdk.processor.span.processed.count: otel.sdk.processor.span.processed
|
||||
otel.sdk.exporter.span.inflight.count: otel.sdk.exporter.span.inflight
|
||||
otel.sdk.exporter.span.exported.count: otel.sdk.exporter.span.exported
|
||||
1.31.0:
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1880
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
android.state: android.app.state
|
||||
io.state: ios.app.state
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
k8s.replication_controller.desired_pods: k8s.replicationcontroller.desired_pods
|
||||
k8s.replication_controller.available_pods: k8s.replicationcontroller.available_pods
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1896
|
||||
- rename_metrics:
|
||||
system.cpu.time: cpu.time
|
||||
system.cpu.utilization: cpu.utilization
|
||||
system.cpu.frequency: cpu.frequency
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1896
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
system.cpu.logical_number: cpu.logical_number
|
||||
1.30.0:
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1632
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
gen_ai.openai.request.seed: gen_ai.request.seed
|
||||
system.network.state: network.connection.state
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1624
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
code.function: code.function.name
|
||||
code.filepath: code.file.path
|
||||
code.lineno: code.line.number
|
||||
code.column: code.column.number
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1734
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.system: db.system.name
|
||||
db.cassandra.coordinator.dc: cassandra.coordinator.dc
|
||||
db.cassandra.coordinator.id: cassandra.coordinator.id
|
||||
db.cassandra.consistency_level: cassandra.consistency.level
|
||||
db.cassandra.idempotence: cassandra.query.idempotent
|
||||
db.cassandra.page_size: cassandra.page.size
|
||||
db.cassandra.speculative_execution_count: cassandra.speculative_execution.count
|
||||
db.cosmosdb.client_id: azure.client.id
|
||||
db.cosmosdb.connection_mode: azure.cosmosdb.connection.mode
|
||||
db.cosmosdb.consistency_level: azure.cosmosdb.consistency.level
|
||||
db.cosmosdb.request_charge: azure.cosmosdb.operation.request_charge
|
||||
db.cosmosdb.request_content_length: azure.cosmosdb.request.body.size
|
||||
db.cosmosdb.regions_contacted: azure.cosmosdb.operation.contacted_regions
|
||||
db.cosmosdb.sub_status_code: azure.cosmosdb.response.sub_status_code
|
||||
db.elasticsearch.node.name: elasticsearch.node.name
|
||||
# db.elasticsearch.path_parts is a template attribute, schema transformation
|
||||
# does not support it, adding as a comment for consistency
|
||||
# db.elasticsearch.path_parts.<key> -> db.operation.parameter.<key>
|
||||
metrics:
|
||||
changes:
|
||||
- rename_metrics:
|
||||
db.client.cosmosdb.operation.request_charge: azure.cosmosdb.client.operation.request_charge
|
||||
db.client.cosmosdb.active_instance.count: azure.cosmosdb.client.active_instance.count
|
||||
|
||||
1.29.0:
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1520
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
process.executable.build_id.profiling: process.executable.build_id.htlhash
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1383
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
vcs.repository.change.id: vcs.change.id
|
||||
vcs.repository.change.title: vcs.change.title
|
||||
vcs.repository.ref.name: vcs.ref.head.name
|
||||
vcs.repository.ref.revision: vcs.ref.head.revision
|
||||
vcs.repository.ref.type: vcs.ref.head.type
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1492
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
system.device: network.interface.name
|
||||
apply_to_metrics:
|
||||
- container.network.io
|
||||
- system.network.dropped
|
||||
- system.network.errors
|
||||
- system.network.io
|
||||
- system.network.connections
|
||||
1.28.0:
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1422
|
||||
- rename_metrics:
|
||||
messaging.client.published.messages: messaging.client.sent.messages
|
||||
1.27.0:
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1216
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
tls.client.server_name: server.address
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1075
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
deployment.environment: deployment.environment.name
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1245
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.kafka.message.offset: messaging.kafka.offset
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/815
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.kafka.consumer.group: messaging.consumer.group.name
|
||||
messaging.rocketmq.client_group: messaging.consumer.group.name
|
||||
messaging.eventhubs.consumer.group: messaging.consumer.group.name
|
||||
messaging.servicebus.destination.subscription_name: messaging.destination.subscription.name
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1200
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
gen_ai.usage.completion_tokens: gen_ai.usage.output_tokens
|
||||
gen_ai.usage.prompt_tokens: gen_ai.usage.input_tokens
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1002
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.elasticsearch.cluster.name: db.namespace
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1125
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.client.connections.state: db.client.connection.state
|
||||
apply_to_metrics:
|
||||
- db.client.connection.count
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.client.connections.pool.name: db.client.connection.pool.name
|
||||
apply_to_metrics:
|
||||
- db.client.connection.count
|
||||
- db.client.connection.idle.max
|
||||
- db.client.connection.idle.min
|
||||
- db.client.connection.max
|
||||
- db.client.connection.pending_requests
|
||||
- db.client.connection.timeouts
|
||||
- db.client.connection.create_time
|
||||
- db.client.connection.wait_time
|
||||
- db.client.connection.use_time
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1006
|
||||
- rename_metrics:
|
||||
messaging.publish.messages: messaging.client.published.messages
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1026
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
system.cpu.state: cpu.mode
|
||||
process.cpu.state: cpu.mode
|
||||
container.cpu.state: cpu.mode
|
||||
apply_to_metrics:
|
||||
- system.cpu.time
|
||||
- system.cpu.utilization
|
||||
- process.cpu.time
|
||||
- process.cpu.utilization
|
||||
- container.cpu.time
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/1265
|
||||
- rename_metrics:
|
||||
jvm.buffer.memory.usage: jvm.buffer.memory.used
|
||||
1.26.0:
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/966
|
||||
- rename_metrics:
|
||||
db.client.connections.usage: db.client.connection.count
|
||||
db.client.connections.idle.max: db.client.connection.idle.max
|
||||
db.client.connections.idle.min: db.client.connection.idle.min
|
||||
db.client.connections.max: db.client.connection.max
|
||||
db.client.connections.pending_requests: db.client.connection.pending_requests
|
||||
db.client.connections.timeouts: db.client.connection.timeouts
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/948
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.client_id: messaging.client.id
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/909
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
state: db.client.connections.state
|
||||
apply_to_metrics:
|
||||
- db.client.connections.usage
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
pool.name: db.client.connections.pool.name
|
||||
apply_to_metrics:
|
||||
- db.client.connections.usage
|
||||
- db.client.connections.idle.max
|
||||
- db.client.connections.idle.min
|
||||
- db.client.connections.max
|
||||
- db.client.connections.pending_requests
|
||||
- db.client.connections.timeouts
|
||||
- db.client.connections.create_time
|
||||
- db.client.connections.wait_time
|
||||
- db.client.connections.use_time
|
||||
all:
|
||||
changes:
|
||||
# https://github:com/open-telemetry/semantic-conventions/pull/731/
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
enduser.id: user.id
|
||||
|
||||
1.25.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/911
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.name: db.namespace
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/870
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.sql.table: db.collection.name
|
||||
db.mongodb.collection: db.collection.name
|
||||
db.cosmosdb.container: db.collection.name
|
||||
db.cassandra.table: db.collection.name
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/798
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.kafka.destination.partition: messaging.destination.partition.id
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/875
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.operation: db.operation.name
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/913
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.operation: messaging.operation.type
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/866
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.statement: db.query.text
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/484
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
system.processes.status: system.process.status
|
||||
apply_to_metrics:
|
||||
- system.processes.count
|
||||
- rename_metrics:
|
||||
system.processes.count: system.process.count
|
||||
system.processes.created: system.process.created
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/625
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
container.labels: container.label
|
||||
k8s.pod.labels: k8s.pod.label
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/330
|
||||
- rename_metrics:
|
||||
process.threads: process.thread.count
|
||||
process.open_file_descriptors: process.open_file_descriptor.count
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
state: process.cpu.state
|
||||
apply_to_metrics:
|
||||
- process.cpu.time
|
||||
- process.cpu.utilization
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
direction: disk.io.direction
|
||||
apply_to_metrics:
|
||||
- process.disk.io
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
type: process.context_switch_type
|
||||
apply_to_metrics:
|
||||
- process.context_switches
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
direction: network.io.direction
|
||||
apply_to_metrics:
|
||||
- process.network.io
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
type: process.paging.fault_type
|
||||
apply_to_metrics:
|
||||
- process.paging.faults
|
||||
all:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/854
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
message.type: rpc.message.type
|
||||
message.id: rpc.message.id
|
||||
message.compressed_size: rpc.message.compressed_size
|
||||
message.uncompressed_size: rpc.message.uncompressed_size
|
||||
|
||||
1.24.0:
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/536
|
||||
- rename_metrics:
|
||||
jvm.memory.usage: jvm.memory.used
|
||||
jvm.memory.usage_after_last_gc: jvm.memory.used_after_last_gc
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/530
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
system.network.io.direction: network.io.direction
|
||||
system.disk.io.direction: disk.io.direction
|
||||
1.23.1:
|
||||
1.23.0:
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/20
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
thread.daemon: jvm.thread.daemon
|
||||
apply_to_metrics:
|
||||
- jvm.thread.count
|
||||
1.22.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/229
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.message.payload_size_bytes: messaging.message.body.size
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/374
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
http.resend_count: http.request.resend_count
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/224
|
||||
- rename_metrics:
|
||||
http.client.duration: http.client.request.duration
|
||||
http.server.duration: http.server.request.duration
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/241
|
||||
- rename_metrics:
|
||||
process.runtime.jvm.memory.usage: jvm.memory.usage
|
||||
process.runtime.jvm.memory.committed: jvm.memory.committed
|
||||
process.runtime.jvm.memory.limit: jvm.memory.limit
|
||||
process.runtime.jvm.memory.usage_after_last_gc: jvm.memory.usage_after_last_gc
|
||||
process.runtime.jvm.gc.duration: jvm.gc.duration
|
||||
# also https://github.com/open-telemetry/semantic-conventions/pull/252
|
||||
process.runtime.jvm.threads.count: jvm.thread.count
|
||||
# also https://github.com/open-telemetry/semantic-conventions/pull/252
|
||||
process.runtime.jvm.classes.loaded: jvm.class.loaded
|
||||
# also https://github.com/open-telemetry/semantic-conventions/pull/252
|
||||
process.runtime.jvm.classes.unloaded: jvm.class.unloaded
|
||||
# also https://github.com/open-telemetry/semantic-conventions/pull/252
|
||||
# and https://github.com/open-telemetry/semantic-conventions/pull/60
|
||||
process.runtime.jvm.classes.current_loaded: jvm.class.count
|
||||
process.runtime.jvm.cpu.time: jvm.cpu.time
|
||||
process.runtime.jvm.cpu.recent_utilization: jvm.cpu.recent_utilization
|
||||
process.runtime.jvm.memory.init: jvm.memory.init
|
||||
process.runtime.jvm.system.cpu.utilization: jvm.system.cpu.utilization
|
||||
process.runtime.jvm.system.cpu.load_1m: jvm.system.cpu.load_1m
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/253
|
||||
process.runtime.jvm.buffer.usage: jvm.buffer.memory.usage
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/253
|
||||
process.runtime.jvm.buffer.limit: jvm.buffer.memory.limit
|
||||
process.runtime.jvm.buffer.count: jvm.buffer.count
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/20
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
type: jvm.memory.type
|
||||
pool: jvm.memory.pool.name
|
||||
apply_to_metrics:
|
||||
- jvm.memory.usage
|
||||
- jvm.memory.committed
|
||||
- jvm.memory.limit
|
||||
- jvm.memory.usage_after_last_gc
|
||||
- jvm.memory.init
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
name: jvm.gc.name
|
||||
action: jvm.gc.action
|
||||
apply_to_metrics:
|
||||
- jvm.gc.duration
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
daemon: thread.daemon
|
||||
apply_to_metrics:
|
||||
- jvm.threads.count
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
pool: jvm.buffer.pool.name
|
||||
apply_to_metrics:
|
||||
- jvm.buffer.memory.usage
|
||||
- jvm.buffer.memory.limit
|
||||
- jvm.buffer.count
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/89
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
state: system.cpu.state
|
||||
cpu: system.cpu.logical_number
|
||||
apply_to_metrics:
|
||||
- system.cpu.time
|
||||
- system.cpu.utilization
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
state: system.memory.state
|
||||
apply_to_metrics:
|
||||
- system.memory.usage
|
||||
- system.memory.utilization
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
state: system.paging.state
|
||||
apply_to_metrics:
|
||||
- system.paging.usage
|
||||
- system.paging.utilization
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
type: system.paging.type
|
||||
direction: system.paging.direction
|
||||
apply_to_metrics:
|
||||
- system.paging.faults
|
||||
- system.paging.operations
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
device: system.device
|
||||
direction: system.disk.direction
|
||||
apply_to_metrics:
|
||||
- system.disk.io
|
||||
- system.disk.operations
|
||||
- system.disk.io_time
|
||||
- system.disk.operation_time
|
||||
- system.disk.merged
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
device: system.device
|
||||
state: system.filesystem.state
|
||||
type: system.filesystem.type
|
||||
mode: system.filesystem.mode
|
||||
mountpoint: system.filesystem.mountpoint
|
||||
apply_to_metrics:
|
||||
- system.filesystem.usage
|
||||
- system.filesystem.utilization
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
device: system.device
|
||||
direction: system.network.direction
|
||||
protocol: network.protocol
|
||||
state: system.network.state
|
||||
apply_to_metrics:
|
||||
- system.network.dropped
|
||||
- system.network.packets
|
||||
- system.network.errors
|
||||
- system.network.io
|
||||
- system.network.connections
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
status: system.processes.status
|
||||
apply_to_metrics:
|
||||
- system.processes.count
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/247
|
||||
- rename_metrics:
|
||||
http.server.request.size: http.server.request.body.size
|
||||
http.server.response.size: http.server.response.body.size
|
||||
resources:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/178
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
telemetry.auto.version: telemetry.distro.version
|
||||
1.21.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3336
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.kafka.client_id: messaging.client_id
|
||||
messaging.rocketmq.client_id: messaging.client_id
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3402
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
# net.peer.(name|port) attributes were usually populated on client side
|
||||
# so they should be usually translated to server.(address|port)
|
||||
# net.host.* attributes were only populated on server side
|
||||
net.host.name: server.address
|
||||
net.host.port: server.port
|
||||
# was only populated on client side
|
||||
net.sock.peer.name: server.socket.domain
|
||||
# net.sock.peer.(addr|port) mapping is not possible
|
||||
# since they applied to both client and server side
|
||||
# were only populated on server side
|
||||
net.sock.host.addr: server.socket.address
|
||||
net.sock.host.port: server.socket.port
|
||||
http.client_ip: client.address
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3426
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
net.protocol.name: network.protocol.name
|
||||
net.protocol.version: network.protocol.version
|
||||
net.host.connection.type: network.connection.type
|
||||
net.host.connection.subtype: network.connection.subtype
|
||||
net.host.carrier.name: network.carrier.name
|
||||
net.host.carrier.mcc: network.carrier.mcc
|
||||
net.host.carrier.mnc: network.carrier.mnc
|
||||
net.host.carrier.icc: network.carrier.icc
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3355
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
http.method: http.request.method
|
||||
http.status_code: http.response.status_code
|
||||
http.scheme: url.scheme
|
||||
http.url: url.full
|
||||
http.request_content_length: http.request.body.size
|
||||
http.response_content_length: http.response.body.size
|
||||
metrics:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/semantic-conventions/pull/53
|
||||
- rename_metrics:
|
||||
process.runtime.jvm.cpu.utilization: process.runtime.jvm.cpu.recent_utilization
|
||||
1.20.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3272
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
net.app.protocol.name: net.protocol.name
|
||||
net.app.protocol.version: net.protocol.version
|
||||
1.19.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3209
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
faas.execution: faas.invocation_id
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3188
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
faas.id: cloud.resource_id
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3190
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
http.user_agent: user_agent.original
|
||||
resources:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/3190
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
browser.user_agent: user_agent.original
|
||||
1.18.0:
|
||||
1.17.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/2957
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
messaging.consumer_id: messaging.consumer.id
|
||||
messaging.protocol: net.app.protocol.name
|
||||
messaging.protocol_version: net.app.protocol.version
|
||||
messaging.destination: messaging.destination.name
|
||||
messaging.temp_destination: messaging.destination.temporary
|
||||
messaging.destination_kind: messaging.destination.kind
|
||||
messaging.message_id: messaging.message.id
|
||||
messaging.conversation_id: messaging.message.conversation_id
|
||||
messaging.message_payload_size_bytes: messaging.message.payload_size_bytes
|
||||
messaging.message_payload_compressed_size_bytes: messaging.message.payload_compressed_size_bytes
|
||||
messaging.rabbitmq.routing_key: messaging.rabbitmq.destination.routing_key
|
||||
messaging.kafka.message_key: messaging.kafka.message.key
|
||||
messaging.kafka.partition: messaging.kafka.destination.partition
|
||||
messaging.kafka.tombstone: messaging.kafka.message.tombstone
|
||||
messaging.rocketmq.message_type: messaging.rocketmq.message.type
|
||||
messaging.rocketmq.message_tag: messaging.rocketmq.message.tag
|
||||
messaging.rocketmq.message_keys: messaging.rocketmq.message.keys
|
||||
messaging.kafka.consumer_group: messaging.kafka.consumer.group
|
||||
1.16.0:
|
||||
1.15.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/2743
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
http.retry_count: http.resend_count
|
||||
1.14.0:
|
||||
1.13.0:
|
||||
spans:
|
||||
changes:
|
||||
# https://github.com/open-telemetry/opentelemetry-specification/pull/2614
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
net.peer.ip: net.sock.peer.addr
|
||||
net.host.ip: net.sock.host.addr
|
||||
1.12.0:
|
||||
1.11.0:
|
||||
1.10.0:
|
||||
1.9.0:
|
||||
1.8.0:
|
||||
spans:
|
||||
changes:
|
||||
- rename_attributes:
|
||||
attribute_map:
|
||||
db.cassandra.keyspace: db.name
|
||||
db.hbase.namespace: db.name
|
||||
1.7.0:
|
||||
1.6.1:
|
||||
1.5.0:
|
||||
1.4.0:
|
||||
@@ -18,6 +18,7 @@ pytest_plugins = [
|
||||
"fixtures.logs",
|
||||
"fixtures.traces",
|
||||
"fixtures.metrics",
|
||||
"fixtures.queriercommon",
|
||||
"fixtures.metadata",
|
||||
"fixtures.meter",
|
||||
"fixtures.browser",
|
||||
@@ -31,6 +32,7 @@ pytest_plugins = [
|
||||
"fixtures.seeder",
|
||||
"fixtures.serviceaccount",
|
||||
"fixtures.role",
|
||||
"fixtures.savedview",
|
||||
"fixtures.seed_golden_dataset",
|
||||
]
|
||||
|
||||
|
||||
3
tests/fixtures/clickhouse.py
vendored
3
tests/fixtures/clickhouse.py
vendored
@@ -329,9 +329,6 @@ def clickhouse(
|
||||
request: pytest.FixtureRequest,
|
||||
pytestconfig: pytest.Config,
|
||||
) -> types.TestContainerClickhouse:
|
||||
"""
|
||||
Package-scoped fixture for Clickhouse TestContainer.
|
||||
"""
|
||||
return create_clickhouse(
|
||||
tmpfs=tmpfs,
|
||||
network=network,
|
||||
|
||||
2
tests/fixtures/cloudintegrations.py
vendored
2
tests/fixtures/cloudintegrations.py
vendored
@@ -1,5 +1,3 @@
|
||||
"""Fixtures for cloud integration tests."""
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from http import HTTPStatus
|
||||
|
||||
30
tests/fixtures/dashboards.py
vendored
Normal file
30
tests/fixtures/dashboards.py
vendored
Normal file
@@ -0,0 +1,30 @@
|
||||
from http import HTTPStatus
|
||||
|
||||
import requests
|
||||
|
||||
from fixtures import types
|
||||
|
||||
DASHBOARDS_BASE_URL = "/api/v2/dashboards"
|
||||
# MaxListLimit caps a single list page, so wiping a shared DB has to drain pages
|
||||
# until the list comes back empty.
|
||||
MAX_LIST_LIMIT = 200
|
||||
|
||||
|
||||
def delete_all_dashboards(signoz: types.SigNoz, token: str) -> None:
|
||||
while True:
|
||||
response = requests.get(
|
||||
signoz.self.host_configs["8080"].get(f"{DASHBOARDS_BASE_URL}?limit={MAX_LIST_LIMIT}"),
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
timeout=5,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK, response.text
|
||||
dashboards = response.json()["data"]["dashboards"]
|
||||
if not dashboards:
|
||||
return
|
||||
for dashboard in dashboards:
|
||||
del_res = requests.delete(
|
||||
signoz.self.host_configs["8080"].get(f"{DASHBOARDS_BASE_URL}/{dashboard['id']}"),
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
timeout=5,
|
||||
)
|
||||
assert del_res.status_code == HTTPStatus.NO_CONTENT, del_res.text
|
||||
6
tests/fixtures/http.py
vendored
6
tests/fixtures/http.py
vendored
@@ -24,9 +24,6 @@ def zeus(
|
||||
request: pytest.FixtureRequest,
|
||||
pytestconfig: pytest.Config,
|
||||
) -> types.TestContainerDocker:
|
||||
"""
|
||||
Package-scoped fixture for running zeus
|
||||
"""
|
||||
|
||||
def create() -> types.TestContainerDocker:
|
||||
container = WireMockContainer(image="wiremock/wiremock:2.35.1-1", secure=False)
|
||||
@@ -76,9 +73,6 @@ def gateway(
|
||||
request: pytest.FixtureRequest,
|
||||
pytestconfig: pytest.Config,
|
||||
) -> types.TestContainerDocker:
|
||||
"""
|
||||
Package-scoped fixture for running gateway
|
||||
"""
|
||||
|
||||
def create() -> types.TestContainerDocker:
|
||||
container = WireMockContainer(image="wiremock/wiremock:2.35.1-1", secure=False)
|
||||
|
||||
23
tests/fixtures/idp.py
vendored
23
tests/fixtures/idp.py
vendored
@@ -7,6 +7,7 @@ import pytest
|
||||
import requests
|
||||
from keycloak import KeycloakAdmin
|
||||
from selenium import webdriver
|
||||
from selenium.common.exceptions import WebDriverException
|
||||
from selenium.webdriver.common.by import By
|
||||
from selenium.webdriver.support import expected_conditions as EC
|
||||
from selenium.webdriver.support.wait import WebDriverWait
|
||||
@@ -370,18 +371,26 @@ def idp_login(driver: webdriver.Chrome) -> Callable[[str, str], None]:
|
||||
password_field.send_keys(password)
|
||||
|
||||
# Click the login button
|
||||
idp_host = urlparse(driver.current_url).netloc
|
||||
login_button = wait.until(EC.element_to_be_clickable((By.ID, "kc-login")))
|
||||
login_button.click()
|
||||
|
||||
# Wait till kc-login element has vanished from the page, which means that a redirection is taking place.
|
||||
wait.until(EC.invisibility_of_element((By.ID, "kc-login")))
|
||||
# Wait till the browser has left the idp host — not just the login page: keycloak's SAML flow inserts an
|
||||
# auto-submitting interstitial on the idp whose POST is what creates the user in signoz. The button is
|
||||
# re-queried per poll; a mid-navigation WebDriverException (detached node) just retries the poll.
|
||||
def _left_idp(drv: webdriver.Chrome) -> bool:
|
||||
try:
|
||||
return urlparse(drv.current_url).netloc != idp_host and not drv.find_elements(By.ID, "kc-login")
|
||||
except WebDriverException:
|
||||
return False
|
||||
|
||||
wait.until(_left_idp)
|
||||
|
||||
return _idp_login
|
||||
|
||||
|
||||
@pytest.fixture(name="create_group_idp", scope="function")
|
||||
def create_group_idp(idp: types.TestContainerIDP) -> Callable[[str], str]:
|
||||
"""Creates a group in Keycloak IDP."""
|
||||
client = KeycloakAdmin(
|
||||
server_url=idp.container.host_configs["6060"].base(),
|
||||
username=IDP_ROOT_USERNAME,
|
||||
@@ -410,7 +419,6 @@ def create_user_idp_with_groups(
|
||||
idp: types.TestContainerIDP,
|
||||
create_group_idp: Callable[[str], str], # pylint: disable=redefined-outer-name
|
||||
) -> Callable[[str, str, bool, list[str]], None]:
|
||||
"""Creates a user in Keycloak IDP with specified groups."""
|
||||
client = KeycloakAdmin(
|
||||
server_url=idp.container.host_configs["6060"].base(),
|
||||
username=IDP_ROOT_USERNAME,
|
||||
@@ -458,7 +466,6 @@ def add_user_to_group(
|
||||
idp: types.TestContainerIDP,
|
||||
create_group_idp: Callable[[str], str], # pylint: disable=redefined-outer-name
|
||||
) -> Callable[[str, str], None]:
|
||||
"""Adds an existing user to a group."""
|
||||
client = KeycloakAdmin(
|
||||
server_url=idp.container.host_configs["6060"].base(),
|
||||
username=IDP_ROOT_USERNAME,
|
||||
@@ -479,7 +486,6 @@ def create_user_idp_with_role(
|
||||
idp: types.TestContainerIDP,
|
||||
create_group_idp: Callable[[str], str], # pylint: disable=redefined-outer-name
|
||||
) -> Callable[[str, str, bool, str, list[str]], None]:
|
||||
"""Creates a user in Keycloak IDP with a custom role attribute and optional groups."""
|
||||
client = KeycloakAdmin(
|
||||
server_url=idp.container.host_configs["6060"].base(),
|
||||
username=IDP_ROOT_USERNAME,
|
||||
@@ -527,7 +533,6 @@ def create_user_idp_with_role(
|
||||
|
||||
@pytest.fixture(name="setup_user_profile", scope="package")
|
||||
def setup_user_profile(idp: types.TestContainerIDP) -> Callable[[], None]:
|
||||
"""Setup Keycloak User Profile with signoz_role attribute."""
|
||||
|
||||
def _setup_user_profile() -> None:
|
||||
client = KeycloakAdmin(
|
||||
@@ -568,7 +573,6 @@ def setup_user_profile(idp: types.TestContainerIDP) -> Callable[[], None]:
|
||||
|
||||
|
||||
def _ensure_groups_client_scope(client: KeycloakAdmin) -> None:
|
||||
"""Create 'groups' client scope if it doesn't exist."""
|
||||
# Check if groups scope exists
|
||||
scopes = client.get_client_scopes()
|
||||
groups_scope_exists = any(s.get("name") == "groups" for s in scopes)
|
||||
@@ -619,7 +623,6 @@ def _ensure_groups_client_scope(client: KeycloakAdmin) -> None:
|
||||
|
||||
|
||||
def get_oidc_domain(signoz: types.SigNoz, admin_token: str) -> dict:
|
||||
"""Helper to get the OIDC domain."""
|
||||
response = requests.get(
|
||||
signoz.self.host_configs["8080"].get("/api/v1/domains"),
|
||||
headers={"Authorization": f"Bearer {admin_token}"},
|
||||
@@ -632,7 +635,6 @@ def get_oidc_domain(signoz: types.SigNoz, admin_token: str) -> dict:
|
||||
|
||||
|
||||
def get_user_by_email(signoz: types.SigNoz, admin_token: str, email: str) -> dict:
|
||||
"""Helper to get a user by email."""
|
||||
response = requests.get(
|
||||
signoz.self.host_configs["8080"].get("/api/v1/user"),
|
||||
timeout=2,
|
||||
@@ -653,7 +655,6 @@ def perform_oidc_login(
|
||||
email: str,
|
||||
password: str,
|
||||
) -> None:
|
||||
"""Helper to perform OIDC login flow."""
|
||||
session_context = get_session_context(email)
|
||||
url = session_context["orgs"][0]["authNSupport"]["callback"][0]["url"]
|
||||
parsed_url = urlparse(url)
|
||||
|
||||
2
tests/fixtures/inframonitoring.py
vendored
2
tests/fixtures/inframonitoring.py
vendored
@@ -1,5 +1,3 @@
|
||||
"""Shared constants/helpers for v2 infra-monitoring pod-status tests."""
|
||||
|
||||
# All 18 PodCountsByStatus buckets (camelCase, matches inframonitoringtypes.PodCountsByStatus / the API response).
|
||||
STATUS_BUCKETS = (
|
||||
"pending",
|
||||
|
||||
80
tests/fixtures/jsontypes.py
vendored
80
tests/fixtures/jsontypes.py
vendored
@@ -1,9 +1,3 @@
|
||||
"""
|
||||
Simpler version of metadataexporter for exporting jsontypes for test fixtures.
|
||||
This exports JSON type metadata to the path_types table by parsing JSON bodies
|
||||
and extracting all paths with their types, similar to how the real metadataexporter works.
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import json
|
||||
from abc import ABC
|
||||
@@ -21,8 +15,6 @@ from fixtures import types
|
||||
|
||||
|
||||
class JSONPathType(ABC):
|
||||
"""Represents a JSON path with its type information"""
|
||||
|
||||
field_name: str
|
||||
field_data_type: str
|
||||
last_seen: np.uint64
|
||||
@@ -44,7 +36,6 @@ class JSONPathType(ABC):
|
||||
self.last_seen = np.uint64(int(last_seen.timestamp() * 1e9))
|
||||
|
||||
def np_arr(self) -> np.array:
|
||||
"""Return path type data as numpy array for database insertion"""
|
||||
return np.array([self.signal, self.field_context, self.field_name, self.field_data_type, self.last_seen])
|
||||
|
||||
|
||||
@@ -145,7 +136,7 @@ def _python_type_to_clickhouse_type(value: Any) -> str:
|
||||
elif isinstance(value, dict):
|
||||
return "json"
|
||||
else:
|
||||
return "string" # Default fallback
|
||||
return "string"
|
||||
|
||||
|
||||
def _extract_json_paths(
|
||||
@@ -154,19 +145,7 @@ def _extract_json_paths(
|
||||
path_types: dict[str, set[str]] | None = None,
|
||||
level: int = 0,
|
||||
) -> dict[str, set[str]]:
|
||||
"""
|
||||
Recursively extract all paths and their types from a JSON object.
|
||||
Matches metadataexporter's analyzePValue logic.
|
||||
|
||||
Args:
|
||||
obj: The JSON object to traverse
|
||||
current_path: Current path being built (e.g., "user.name")
|
||||
path_types: Dictionary mapping paths to sets of types found
|
||||
level: Current nesting level (for depth limiting)
|
||||
|
||||
Returns:
|
||||
Dictionary mapping paths to sets of type strings
|
||||
"""
|
||||
"""Matches metadataexporter's analyzePValue logic."""
|
||||
if path_types is None:
|
||||
path_types = {}
|
||||
|
||||
@@ -179,17 +158,14 @@ def _extract_json_paths(
|
||||
# Matches Go walkMap which recurses without calling ta.record on the map node.
|
||||
|
||||
for key, value in obj.items():
|
||||
# Build the path for this key
|
||||
if current_path:
|
||||
new_path = f"{current_path}.{key}"
|
||||
else:
|
||||
new_path = key
|
||||
|
||||
# Recurse into the value
|
||||
_extract_json_paths(value, new_path, path_types, level + 1)
|
||||
|
||||
elif isinstance(obj, list):
|
||||
# Skip empty arrays
|
||||
if len(obj) == 0:
|
||||
return path_types
|
||||
|
||||
@@ -246,17 +222,6 @@ def _parse_json_bodies_and_extract_paths(
|
||||
json_bodies: list[str],
|
||||
timestamp: datetime.datetime | None = None,
|
||||
) -> list[JSONPathType]:
|
||||
"""
|
||||
Parse JSON bodies and extract all paths with their types.
|
||||
This mimics the behavior of metadataexporter.
|
||||
|
||||
Args:
|
||||
json_bodies: List of JSON body strings to parse
|
||||
timestamp: Timestamp to use for last_seen (defaults to now)
|
||||
|
||||
Returns:
|
||||
List of JSONPathType objects with all discovered paths and types
|
||||
"""
|
||||
if timestamp is None:
|
||||
timestamp = datetime.datetime.now()
|
||||
|
||||
@@ -268,11 +233,9 @@ def _parse_json_bodies_and_extract_paths(
|
||||
parsed = json.loads(json_body)
|
||||
_extract_json_paths(parsed, "", all_path_types, level=0)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
# Skip invalid JSON
|
||||
continue
|
||||
|
||||
# Convert to list of JSONPathType objects
|
||||
# Each path can have multiple types, so we create one JSONPathType per type
|
||||
# Each path can have multiple types -> one JSONPathType per type
|
||||
path_type_objects: list[JSONPathType] = []
|
||||
for path, types_set in all_path_types.items():
|
||||
for type_str in types_set:
|
||||
@@ -285,64 +248,34 @@ def _parse_json_bodies_and_extract_paths(
|
||||
def export_json_types(
|
||||
clickhouse: types.TestContainerClickhouse,
|
||||
) -> Generator[Callable[[list[JSONPathType] | list[str] | list[Any]], None], Any]:
|
||||
"""
|
||||
Fixture for exporting JSON type metadata to the path_types table.
|
||||
This is a simpler version of metadataexporter for test fixtures.
|
||||
"""Write JSON path/type metadata the way the real metadataexporter would.
|
||||
|
||||
The function can accept:
|
||||
1. List of JSONPathType objects (manual specification)
|
||||
2. List of JSON body strings (auto-extract paths)
|
||||
3. List of Logs objects (extract from body_json field)
|
||||
|
||||
Usage examples:
|
||||
# Manual specification
|
||||
export_json_types([
|
||||
JSONPathType(field_name="user.name", field_data_type="string"),
|
||||
JSONPathType(field_name="user.age", field_data_type="int64"),
|
||||
])
|
||||
|
||||
# Auto-extract from JSON strings
|
||||
export_json_types([
|
||||
'{"user": {"name": "alice", "age": 25}}',
|
||||
'{"user": {"name": "bob", "age": 30}}',
|
||||
])
|
||||
|
||||
# Auto-extract from Logs objects
|
||||
export_json_types(logs_list)
|
||||
Accepts JSONPathType objects (manual specification), raw JSON body strings,
|
||||
or Logs objects (paths auto-extracted from the JSON body).
|
||||
"""
|
||||
|
||||
def _export_json_types(
|
||||
data: list[JSONPathType] | list[str] | list[Any], # List[Logs] but avoiding circular import
|
||||
) -> None:
|
||||
"""
|
||||
Export JSON type metadata to signoz_metadata.distributed_field_keys table.
|
||||
This table stores signal, context, path, and type information for body JSON fields.
|
||||
"""
|
||||
path_types: list[JSONPathType] = []
|
||||
|
||||
if len(data) == 0:
|
||||
return
|
||||
|
||||
# Determine input type and convert to JSONPathType list
|
||||
first_item = data[0]
|
||||
|
||||
if isinstance(first_item, JSONPathType):
|
||||
# Already JSONPathType objects
|
||||
path_types = data # type: ignore
|
||||
elif isinstance(first_item, str):
|
||||
# List of JSON strings - parse and extract paths
|
||||
path_types = _parse_json_bodies_and_extract_paths(data) # type: ignore
|
||||
else:
|
||||
# Assume it's a list of Logs objects - extract body_v2
|
||||
json_bodies: list[str] = []
|
||||
for log in data: # type: ignore
|
||||
# Try to get body_v2 attribute
|
||||
if hasattr(log, "body_v2") and log.body_v2:
|
||||
json_bodies.append(log.body_v2)
|
||||
elif hasattr(log, "body") and log.body:
|
||||
# Fallback to body if body_v2 not available
|
||||
try:
|
||||
# Try to parse as JSON
|
||||
json.loads(log.body)
|
||||
json_bodies.append(log.body)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
@@ -369,7 +302,6 @@ def export_json_types(
|
||||
|
||||
yield _export_json_types
|
||||
|
||||
# Cleanup - truncate the local table after tests (following pattern from logs fixture)
|
||||
clickhouse.conn.query(f"TRUNCATE TABLE signoz_metadata.field_keys ON CLUSTER '{clickhouse.env['SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_CLUSTER']}' SYNC")
|
||||
|
||||
|
||||
|
||||
3
tests/fixtures/keeper.py
vendored
3
tests/fixtures/keeper.py
vendored
@@ -109,9 +109,6 @@ def keeper(
|
||||
request: pytest.FixtureRequest,
|
||||
pytestconfig: pytest.Config,
|
||||
) -> types.TestContainerDocker:
|
||||
"""
|
||||
Package-scoped fixture for ClickHouse Keeper TestContainer.
|
||||
"""
|
||||
return create_clickhouse_keeper(
|
||||
tmpfs=tmpfs,
|
||||
network=network,
|
||||
|
||||
3
tests/fixtures/keycloak.py
vendored
3
tests/fixtures/keycloak.py
vendored
@@ -19,9 +19,6 @@ def idp(
|
||||
request: pytest.FixtureRequest,
|
||||
pytestconfig: pytest.Config,
|
||||
) -> types.TestContainerIDP:
|
||||
"""
|
||||
Package-scoped fixture for running an idp for SSO/SAML
|
||||
"""
|
||||
|
||||
def create() -> types.TestContainerIDP:
|
||||
container = KeycloakContainer(
|
||||
|
||||
3
tests/fixtures/logs.py
vendored
3
tests/fixtures/logs.py
vendored
@@ -311,7 +311,6 @@ class Logs(ABC):
|
||||
self.attribute_keys.append(LogsResourceOrAttributeKeys(name="severity_number", datatype="float64"))
|
||||
|
||||
def _get_severity_number(self, severity_text: str) -> np.uint8:
|
||||
"""Convert severity text to numeric value"""
|
||||
severity_map = {
|
||||
"TRACE": 1,
|
||||
"DEBUG": 5,
|
||||
@@ -324,7 +323,6 @@ class Logs(ABC):
|
||||
return np.uint8(severity_map.get(severity_text.upper(), 9)) # Default to INFO
|
||||
|
||||
def np_arr(self) -> np.array:
|
||||
"""Return log data as numpy array for database insertion"""
|
||||
return np.array(
|
||||
[
|
||||
self.ts_bucket_start,
|
||||
@@ -356,7 +354,6 @@ class Logs(ABC):
|
||||
cls,
|
||||
data: dict,
|
||||
) -> "Logs":
|
||||
"""Create a Logs instance from a dict."""
|
||||
# parse timestamp from iso format
|
||||
timestamp = parse_timestamp(data["timestamp"])
|
||||
return cls(
|
||||
|
||||
12
tests/fixtures/metrics.py
vendored
12
tests/fixtures/metrics.py
vendored
@@ -374,6 +374,7 @@ class Metrics(ABC):
|
||||
file_path: str,
|
||||
base_time: datetime.datetime | None = None,
|
||||
metric_name_override: str | None = None,
|
||||
label_substitutions: dict[str, str] | None = None,
|
||||
) -> list["Metrics"]:
|
||||
"""
|
||||
Load metrics from a JSONL file.
|
||||
@@ -385,6 +386,9 @@ class Metrics(ABC):
|
||||
base_time: If provided, all timestamps are shifted so the earliest
|
||||
timestamp in the file maps to base_time
|
||||
metric_name_override: If provided, overrides metric_name for all metrics
|
||||
label_substitutions: If provided, any label whose value equals a key is
|
||||
rewritten to that key's value (placeholder substitution,
|
||||
e.g. {"__START_TIME__": start_time.isoformat()})
|
||||
"""
|
||||
data_list = []
|
||||
with open(file_path, encoding="utf-8") as f:
|
||||
@@ -392,7 +396,13 @@ class Metrics(ABC):
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
data_list.append(json.loads(line))
|
||||
data = json.loads(line)
|
||||
if label_substitutions:
|
||||
labels = data.get("labels", {})
|
||||
for key, value in labels.items():
|
||||
if value in label_substitutions:
|
||||
labels[key] = label_substitutions[value]
|
||||
data_list.append(data)
|
||||
|
||||
if not data_list:
|
||||
return []
|
||||
|
||||
3
tests/fixtures/migrator.py
vendored
3
tests/fixtures/migrator.py
vendored
@@ -92,9 +92,6 @@ def migrator(
|
||||
request: pytest.FixtureRequest,
|
||||
pytestconfig: pytest.Config,
|
||||
) -> types.Operation:
|
||||
"""
|
||||
Package-scoped fixture for running schema migrations.
|
||||
"""
|
||||
return create_migrator(
|
||||
network=network,
|
||||
clickhouse=clickhouse,
|
||||
|
||||
3
tests/fixtures/network.py
vendored
3
tests/fixtures/network.py
vendored
@@ -13,9 +13,6 @@ logger = setup_logger(__name__)
|
||||
|
||||
@pytest.fixture(name="network", scope="package")
|
||||
def network(request: pytest.FixtureRequest, pytestconfig: pytest.Config) -> types.Network:
|
||||
"""
|
||||
Package-Scoped fixture for creating a network
|
||||
"""
|
||||
|
||||
def create() -> types.Network:
|
||||
nw = Network()
|
||||
|
||||
3
tests/fixtures/postgres.py
vendored
3
tests/fixtures/postgres.py
vendored
@@ -13,9 +13,6 @@ logger = setup_logger(__name__)
|
||||
|
||||
@pytest.fixture(name="postgres", scope="package")
|
||||
def postgres(network: Network, request: pytest.FixtureRequest, pytestconfig: pytest.Config) -> types.TestContainerSQL:
|
||||
"""
|
||||
Package-scoped fixture for PostgreSQL TestContainer.
|
||||
"""
|
||||
|
||||
def create() -> types.TestContainerSQL:
|
||||
version = request.config.getoption("--postgres-version")
|
||||
|
||||
48
tests/fixtures/querier.py
vendored
48
tests/fixtures/querier.py
vendored
@@ -704,6 +704,7 @@ def build_raw_query(
|
||||
order: list[dict] | None = None,
|
||||
limit: int | None = None,
|
||||
filter_expression: str | None = None,
|
||||
select_fields: list[dict] | None = None,
|
||||
step_interval: int = DEFAULT_STEP_INTERVAL,
|
||||
disabled: bool = False,
|
||||
) -> dict:
|
||||
@@ -723,6 +724,9 @@ def build_raw_query(
|
||||
if filter_expression:
|
||||
spec["filter"] = {"expression": filter_expression}
|
||||
|
||||
if select_fields:
|
||||
spec["selectFields"] = select_fields
|
||||
|
||||
return {"type": "builder_query", "spec": spec}
|
||||
|
||||
|
||||
@@ -1105,3 +1109,47 @@ def make_scalar_query_request(
|
||||
"formatOptions": {"formatTableResultForUI": True, "fillGaps": False},
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def run_query_case(signoz: types.SigNoz, token: str, now: datetime, case: dict[str, Any]) -> None:
|
||||
start_ms = case.get("startMs", int((now - timedelta(seconds=10)).timestamp() * 1000))
|
||||
end_ms = case.get("endMs", int(now.timestamp() * 1000))
|
||||
|
||||
if case["requestType"] == "raw":
|
||||
query = build_raw_query(
|
||||
name=case["name"],
|
||||
signal="logs",
|
||||
filter_expression=case.get("expression"),
|
||||
order=case.get("order") or [build_order_by("timestamp", "desc")],
|
||||
limit=case.get("limit", 100),
|
||||
step_interval=case.get("stepInterval") or 60,
|
||||
)
|
||||
else:
|
||||
aggregation = case.get("aggregation")
|
||||
if aggregation and not isinstance(aggregation, list):
|
||||
aggregations = [build_aggregation(aggregation)]
|
||||
elif aggregation:
|
||||
aggregations = aggregation
|
||||
else:
|
||||
aggregations = []
|
||||
query = build_scalar_query(
|
||||
name=case["name"],
|
||||
signal="logs",
|
||||
aggregations=aggregations,
|
||||
group_by=case.get("groupBy"),
|
||||
order=case.get("order"),
|
||||
limit=case.get("limit", 100),
|
||||
filter_expression=case.get("expression"),
|
||||
step_interval=case.get("stepInterval") or 60,
|
||||
)
|
||||
|
||||
response = make_query_request(
|
||||
signoz=signoz,
|
||||
token=token,
|
||||
start_ms=start_ms,
|
||||
end_ms=end_ms,
|
||||
queries=[query],
|
||||
request_type=case["requestType"],
|
||||
)
|
||||
assert response.status_code == 200, f"HTTP {response.status_code} for case '{case['name']}': {response.text}"
|
||||
assert case["validate"](response), f"Validation failed for case '{case['name']}': {response.json()}"
|
||||
|
||||
5
tests/fixtures/querierai.py
vendored
5
tests/fixtures/querierai.py
vendored
@@ -1,8 +1,3 @@
|
||||
"""
|
||||
Trace builders for the querierai suite. Every builder pins its spans a few seconds
|
||||
before the given `now` so `query_window(now)` covers them.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
|
||||
|
||||
124
tests/fixtures/queriercommon.py
vendored
Normal file
124
tests/fixtures/queriercommon.py
vendored
Normal file
@@ -0,0 +1,124 @@
|
||||
"""Seed data for the queriercommon keyless-semantics tests.
|
||||
|
||||
Three identities exist in every signal. GOLD and SILVER carry the test keys.
|
||||
NONE carries no key at all. The tests assert which identities a filter
|
||||
returns, so the membership of NONE is the point of every case.
|
||||
|
||||
The attribute names are outside every semantic-convention family, so the
|
||||
seeded data pins base behavior with any semconv overlay state.
|
||||
"""
|
||||
|
||||
from collections.abc import Callable, Generator
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
import pytest
|
||||
|
||||
from fixtures.logs import Logs
|
||||
from fixtures.metrics import Metrics
|
||||
from fixtures.querier import aligned_epoch
|
||||
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
|
||||
|
||||
PREFIX = "keyless-sem"
|
||||
STRING_KEY = "tenant.tier"
|
||||
NUMBER_KEY = "retry.count"
|
||||
METRIC_NAME = "keyless_semantics_gauge"
|
||||
METRIC_LABEL = "tenant_tier"
|
||||
|
||||
# Row identities, keyed by the value of the string key that each row carries.
|
||||
GOLD = f"{PREFIX}-gold"
|
||||
SILVER = f"{PREFIX}-silver"
|
||||
NONE = f"{PREFIX}-none" # carries no string key and no number key
|
||||
|
||||
# (identity, string-key value, number-key value, insert offset)
|
||||
_ROWS = [
|
||||
(GOLD, "gold", 0, timedelta(seconds=3)),
|
||||
(SILVER, "silver", 5, timedelta(seconds=2)),
|
||||
(NONE, None, None, timedelta(seconds=1)),
|
||||
]
|
||||
|
||||
|
||||
def _resources(identity: str, tier: str | None) -> dict:
|
||||
base = {"service.name": identity}
|
||||
if tier is not None:
|
||||
base[STRING_KEY] = tier
|
||||
return base
|
||||
|
||||
|
||||
def _attributes(tier: str | None, retries: int | None) -> dict:
|
||||
attrs: dict = {}
|
||||
if tier is not None:
|
||||
attrs[STRING_KEY] = tier
|
||||
if retries is not None:
|
||||
attrs[NUMBER_KEY] = retries
|
||||
return attrs
|
||||
|
||||
|
||||
@pytest.fixture(name="keyless_rows", scope="function")
|
||||
def keyless_rows(
|
||||
insert_logs: Callable[[list[Logs]], None],
|
||||
insert_traces: Callable[[list[Traces]], None],
|
||||
) -> Generator[datetime]:
|
||||
"""Inserts one span and one log per identity: GOLD (string "gold",
|
||||
number 0), SILVER (string "silver", number 5), and NONE (no keys).
|
||||
Yields the base timestamp. Span name and log body are the identity."""
|
||||
now = datetime.now(tz=UTC).replace(microsecond=0) - timedelta(minutes=1)
|
||||
|
||||
insert_traces(
|
||||
[
|
||||
Traces(
|
||||
timestamp=now - offset,
|
||||
duration=timedelta(milliseconds=10),
|
||||
trace_id=TraceIdGenerator.trace_id(),
|
||||
span_id=TraceIdGenerator.span_id(),
|
||||
name=identity,
|
||||
kind=TracesKind.SPAN_KIND_SERVER,
|
||||
status_code=TracesStatusCode.STATUS_CODE_OK,
|
||||
resources=_resources(identity, tier),
|
||||
attributes=_attributes(tier, retries),
|
||||
)
|
||||
for identity, tier, retries, offset in _ROWS
|
||||
]
|
||||
)
|
||||
insert_logs(
|
||||
[
|
||||
Logs(
|
||||
timestamp=now - offset,
|
||||
body=identity,
|
||||
resources=_resources(identity, tier),
|
||||
attributes=_attributes(tier, retries),
|
||||
)
|
||||
for identity, tier, retries, offset in _ROWS
|
||||
]
|
||||
)
|
||||
yield now
|
||||
|
||||
|
||||
@pytest.fixture(name="keyless_series", scope="function")
|
||||
def keyless_series(insert_metrics: Callable[[list[Metrics]], None]) -> Generator[tuple[int, int]]:
|
||||
"""Inserts three gauge series: GOLD and SILVER carry the metric label,
|
||||
NONE does not. The `service` label is the identity. Yields the
|
||||
(start, end) epoch-second window that covers the points."""
|
||||
start = aligned_epoch(timedelta(minutes=30))
|
||||
points = 5
|
||||
|
||||
def labels(identity: str, tier: str | None) -> dict:
|
||||
base = {"service": identity}
|
||||
if tier is not None:
|
||||
base[METRIC_LABEL] = tier
|
||||
return base
|
||||
|
||||
insert_metrics(
|
||||
[
|
||||
Metrics(
|
||||
metric_name=METRIC_NAME,
|
||||
labels=labels(identity, tier),
|
||||
timestamp=datetime.fromtimestamp(start + minute * 60, tz=UTC),
|
||||
value=10.0,
|
||||
type_="Gauge",
|
||||
is_monotonic=False,
|
||||
)
|
||||
for identity, tier in ((GOLD, "gold"), (SILVER, "silver"), (NONE, None))
|
||||
for minute in range(points)
|
||||
]
|
||||
)
|
||||
yield start, start + points * 60
|
||||
3
tests/fixtures/reuse.py
vendored
3
tests/fixtures/reuse.py
vendored
@@ -19,17 +19,14 @@ def teardown(request: pytest.FixtureRequest) -> bool:
|
||||
|
||||
|
||||
def get_cached_resource(pytestconfig: pytest.Config, key: str):
|
||||
"""Get a resource from pytest cache by key."""
|
||||
return pytestconfig.cache.get(key, None)
|
||||
|
||||
|
||||
def set_cached_resource(pytestconfig: pytest.Config, key: str, value):
|
||||
"""Set a resource in pytest cache by key."""
|
||||
pytestconfig.cache.set(key, value)
|
||||
|
||||
|
||||
def remove_cached_resource(pytestconfig: pytest.Config, key: str):
|
||||
"""Remove a resource from pytest cache by key (set to None)."""
|
||||
pytestconfig.cache.set(key, None)
|
||||
|
||||
|
||||
|
||||
2
tests/fixtures/role.py
vendored
2
tests/fixtures/role.py
vendored
@@ -1,5 +1,3 @@
|
||||
"""Fixtures and helpers for role tests."""
|
||||
|
||||
import json
|
||||
from collections.abc import Callable
|
||||
from http import HTTPStatus
|
||||
|
||||
49
tests/fixtures/savedview.py
vendored
Normal file
49
tests/fixtures/savedview.py
vendored
Normal file
@@ -0,0 +1,49 @@
|
||||
"""Fixtures and helpers for saved view tests."""
|
||||
|
||||
from http import HTTPStatus
|
||||
|
||||
import requests
|
||||
|
||||
from fixtures import types
|
||||
|
||||
SAVED_VIEW_BASE = "/api/v2/saved_views"
|
||||
|
||||
|
||||
def _body(name: str, source: str = "logs") -> dict:
|
||||
return {
|
||||
"name": name,
|
||||
"source": source,
|
||||
"data": {
|
||||
"schemaVersion": "v2",
|
||||
"spec": {
|
||||
"displayName": name,
|
||||
"panelType": "table",
|
||||
"queries": [{"type": "builder_query", "spec": {"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}]}}],
|
||||
"selectedFields": [],
|
||||
"display": {"maxLines": 0, "fontSize": "", "format": "", "color": ""},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def create_saved_view(signoz: types.SigNoz, token: str, name: str, source: str = "logs") -> str:
|
||||
"""Create a saved view and return its ID."""
|
||||
resp = requests.post(
|
||||
signoz.self.host_configs["8080"].get(SAVED_VIEW_BASE),
|
||||
json=_body(name, source),
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
timeout=5,
|
||||
)
|
||||
assert resp.status_code == HTTPStatus.CREATED, resp.text
|
||||
return resp.json()["data"]["id"]
|
||||
|
||||
|
||||
def find_saved_view_by_name(signoz: types.SigNoz, token: str, name: str) -> dict:
|
||||
"""Find a saved view by name from the list endpoint."""
|
||||
resp = requests.get(
|
||||
signoz.self.host_configs["8080"].get(SAVED_VIEW_BASE),
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
timeout=5,
|
||||
)
|
||||
assert resp.status_code == HTTPStatus.OK, resp.text
|
||||
return next(view for view in resp.json()["data"] if view["name"] == name)
|
||||
8
tests/fixtures/seed_golden_dataset.py
vendored
8
tests/fixtures/seed_golden_dataset.py
vendored
@@ -1,11 +1,3 @@
|
||||
"""Golden dataset fixture — seeds OTel-demo-shaped metrics, traces, and
|
||||
logs into ClickHouse via the seeder on every test_setup invocation.
|
||||
|
||||
Timestamps are rebased to `now` so panels with default time windows
|
||||
always find data. To refresh the dataset shape on disk, run
|
||||
`uv run python -m fixtures.seed_golden_dataset regenerate`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user