Compare commits

..

6 Commits

Author SHA1 Message Date
srikanthccv
1d9fe687cb feat: resolve semantic convention names in trace queries
A referenced name now resolves to logical fields (LogicalField, #12499).
A slice shows ambiguity: each logical field gets its own condition. A
group shows a semantic-convention family: one field with more than one
spelling. Compilers merge a family with current-wins precedence, and
the response keeps the spelling that the request used.

- FieldMapper gets one new method: ExistsFor, the per-key existence
  primitive. Each mapper already had this knowledge; two rename a
  private method and five expose logic from their condition builders.
- LogicalValueExpr and LogicalExistsExpr build all family SQL in one
  place, only from FieldFor and ExistsFor. No signal implements family
  logic.
- Statement builders prefetch sibling spellings with
  ExpandKeySelectorsForFamilies. The metadata store stays family-blind,
  and autocomplete stays literal.
- The traces and resource-filter condition builders compile per logical
  field. The traces group-by path upgrades legacy candidates to their
  family in a post-pass; the legacy candidate flow does not change.
- Logs, metrics, audit, metadata, and rule-state-history flatten to
  single keys; their SQL does not change.

The resolve_semconv_families feature flag (default: disabled) gates all
family behavior. The components that resolve names hold the flagger and
evaluate the flag at the point of use, with the org from the call, the
same way the logs mapper uses use_json_body. A nil flagger means off,
so resolution stays literal on any path without one. With the flag off,
the generated SQL is the same as main, and tests pin this. The legacy
compatibility-key map stays in place for the same reason.

Assisted-by: Claude Fable 5
2026-08-10 23:11:15 +05:30
Srikanth Chekuri
b9f4fcd681 chore(telemetrytypes): introduce LogicalField (#12499)
Some checks failed
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
Possible options

1. The compatibility keys maps (the approach already in the code).

`backward_compat_keys.go` makes an alias key at metadata time. We
rejected this option because of evidence. The alias key resolves, but it
reads the wrong data. It prepares to `attributes_string['<alias>']`, and
that physical key does not hold the data.

2. The flat multi-key.

`GetKeys` returns multiple keys in order, and the downstream code uses
the list. The option fails on semantics. It removes one piece of
information that the downstream must have. The downstream must know the
difference between two cases:

- Two keys are the same field with two spellings so we can merge them
into one expression.
- Two keys are different fields with the same name. The condition
builder must make one condition for each key. The operator connects the
conditions.

Three failures show the problem:

- Negative operators connect with OR across the keys. A row that has
only one spelling then always matches. Example: `env != 'prod'` matches
each row that does not have one of the two keys.
- A row that has both spellings with different values gets no clear
result.
- A value position (group-by, select) needs exactly one expression for
one field. A flat list cannot point to that expression.

The information must live somewhere.

3. Annotations on `TelemetryFieldKey`

Maintain the `SemconvMembers` and `SemconvMaterializedColumns` fields on
the keys. The information is the same as in option 4. But it's awkward
because "these N keys are one family, in this order" lives in N copies,
one copy on each key.

4. introduce `LogicalField`

The information is the same as in option 3, but the structure holds it:

- The slice is the ambiguity.
- The group is the family.
- The member order is the precedence. The code sorts the members one
time, by family rank, at construction.
- The members point to the metadata entries. The code copies nothing and
changes nothing.
- The identity (signal, context, data type) is on the group. A merge
across contexts or data types is not possible. The design does not avoid
that merge; the design cannot express it.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 13:40:37 +00:00
Aditya Singh
0dd9a156b9 feat(log-details): add highlights section to log details drawer [2/3] (#12425)
## Pull Request

---

### 📄 Summary
> Why does this change exist?  
> What problem does it solve, and why is this the right approach?

**What it does:** Slice 2 of the log-details drawer revamp, a new
**Highlights** row at the
top of the drawer that surfaces a log's key fields as chips. Gated
behind `isLogDetailsV2`
(ships off). Stacked on the header PR (#12310 /
`feat/log-detail-revamp`); the DataViewer
lands in the next PR.

**Change points**

- New Highlight section added. Check screenshot
- Driven by config.
- Severity chip color
- Trace id click opens trace details page in new tab
- Tests updated


#### Screenshots / Screen Recordings (if applicable)
<img width="2158" height="834" alt="image"
src="https://github.com/user-attachments/assets/9b9af5b0-8437-4a6d-8b34-e97b0d55de26"
/>

<img width="2308" height="920" alt="image"
src="https://github.com/user-attachments/assets/69e3702f-cc25-4570-87d2-67a000925705"
/>



#### Issues closed by this PR
> Reference issues using `Closes #issue-number` to enable automatic
closure on merge.

---

###  Change Type
_Select all that apply_

- [x]  Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only

---

### 🐛 Bug Context
> Required if this PR fixes a bug

#### Root Cause
> What caused the issue?  
> Regression, faulty assumption, edge case, refactor, etc.

#### Fix Strategy
> How does this PR address the root cause?

---

### 🧪 Testing Strategy
> How was this change validated?

- Tests added/updated:
- Manual verification:
- Edge cases covered:

---

### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?

- Blast radius:
- Potential regressions:
- Rollback plan:

---

### 📝 Changelog
> Fill only if this affects users, APIs, UI, or documented behavior  
> Use **N/A** for internal or non-user-facing changes

| Field | Value |
|------|-------|
| Deployment Type | Cloud / OSS / Enterprise |
| Change Type | Feature / Bug Fix / Maintenance |
| Description | User-facing summary |

---

### 📋 Checklist
- [ ] Tests added or explicitly not required
- [ ] Manually tested
- [ ] Breaking changes documented
- [ ] Backward compatibility considered

---

## 👀 Notes for Reviewers

<!-- Anything reviewers should keep in mind while reviewing -->

---
2026-08-10 12:05:35 +00:00
Pandey
f44d6c7c84 docs(contributing): document the kind/spec envelope for sum types (#12494)
#### Description

- Documents the kind/spec envelope pattern for sum types in
`docs/contributing/go/types.md`: the envelope shape, why it goes at the
point of variance rather than the resource root, the tagging-style
rationale (adjacently tagged vs internally tagged vs sibling optional
fields), the validating `UnmarshalJSON`, the OpenAPI variant structs,
and the data-migration-vs-storable-twin trade-off for legacy persisted
shapes.
- Examples are generic (`FooConfig` with `bar`/`baz` kinds), with
`RuleThresholdData`, `EvaluationEnvelope` and the dashboard plugins as
the in-tree references.
- Cross-links from `handler.md`'s "`oneOf` with a discriminator"
section, which keeps owning the schema mechanics.
2026-08-10 11:22:33 +00:00
Gaurav Tewari
90d280d871 feat: add ai explorer tab (#12484)
#### Description

So in this PR , we have introduced a new rapper for AI o11y explorer .
we will be introducing new changes inside this tab.

#### Issues closed by this PR


https://github.com/orgs/SigNoz/projects/39/views/11?filterQuery=assignee%3Atewarig&pane=issue&itemId=223105731&issue=SigNoz%7Cengineering-pod%7C5842

#### Screenshots / Screen Recordings

This is just a demo of Shell -


https://github.com/user-attachments/assets/f647def2-f43d-46e7-850c-98330a7232b7



#### Additional Information

We have enabled this Shell now , since anyway. ai o11y is behind a flag.

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-08-10 10:20:18 +00:00
Gaurav Tewari
1ab44d4244 feat(integrations): add GCP cloud integration (#12210)
## Pull Request

---

### 📄 Summary
> Why does this change exist?
> What problem does it solve, and why is this the right approach?

Adds **Google Cloud Platform** as a cloud integration, bringing GCP to
parity with the existing AWS and Azure integrations. Users can now
connect a GCP account and manage which projects are monitored, directly
from the Integrations UI.

The integration reuses the shared cloud-integration surfaces (services
list, service details, account actions, header) and adds the
GCP-specific data-entry flows on top

doc -
https://app.notion.com/p/signoz/GCP-Integration-frontend-requirements-39cfcc6bcd1980b39428e1e66a77893b#39cfcc6bcd19808aa9bccef55661bf9d

Artifacts-
https://claude.ai/design/p/3591c3c0-3cc5-42ca-9ff9-dc8698e4d4dc?file=GCP+Integration.dc.html



The commits are stacked so that **each one type-checks, builds, and runs
on its own** — you can check out any commit and use the GCP page in the
state that commit describes:

1. c5f550d88 — **GCP page, runnable end-to-end minus the drawers**: type
definitions, constants, DTO→UI account mapping, logo, and the wiring
into the shared integration UI (`IntegrationDetailPage` routes the `gcp`
id to `CloudIntegration`, `AccountActions` maps GCP accounts,
`ServiceDetails` handles the GCP service config shape, remove-account
copy). Also narrows the Azure config checks to `resource_groups`, since
the widened config union means `deployment_region` no longer uniquely
identifies Azure. At this commit the GCP page renders and lists
accounts/services; the add-/edit-account buttons are intentionally
no-ops.
2. dc4b4c3d0 — **Add-account drawer**: the cloud account setup drawer
(flow selector, connection secret fields, field validators, and the
`useCloudAccountSetupDrawer` orchestration hook), wired into
`AccountActions`. Secret fields use the shared periscope `CopyButton`,
which gains an optional `onCopy` callback for the toast.
3. 4f45b3a69 — **Edit-account drawer**: the account settings drawer for
an already-connected GCP account, wired into `AccountActions`.
4. 4f3fe7ec6 — **Fix connection-status polling**:
`isOneClickIntegration` only listed AWS and Azure, so
`IntegrationDetailPage` treated GCP as a legacy integration and polled
`GET /integrations/gcp/connection_status` every 5s via
`useGetIntegrationStatus`. GCP has no such legacy status endpoint, so
it's added to the allowlist to disable the poll — bringing it to parity
with AWS/Azure.


#### Screenshots / Screen Recordings (if applicable)
> Include screenshots or screen recordings that clearly show the
behavior before the change and the result after the change.




https://github.com/user-attachments/assets/7bd5f9d3-85ff-452d-85f4-47eb95c0cce6



#### Issues closed by this PR


https://github.com/orgs/SigNoz/projects/39/views/11?filterQuery=assignee%3Atewarig&pane=issue&itemId=212300300&issue=SigNoz%7Cengineering-pod%7C5684

N/A

---

###  Change Type
_Select all that apply_

- [x]  Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only

---

### 🧪 Testing Strategy
> How was this change validated?

- Tests added/updated: 
Will add tests in follow up

---

### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?

- Blast radius: Scoped to the Cloud Integrations feature. Shared
components (`AccountActions`, `ServiceDetails`,
`RemoveIntegrationAccount`) gained GCP branches guarded by `type ===
GCP_SERVICES`, so AWS/Azure paths are unaffected.
- Potential regressions: The Azure config-narrowing change is the only
edit to existing Azure behavior; verified the Azure edit modal still
type-checks and reads `resource_groups`/`deployment_region` correctly.
- Rollback plan: Revert the PR — GCP is additive and gated by provider
type, so removal leaves AWS/Azure untouched.

---

### 📝 Changelog

| Field | Value |
|------|-------|
| Deployment Type | Cloud |
| Change Type | Feature |
| Description | Added Google Cloud Platform cloud integration: connect a
GCP account and manage monitored projects from the Integrations UI. |

---

### 📋 Checklist
- [x] Tests added or explicitly not required
- [x] Manually tested
- [x] Breaking changes documented (none — additive)
- [x] Backward compatibility considered

---

## 👀 Notes for Reviewers

> The PR is best reviewed commit by commit — each commit is
independently green (tsc, lint, build) and runnable, so you can check
out any of the three and exercise the GCP page at that stage.

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-08-10 10:18:54 +00:00
104 changed files with 4405 additions and 343 deletions

View File

@@ -1,9 +0,0 @@
# Contribution guidelines
- When making Go changes, always ensure they follow the contributing guildelines in [`docs/contributing/go/`](../../docs/contributing/go/).
- If any API contract is modified, generate the OpenAPI specs with `make gen-openapi-specs`.
- Always keep the OpenAPI spec generated in a separate commit, so the whole commit can be dropped in case of conflicts during merge. Do not try to resolve conflict in generated files, instead just generate them again.
- Avoid breaking function calls unncessarily into multilines for couple of arguments.
- Try to keep most computational only logic in types package itself related to a domain type, use modules as the orchestraction layer cordinating different layers and all db queries in store layer. Check the serviceaccount modules for inspiration when confused.
- When defining types, keep the structure of file to have any constants and variables first, then exported types and exported methods and then finally the unexported types and methods.
- Never import types or other modules in migration files, duplicate the required type or method to keep migration free from changes.

View File

@@ -2,9 +2,6 @@
- **Follow the template** (`.github/pull_request_template.md`): fill in its headings (Description / Issues closed by this PR / Screenshots / Additional Information). Don't add sections the template doesn't have.
- **Keep only the headings that apply.** Delete every heading that has nothing under it, along with its `<!--...-->` placeholder comment. The body must never contain an empty heading — if only Description applies, the body has exactly that one heading.
- **Keep the description concise and human-readable.** A few non repeatative bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate and not the user agent conversation details.
- **Keep the description concise and human-readable.** A few plain bullets saying what changed and why, for a reviewer skimming it — not a wall of text, not a restatement of the diff, not generated boilerplate.
- **Reference issues with `Closes #issue-number`** under "Issues closed by this PR" so they auto-close on merge. This goes in the PR description only — never in commit messages.
- **Breaking changes can be added in additional information section** if any.
- **AI assistance in commits may optionally be disclosed with an `Assisted-by:` trailer** naming the model (e.g. `Assisted-by: Claude Opus 4.5`) — do NOT use a `Co-authored-by:` trailer for this.
- **Keep the commit body short and human readable** focused on decision made if any. Commit body must not re-iterate the changes done, skip if title is sufficient in conveying the change.
- **Use convensional commit format** for commits and PR title.

1
.gitignore vendored
View File

@@ -232,4 +232,3 @@ pyrightconfig.json
# dev
.dev/
.claude/worktrees/
.claude/settings.local.json

View File

@@ -1,10 +0,0 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"
staged_go_files=$(git diff --cached --name-only --diff-filter=ACM -- '*.go')
if [ -n "$staged_go_files" ]; then
echo "$staged_go_files" | xargs gofmt -l -w
echo "$staged_go_files" | xargs git add
fi
cd frontend && pnpm lint-staged

View File

@@ -81,20 +81,16 @@ devenv-clickhouse-clean: ## Clean all ClickHouse data from filesystem
##############################################################
# go commands
##############################################################
SQLITE_PATH ?= signoz.db
SIGNOZ_APISERVER_ADDRESS ?= 0.0.0.0:8080
.PHONY: go-run-enterprise
go-run-enterprise: ## Runs the enterprise go backend server
@SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=debug \
SIGNOZ_SQLSTORE_SQLITE_PATH=$(SQLITE_PATH) \
SIGNOZ_SQLSTORE_SQLITE_PATH=signoz.db \
SIGNOZ_WEB_ENABLED=false \
SIGNOZ_TOKENIZER_JWT_SECRET=secret \
SIGNOZ_ALERTMANAGER_PROVIDER=signoz \
SIGNOZ_TELEMETRYSTORE_PROVIDER=clickhouse \
SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=tcp://127.0.0.1:9000 \
SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_CLUSTER=cluster \
SIGNOZ_APISERVER_ADDRESS=$(SIGNOZ_APISERVER_ADDRESS) \
go run -race \
$(GO_BUILD_CONTEXT_ENTERPRISE)/*.go server
@@ -105,29 +101,16 @@ go-test: ## Runs go unit tests
.PHONY: go-run-community
go-run-community: ## Runs the community go backend server
@SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=debug \
SIGNOZ_SQLSTORE_SQLITE_PATH=$(SQLITE_PATH) \
SIGNOZ_SQLSTORE_SQLITE_PATH=signoz.db \
SIGNOZ_WEB_ENABLED=false \
SIGNOZ_TOKENIZER_JWT_SECRET=secret \
SIGNOZ_ALERTMANAGER_PROVIDER=signoz \
SIGNOZ_TELEMETRYSTORE_PROVIDER=clickhouse \
SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=tcp://127.0.0.1:9000 \
SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_CLUSTER=cluster \
SIGNOZ_APISERVER_ADDRESS=$(SIGNOZ_APISERVER_ADDRESS) \
go run -race \
$(GO_BUILD_CONTEXT_COMMUNITY)/*.go server
.PHONY: go-stop
go-stop: ## Stops the go backend server listening on SIGNOZ_APISERVER_ADDRESS
@PORT=$(lastword $(subst :, ,$(SIGNOZ_APISERVER_ADDRESS))); \
PIDS=$$(lsof -ti tcp:$$PORT); \
if [ -n "$$PIDS" ]; then \
kill $$PIDS; \
echo "Stopped signoz server on port $$PORT (pid $$PIDS)"; \
else \
echo "No signoz server running on port $$PORT."; \
echo "If it's running on a different port, rerun as: make go-stop SIGNOZ_APISERVER_ADDRESS=host:port"; \
fi
.PHONY: go-build-community $(GO_BUILD_ARCHS_COMMUNITY)
go-build-community: ## Builds the go backend server for community
go-build-community: $(GO_BUILD_ARCHS_COMMUNITY)
@@ -258,8 +241,3 @@ semconv-generate: ## Regenerate semantic-convention families for Go and TypeScri
gen-mocks:
@echo ">> Generating mocks"
@mockery --config .mockery.yml
.PHONY: gen-openapi-specs
gen-openapi-specs:
@go run cmd/enterprise/*.go generate openapi
cd frontend && pnpm generate:api && cd -

View File

@@ -138,8 +138,6 @@ sqlstore:
##################### APIServer #####################
apiserver:
# The TCP address the API server listens on, in the form "host:port".
address: 0.0.0.0:8080
timeout:
# Default request timeout.
default: 60s

View File

@@ -349,7 +349,7 @@ func (Step) JSONSchema() (jsonschema.Schema, error) {
### `oneOf` with a discriminator
For a sum type whose variants are keyed by a property (e.g. `kind`), expose the variants via `JSONSchemaOneOf()` and add a discriminator. Without it, code generators intersect the variants (`A & B & C`) instead of producing a clean discriminated union (`A | B | C`).
For a sum type whose variants are keyed by a property (e.g. `kind`), expose the variants via `JSONSchemaOneOf()` and add a discriminator. Without it, code generators intersect the variants (`A & B & C`) instead of producing a clean discriminated union (`A | B | C`). How to model the sum type itself is covered in [types.md](types.md#sum-types-the-kindspec-envelope) — this section is only about its schema.
The parent keeps its `JSONSchemaOneOf()` (the `oneOf` itself) and *additionally* tags it via `PrepareJSONSchema` with the `x-signoz-discriminator` extension; `signoz.attachDiscriminators` then promotes that marker to a real OpenAPI 3 `discriminator` (and strips the duplicate parent properties) after reflection.

View File

@@ -99,6 +99,69 @@ Each flavor exists for a concrete reason:
The core `AuthDomain` holds the two live halves — `storableAuthDomain` and `authDomainConfig` — and owns business methods such as `Update(config)`. Conversions use the `New<Output>From<Input>` form: `NewAuthDomainFromConfig`, `NewAuthDomainFromStorableAuthDomain`, `NewGettableAuthDomainFromAuthDomain`.
## Sum types: the kind/spec envelope
When a domain type is a *sum type* — exactly one of several variants, selected by a discriminator — model it as an envelope with a `kind` and a `spec`:
```go
type FooConfig struct {
Kind FooKind `json:"kind" required:"true"`
Spec any `json:"spec" required:"true"`
}
```
```json
{ "kind": "bar", "spec": { "url": "...", "timeout": "30s" } }
```
`Kind` is a `valuer.String` enum implementing `Enum()`; `Spec` holds exactly one concrete variant type (`BarSpec`, `BazSpec`, …). `RuleThresholdData` and `EvaluationEnvelope` in `pkg/types/ruletypes/` are the canonical in-tree examples; the dashboard panel/query/variable plugins in `pkg/types/dashboardtypes/` are the same pattern behind generics. (`QueryEnvelope` in querybuildertypes uses `type` as the discriminator key for historical reasons; new envelopes use `kind`.)
### The envelope goes at the point of variance, not the resource root
Put the envelope on the field that actually varies. The resource root is almost never a sum type — a `Foo` has a `name` and an `enabled` flag regardless of which kind it is configured with; only its configuration varies, so the envelope is the `config` field:
```json
{ "name": "my-foo", "enabled": true, "config": { "kind": "bar", "spec": { "...": "..." } } }
```
Hoisting `kind`/`spec` to the root would turn the whole resource into a `oneOf`: every flavor (`PostableFoo`, `UpdatableFoo`, `GettableFoo`) then needs one variant schema per kind, each repeating the common fields; every new common field has to be added to all of them; and generated clients get unions of large objects instead of one small union that narrows on `config.kind`. A root-level `kind` also collides with the resource-model meaning of the word — root `kind` conventionally answers "what resource is this" (`Dashboard`), never "which flavor of config does it hold".
The existing domains already follow this placement:
- **Rules** — plain root; envelopes on the varying fields: `thresholds: {kind, spec}` and `evaluation: {kind, spec}`.
- **Dashboards** — metadata at the root plus one typed `spec`; the unions sit deep inside, at each panel/query/variable plugin (`{kind, spec}` in `perses_plugin_wrappers.go`).
- **Saved views** — root `{schemaVersion, spec}`, where `spec` is a *versioning* envelope holding one fixed type, not a union; the unions are inside it (`spec.queries: [{type, spec}]`). Same word, different job — a versioned body is not a discriminated union.
### Why this tagging style
Of the union encodings in common use, the envelope is the *adjacently tagged* one — tag and payload side by side. Variant payloads stay collision-free, and each kind maps to a named wrapper schema that carries the discriminator, which is exactly what OpenAPI generators need. The alternatives lose on those points: *internally tagged* (`{"kind": "bar", ...fields flattened}`) mixes common and variant fields, admits cross-variant key collisions, and forces every variant schema to redeclare the discriminator; *sibling optional fields* (`{"kind": "bar", "barConfig": {}, "bazConfig": {}}`) is the anti-pattern the first rule below exists to prevent.
The rules that make the envelope work:
- **Never model variants as sibling fields.** A struct with `Bar *BarSpec`, `Baz *BazSpec` next to a discriminator cannot be expressed as an OpenAPI discriminated union, forces nilability checks on every consumer, and silently admits contradictory payloads (kind=bar with a baz spec). The chosen variant *is* the payload.
- **The envelope owns `UnmarshalJSON`.** Decode `kind` first, then switch on it to decode and validate the matching concrete type into `Spec`. Unknown kinds and missing specs are rejected at the boundary:
```go
func (typ *FooConfig) UnmarshalJSON(data []byte) error {
var raw map[string]json.RawMessage
// ... unmarshal raw, decode raw["kind"] ...
switch kind {
case FooKindBar:
spec := BarSpec{}
if err := json.Unmarshal(raw["spec"], &spec); err != nil {
return err
}
typ.Spec = spec
// ... one case per kind, default rejects ...
}
typ.Kind = kind
return nil
}
```
- **Consumers type-assert on `Spec`** (`config.Spec.(BarSpec)`) after switching on `Kind`. If assertion sites multiply, add typed accessors on the envelope (see `EvaluationEnvelope.GetEvaluation()`).
- **OpenAPI needs one unexported variant struct per kind** (`fooConfigBar{Kind; Spec BarSpec}`), exposed via `JSONSchemaOneOf()` and mapped via `PrepareJSONSchema` with the `x-signoz-discriminator` extension. The schema mechanics are covered in [handler.md](handler.md#oneof-with-a-discriminator).
- **A legacy persisted shape gets a data migration or a `StorableX`.** When rows were written before the envelope existed, prefer an idempotent `sqlmigration` that rewrites them into the new shape, so the storable type simply nests the envelope. Only when the old shape must keep being written (external writers, rollback windows) keep it in a storable twin and convert at the type boundary.
## Conventions that tie the flavors together
- **Conversions** use either a `New<Output>From<Input>` constructor — e.g. `NewChannelFromReceiver`, `NewGettableAuthDomainFromAuthDomain` — or a receiver-style `ToY()` method. Both forms coexist in the codebase; use whichever fits the call site.
@@ -139,6 +202,8 @@ Both are optional. Do not introduce them if `PostableX` already covers the case.
- Every domain package defines the core type `X`. Only `X` is mandatory.
- Add `PostableX` / `GettableX` / `UpdatableX` / `StorableX` one at a time, only when the shape actually diverges from `X`.
- Model sum types as a `{kind, spec}` envelope with a validating `UnmarshalJSON` — never as sibling variant fields next to a discriminator.
- The envelope goes on the field that varies, never at the resource root — common fields stay on the resource, outside the union.
- Domain logic lives on `X`, not on the flavor types.
- Conversions can be a `New<Output>From<Input>` constructor or a receiver-style `ToY()` method — pick whichever reads best at the call site.
- Use a type alias when two shapes are truly identical.

View File

@@ -130,7 +130,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
s := &Server{
config: config,
signoz: signoz,
httpHostPort: config.APIServer.Address,
httpHostPort: baseconst.HTTPHostPort,
unavailableChannel: make(chan healthcheck.Status),
usageManager: usageManager,
}
@@ -235,7 +235,7 @@ func (s *Server) initListeners() error {
var err error
publicHostPort := s.httpHostPort
if publicHostPort == "" {
return fmt.Errorf("apiserver.address is required")
return fmt.Errorf("baseconst.HTTPHostPort is required")
}
s.httpConn, err = net.Listen("tcp", publicHostPort)

4
frontend/.husky/pre-commit Executable file
View File

@@ -0,0 +1,4 @@
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"
cd frontend && pnpm lint-staged

View File

@@ -20,7 +20,7 @@
"jest:coverage": "jest --coverage",
"jest:watch": "jest --watch",
"postinstall": "pnpm i18n:generate-hash && (is-ci || pnpm husky:configure) && node scripts/update-registry.cjs",
"husky:configure": "cd .. && husky install .husky && chmod ug+x .husky/*",
"husky:configure": "cd .. && husky install frontend/.husky && cd frontend && chmod ug+x .husky/*",
"commitlint": "commitlint --edit $1",
"test": "jest",
"test:changedsince": "jest --changedSince=main --coverage --silent",

View File

@@ -527,6 +527,13 @@ const routes: AppRoutes[] = [
key: 'AI_OBSERVABILITY_OVERVIEW',
isPrivate: true,
},
{
path: ROUTES.AI_OBSERVABILITY_EXPLORER,
exact: true,
component: LLMObservabilityPage,
key: 'AI_OBSERVABILITY_EXPLORER',
isPrivate: true,
},
{
path: ROUTES.AI_OBSERVABILITY_CONFIGURATION,
exact: true,

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 128 128"><path fill="#ea4535" d="M80.6 40.3h.4l-.2-.2 14-14v-.3c-11.8-10.4-28.1-14-43.2-9.5C36.5 20.8 24.9 32.8 20.7 48c.2-.1.5-.2.8-.2 5.2-3.4 11.4-5.4 17.9-5.4 2.2 0 4.3.2 6.4.6.1-.1.2-.1.3-.1 9-9.9 24.2-11.1 34.6-2.6h-.1z"/><path fill="#557ebf" d="M108.1 47.8c-2.3-8.5-7.1-16.2-13.8-22.1L80 39.9c6 4.9 9.5 12.3 9.3 20v2.5c16.9 0 16.9 25.2 0 25.2H63.9v20h-.1l.1.2h25.4c14.6.1 27.5-9.3 31.8-23.1 4.3-13.8-1-28.8-13-36.9z"/><path fill="#36a852" d="M39 107.9h26.3V87.7H39c-1.9 0-3.7-.4-5.4-1.1l-15.2 14.6v.2c6 4.3 13.2 6.6 20.7 6.6z"/><path fill="#f9bc15" d="M40.2 41.9c-14.9.1-28.1 9.3-32.9 22.8-4.8 13.6 0 28.5 11.8 37.3l15.6-14.9c-8.6-3.7-10.6-14.5-4-20.8 6.6-6.4 17.8-4.4 21.7 3.8L68 55.2C61.4 46.9 51.1 42 40.2 42.1z"/></svg>

After

Width:  |  Height:  |  Size: 805 B

View File

@@ -23,6 +23,13 @@
font-weight: 400;
line-height: 20px; /* 142.857% */
letter-spacing: -0.07px;
.cloud-service-data-collected-table-heading-info {
display: inline-flex;
align-items: center;
color: var(--l3-foreground);
cursor: help;
}
}
.cloud-service-data-collected-table-logs {
@@ -32,3 +39,9 @@
}
}
}
.cloud-service-data-collected-table-tooltip {
max-width: 280px;
white-space: normal;
word-break: break-word;
}

View File

@@ -3,16 +3,19 @@ import {
CloudintegrationtypesCollectedLogAttributeDTO,
CloudintegrationtypesCollectedMetricDTO,
} from 'api/generated/services/sigNoz.schemas';
import { BarChart, ScrollText } from '@signozhq/icons';
import { BarChart, Info, ScrollText } from '@signozhq/icons';
import { TooltipProvider, TooltipSimple } from '@signozhq/ui/tooltip';
import './CloudServiceDataCollected.styles.scss';
function CloudServiceDataCollected({
logsData,
metricsData,
metricsInfoTooltip,
}: {
logsData: CloudintegrationtypesCollectedLogAttributeDTO[] | null | undefined;
metricsData: CloudintegrationtypesCollectedMetricDTO[] | null | undefined;
metricsInfoTooltip?: string;
}): JSX.Element {
const logsColumns = [
{
@@ -84,6 +87,25 @@ function CloudServiceDataCollected({
<div className="cloud-service-data-collected-table-heading">
<BarChart size={14} />
Metrics
{metricsInfoTooltip && (
<TooltipProvider>
<TooltipSimple
title={metricsInfoTooltip}
side="top"
tooltipContentProps={{
className: 'cloud-service-data-collected-table-tooltip',
}}
>
<span
className="cloud-service-data-collected-table-heading-info"
aria-label="About the metrics listed below"
data-testid="data-collected-metrics-info"
>
<Info size={12} />
</span>
</TooltipSimple>
</TooltipProvider>
)}
</div>
<Table
columns={metricsColumns}
@@ -97,4 +119,8 @@ function CloudServiceDataCollected({
);
}
CloudServiceDataCollected.defaultProps = {
metricsInfoTooltip: undefined,
};
export default CloudServiceDataCollected;

View File

@@ -0,0 +1,46 @@
.highlights {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 12px 16px;
padding: 12px 0;
// Constrain each KeyValueLabel (the grid items) to its cell.
:global(.key-value-label) {
width: auto;
min-width: 0;
overflow: hidden;
}
}
.valueBadge {
--badge-font-size: 13px;
box-sizing: border-box;
max-width: 100%;
min-width: 0;
}
// Truncating text inside a badge
.badgeText {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.serviceDot {
width: 6px;
height: 6px;
border-radius: 50%;
background: var(--accent-forest);
flex-shrink: 0;
margin-right: 4px;
}
.traceLink {
display: inline-block;
max-width: 100%;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--accent-primary);
}

View File

@@ -0,0 +1,36 @@
import KeyValueLabel from 'periscope/components/KeyValueLabel';
import { ILog } from 'types/api/logs/log';
import { LOG_HIGHLIGHTS } from './config';
import styles from './LogHighlights.module.scss';
interface LogHighlightsProps {
log: ILog;
}
function LogHighlights({ log }: LogHighlightsProps): JSX.Element | null {
const fields = LOG_HIGHLIGHTS.map((field) => ({
key: field.key,
label: field.label,
value: field.render(log),
})).filter((field) => field.value != null);
if (fields.length === 0) {
return null;
}
return (
<div className={styles.highlights} data-testid="log-details-highlights">
{fields.map((field) => (
<KeyValueLabel
key={field.key}
badgeKey={field.label}
badgeValue={field.value}
direction="column"
/>
))}
</div>
);
}
export default LogHighlights;

View File

@@ -0,0 +1,23 @@
import { Link } from 'react-router-dom';
import styles from './LogHighlights.module.scss';
interface TraceIdFieldProps {
traceId: string;
}
function TraceIdField({ traceId }: TraceIdFieldProps): JSX.Element {
return (
<Link
to={{ pathname: `/trace/${traceId}` }}
target="_blank"
rel="noreferrer"
className={styles.traceLink}
title={traceId}
>
{traceId}
</Link>
);
}
export default TraceIdField;

View File

@@ -0,0 +1,102 @@
import { ReactNode } from 'react';
import { Badge, BadgeColor } from '@signozhq/ui/badge';
import { LogType } from 'components/Logs/LogStateIndicator/LogStateIndicator';
import { getLogIndicatorType } from 'components/Logs/LogStateIndicator/utils';
import { ILog } from 'types/api/logs/log';
import styles from './LogHighlights.module.scss';
import TraceIdField from './TraceIdField';
// Severity badge color mirrors the LogStateIndicator bar
const SEVERITY_COLOR: Record<string, BadgeColor> = {
[LogType.TRACE]: 'forest',
[LogType.DEBUG]: 'aqua',
[LogType.INFO]: 'robin',
[LogType.WARN]: 'amber',
[LogType.ERROR]: 'cherry',
[LogType.FATAL]: 'sakura',
};
export interface LogHighlightConfig {
key: string;
label: string;
render: (log: ILog) => ReactNode | null;
}
// Resource/attribute lookup (keys like `service.name` live in resources_string,
// occasionally attributes_string). Typed loosely as these are string maps.
const getAttr = (log: ILog, key: string): string =>
(log.resources_string as unknown as Record<string, string>)?.[key] ||
(log.attributes_string as unknown as Record<string, string>)?.[key] ||
'';
const valueBadge = (
value: string,
options?: { prefix?: ReactNode; color?: BadgeColor },
): ReactNode => (
<Badge color={options?.color ?? 'vanilla'} className={styles.valueBadge}>
{options?.prefix}
<span className={styles.badgeText} title={value}>
{value}
</span>
</Badge>
);
export const LOG_HIGHLIGHTS: LogHighlightConfig[] = [
{
key: 'service',
label: 'SERVICE',
render: (log): ReactNode | null => {
const value = getAttr(log, 'service.name');
return value
? valueBadge(value, {
prefix: <span className={styles.serviceDot} />,
})
: null;
},
},
{
key: 'severity',
label: 'SEVERITY',
render: (log): ReactNode | null => {
if (!log.severity_text) {
return null;
}
return valueBadge(log.severity_text, {
color: SEVERITY_COLOR[getLogIndicatorType(log)] ?? 'vanilla',
});
},
},
{
key: 'namespace',
label: 'NAMESPACE',
render: (log): ReactNode | null => {
const value = getAttr(log, 'service.namespace');
return value ? valueBadge(value) : null;
},
},
{
key: 'environment',
label: 'ENVIRONMENT',
render: (log): ReactNode | null => {
const value = getAttr(log, 'deployment.environment');
return value ? valueBadge(value) : null;
},
},
{
key: 'traceId',
label: 'TRACE ID',
render: (log): ReactNode | null => {
const traceId = log.trace_id || log.traceId;
return traceId ? <TraceIdField traceId={traceId} /> : null;
},
},
{
key: 'spanId',
label: 'SPAN ID',
render: (log): ReactNode | null => {
const spanId = log.span_id || log.spanID;
return spanId ? valueBadge(spanId) : null;
},
},
];

View File

@@ -115,6 +115,45 @@ describe('LogDetail drawer — header (isLogDetailsV2)', () => {
expect(screen.queryByText('Open in Explorer')).not.toBeInTheDocument();
});
it('renders Highlights for fields present on the log, omitting absent ones', () => {
const logWithMeta = {
...mockLog,
severity_text: 'ERROR',
trace_id: 'trace-abc',
resources_string: {
'service.name': 'checkout',
'deployment.environment': 'production',
},
} as unknown as ILog;
renderDrawer({ log: logWithMeta });
const highlights = screen.getByTestId('log-details-highlights');
expect(highlights).toHaveTextContent('SEVERITY');
expect(highlights).toHaveTextContent('ERROR');
expect(highlights).toHaveTextContent('SERVICE');
expect(highlights).toHaveTextContent('checkout');
expect(highlights).toHaveTextContent('ENVIRONMENT');
expect(highlights).toHaveTextContent('production');
expect(highlights).toHaveTextContent('TRACE ID');
// Absent fields are omitted (no namespace / span id on this log).
expect(highlights).not.toHaveTextContent('NAMESPACE');
expect(highlights).not.toHaveTextContent('SPAN ID');
});
it('links the trace id highlight to the trace detail in a new tab', () => {
const logWithTrace = {
...mockLog,
trace_id: 'trace-abc',
} as unknown as ILog;
renderDrawer({ log: logWithTrace });
const link = screen.getByRole('link', { name: 'trace-abc' });
expect(link).toHaveAttribute('target', '_blank');
expect(link.getAttribute('href')).toContain('/trace/trace-abc');
});
it('navigates to the next / previous log with the Down / Up arrow keys', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
const logs = [makeLog('log-0'), makeLog('log-1'), makeLog('log-2')];

View File

@@ -55,6 +55,7 @@ import { isLogDetailsV2, RESOURCE_KEYS, VIEW_TYPES, VIEWS } from './constants';
import { LogDetailInnerProps, LogDetailProps } from './LogDetail.interfaces';
import LogDetailsHeader from './LogDetailsHeader/LogDetailsHeader';
import { useLogNavigation } from './LogDetailsHeader/useLogNavigation';
import LogHighlights from './LogHighlights/LogHighlights';
import './LogDetails.styles.scss';
@@ -399,6 +400,8 @@ function LogDetailInner({
<div className="log-overflow-shadow">&nbsp;</div>
</div>
{isLogDetailsV2 && <LogHighlights log={log} />}
<div className="tabs-and-search">
<ToggleGroupSimple
type="single"

View File

@@ -92,6 +92,7 @@ const ROUTES = {
AI_OBSERVABILITY_ATTRIBUTE_MAPPING: '/ai-observability/attribute-mapping',
AI_OBSERVABILITY_BASE: '/ai-observability',
AI_OBSERVABILITY_OVERVIEW: '/ai-observability/overview',
AI_OBSERVABILITY_EXPLORER: '/ai-observability/explorer',
AI_OBSERVABILITY_CONFIGURATION: '/ai-observability/configuration',
} as const;

View File

@@ -17,9 +17,12 @@ import { ChevronDown, Dot, PencilLine, Plug, Plus } from '@signozhq/icons';
import AzureCloudAccountSetupModal from '../../AzureCloudServices/AddNewAccount/CloudAccountSetupModal';
import AzureAccountSettingsModal from '../../AzureCloudServices/EditAccount/AccountSettingsModal';
import GcpCloudAccountSetupDrawer from '../../GoogleCloudPlatform/AddNewAccount/CloudAccountSetupDrawer';
import GcpAccountSettingsDrawer from '../../GoogleCloudPlatform/EditAccount/AccountSettingsDrawer';
import {
mapAccountDtoToAwsCloudAccount,
mapAccountDtoToAzureCloudAccount,
mapAccountDtoToGcpCloudAccount,
} from '../../mapCloudAccountFromDto';
import AwsCloudAccountSetupModal from '../AddNewAccount/CloudAccountSetupModal';
import AwsAccountSettingsModal from '../EditAccount/AccountSettingsModal';
@@ -156,6 +159,18 @@ function AccountActions({ type }: { type: IntegrationType }): JSX.Element {
});
}
if (type === IntegrationType.GCP_SERVICES) {
raw.forEach((account) => {
if (!account) {
return;
}
const mapped = mapAccountDtoToGcpCloudAccount(account);
if (mapped) {
mappedAccounts.push(mapped);
}
});
}
return mappedAccounts;
}, [listAccountsResponse, type]);
@@ -207,13 +222,23 @@ function AccountActions({ type }: { type: IntegrationType }): JSX.Element {
// log telemetry event when an account is viewed.
useEffect(() => {
if (activeAccount) {
const { config } = activeAccount;
let enabledRegions: string[];
if ('regions' in config) {
// AWS
enabledRegions = config.regions;
} else if ('resource_groups' in config) {
// Azure
enabledRegions = config.resource_groups;
} else {
// GCP
enabledRegions = config.project_ids;
}
logEvent(`${type} Integration: Account viewed`, {
cloudAccountId: activeAccount?.cloud_account_id,
status: activeAccount?.status,
enabledRegions:
'regions' in activeAccount.config
? activeAccount.config.regions
: activeAccount.config.resource_groups,
enabledRegions,
});
}
}, [activeAccount, type]);
@@ -260,6 +285,11 @@ function AccountActions({ type }: { type: IntegrationType }): JSX.Element {
onClose={(): void => setIsIntegrationModalOpen(false)}
/>
)}
{type === IntegrationType.GCP_SERVICES && (
<GcpCloudAccountSetupDrawer
onClose={(): void => setIsIntegrationModalOpen(false)}
/>
)}
</>
)}
@@ -281,6 +311,13 @@ function AccountActions({ type }: { type: IntegrationType }): JSX.Element {
setActiveAccount={setActiveAccount}
/>
)}
{type === IntegrationType.GCP_SERVICES && (
<GcpAccountSettingsDrawer
onClose={(): void => setIsAccountSettingsModalOpen(false)}
account={activeAccount}
setActiveAccount={setActiveAccount}
/>
)}
</>
)}
</div>

View File

@@ -46,14 +46,34 @@ const EMPTY_FORM_VALUES: ServiceConfigFormValues = {
s3BucketsByRegion: {},
};
const GCP_METRICS_INFO_TOOLTIP =
'These are suggested metrics for your OpenTelemetry Collector Configuration. The metrics you actually receive may vary based on the metrics listed in your collector config.';
function getIntegrationServiceConfig(
type: IntegrationType,
serviceDetailsData?: ServiceDetailsData,
):
| { logs?: { enabled?: boolean }; metrics?: { enabled?: boolean } }
| undefined {
const config = serviceDetailsData?.cloudIntegrationService?.config;
if (type === IntegrationType.AWS_SERVICES) {
return config?.aws;
}
if (type === IntegrationType.GCP_SERVICES) {
return config?.gcp;
}
return config?.azure;
}
function getInitialFormValues(
type: IntegrationType,
serviceDetailsData?: ServiceDetailsData,
): ServiceConfigFormValues {
const integrationConfig =
type === IntegrationType.AWS_SERVICES
? serviceDetailsData?.cloudIntegrationService?.config?.aws
: serviceDetailsData?.cloudIntegrationService?.config?.azure;
const integrationConfig = getIntegrationServiceConfig(
type,
serviceDetailsData,
);
return {
logsEnabled: integrationConfig?.logs?.enabled || false,
@@ -98,16 +118,21 @@ function getServiceConfigPayload({
};
}
return {
azure: {
logs: {
enabled: isLogsSupported ? logsEnabled : false,
},
metrics: {
enabled: isMetricsSupported ? metricsEnabled : false,
},
// Azure and GCP share the same simple logs/metrics enable-flag shape.
const signalConfig = {
logs: {
enabled: isLogsSupported ? logsEnabled : false,
},
metrics: {
enabled: isMetricsSupported ? metricsEnabled : false,
},
};
if (type === IntegrationType.GCP_SERVICES) {
return { gcp: signalConfig };
}
return { azure: signalConfig };
}
function ServiceDetails({
@@ -162,10 +187,10 @@ function ServiceDetails({
? isAccountServiceLoading
: isReadOnlyServiceLoading;
const integrationConfig =
type === IntegrationType.AWS_SERVICES
? serviceDetailsData?.cloudIntegrationService?.config?.aws
: serviceDetailsData?.cloudIntegrationService?.config?.azure;
const integrationConfig = getIntegrationServiceConfig(
type,
serviceDetailsData,
);
const isServiceEnabledInPersistedConfig =
Boolean(integrationConfig?.logs?.enabled) ||
Boolean(integrationConfig?.metrics?.enabled);
@@ -477,6 +502,11 @@ function ServiceDetails({
<CloudServiceDataCollected
logsData={serviceDetailsData?.dataCollected?.logs || []}
metricsData={serviceDetailsData?.dataCollected?.metrics || []}
metricsInfoTooltip={
type === IntegrationType.GCP_SERVICES
? GCP_METRICS_INFO_TOOLTIP
: undefined
}
/>
</div>
);

View File

@@ -36,8 +36,12 @@ function AccountSettingsModal({
const queryClient = useQueryClient();
// `account.config` is the shared per-provider union (Azure | AWS | GCP).
// Narrow to Azure by `resource_groups` (Azure-only) rather than
// `deployment_region`, which GCP also has — so it no longer identifies
// Azure uniquely.
const azureConfig = useMemo(
() => ('deployment_region' in account.config ? account.config : null),
() => ('resource_groups' in account.config ? account.config : null),
[account.config],
);

View File

@@ -0,0 +1,142 @@
.setupDrawer {
--dialog-header-padding: var(--spacing-10) var(--spacing-12);
--dialog-footer-padding: var(--spacing-8) var(--spacing-12);
// Input and ComboboxSimple default to --border borders, inherited text and
// --muted-foreground placeholders; the drawer wants the dimmer --l2-border with
// brighter values and duller placeholders. Backgrounds are left alone — both
// components default to transparent and every field sits on an --l2-background
// surface already. Focus borders stay at their per-component defaults.
--input-border-color: var(--l2-border);
--input-hover-border-color: var(--l2-border);
--input-foreground: var(--l1-foreground);
--input-placeholder-color: var(--l3-foreground);
--combobox-trigger-border-color: var(--l2-border);
// Bounded flex column so the header/footer stay put and only the body
// scrolls when content overflows.
display: flex;
flex-direction: column;
overflow: hidden;
[data-slot='drawer-header'],
[data-slot='drawer-footer'] {
flex-shrink: 0;
}
// The drawer body renders inside [data-slot='drawer-description'] — this is
// the only region allowed to scroll.
[data-slot='drawer-description'] {
display: flex;
flex: 1 1 auto;
flex-direction: column;
gap: var(--spacing-10);
min-height: 0;
padding: var(--spacing-10) var(--spacing-12);
overflow-y: auto;
}
[data-slot='select-content'] {
width: var(--radix-select-trigger-width);
}
[data-slot='combobox-content'] {
z-index: 5;
background: var(--l1-background);
border: 1px solid var(--l2-border);
border-radius: var(--radius-2);
}
// Selected region value: bright, like every other field's text.
[data-slot='combobox-value'] {
color: var(--l1-foreground);
}
// Empty trigger (.regionEmpty, set from the RHF value): dull the placeholder text.
.regionEmpty [data-slot='combobox-value'] {
color: var(--l3-foreground);
}
}
.title {
h3 {
margin: 0;
font-size: var(--periscope-font-size-medium);
font-weight: var(--font-weight-semibold);
}
}
.footerContainer {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
width: 100%;
}
.footer {
display: flex;
align-items: center;
justify-content: space-between;
width: 100%;
padding: 0;
}
.drawerSection {
composes: drawerSection from './shared.module.scss';
}
.mono {
composes: mono from './shared.module.scss';
}
.fullWidth {
width: 100%;
}
.fieldError {
composes: fieldError from './shared.module.scss';
}
.projectIdsSelect {
:global(.ant-select-selector) {
min-height: 36px;
background: var(--l2-background);
border: 1px solid var(--l2-border) !important;
}
&:hover :global(.ant-select-selector),
&:global(.ant-select-focused) :global(.ant-select-selector) {
border-color: var(--l2-border);
box-shadow: none;
}
// antd defaults to 14px; pin to 13px to line up with the Input/Combobox fields.
:global(.ant-select-selection-placeholder),
:global(.ant-select-selection-search-input),
:global(.ant-select-selection-item) {
font-size: var(--periscope-font-size-base);
}
:global(.ant-select-selection-placeholder) {
color: var(--l3-foreground);
}
:global(.ant-select-selection-search-input) {
color: var(--l1-foreground);
}
:global(.ant-select-selection-item) {
color: var(--l1-foreground);
background: var(--l2-background);
border: 1px solid var(--l2-border);
border-radius: var(--radius-2);
}
:global(.ant-select-selection-item-remove) {
color: var(--l3-foreground);
&:hover {
color: var(--l1-foreground);
}
}
}

View File

@@ -0,0 +1,288 @@
import { useEffect, useState } from 'react';
import { Button } from '@signozhq/ui/button';
import { Callout } from '@signozhq/ui/callout';
import { ComboboxSimple } from '@signozhq/ui/combobox';
import { DrawerWrapper } from '@signozhq/ui/drawer';
import { Input } from '@signozhq/ui/input';
import { Typography } from '@signozhq/ui/typography';
import { Select } from 'antd';
import cx from 'classnames';
import { GCP_REGIONS } from 'container/Integrations/constants';
import { IntegrationModalProps } from 'container/Integrations/HeroSection/types';
import { useCloudAccountSetupDrawer } from 'hooks/integration/gcp/useCloudAccountSetupDrawer';
import { Controller, useForm } from 'react-hook-form';
import { popupContainer } from 'utils/selectPopupContainer';
import ConnectionSecretsFields from './ConnectionSecretsFields';
import FieldLabel from './FieldLabel';
import FlowSelector from './FlowSelector';
import SetupGuideCallout from './SetupGuideCallout';
import { GcpSetupFormValues, SetupFlow } from './types';
import styles from './CloudAccountSetupDrawer.module.scss';
const REGION_ITEMS = GCP_REGIONS.map((region) => ({
value: region.value,
label: `${region.label} (${region.value})`,
}));
const DEFAULT_VALUES: GcpSetupFormValues = {
accountName: '',
deploymentProjectId: '',
deploymentRegion: '',
projectIds: [],
sigNozApiUrl: '',
sigNozApiKey: '',
ingestionUrl: '',
ingestionKey: '',
};
function CloudAccountSetupDrawer({
onClose,
}: IntegrationModalProps): JSX.Element {
const {
isLoading,
connectAccount,
handleClose,
connectionParams,
isConnectionParamsLoading,
submitError,
clearSubmitError,
} = useCloudAccountSetupDrawer({ onClose });
const { control, handleSubmit, setValue } = useForm<GcpSetupFormValues>({
defaultValues: DEFAULT_VALUES,
});
const [flow, setFlow] = useState<SetupFlow>('manual');
// Pre-fill the deployment/ingestion fields with the fetched credentials.
useEffect(() => {
if (!connectionParams) {
return;
}
setValue('sigNozApiUrl', connectionParams.sigNozApiUrl);
setValue('sigNozApiKey', connectionParams.sigNozApiKey);
setValue('ingestionUrl', connectionParams.ingestionUrl);
setValue('ingestionKey', connectionParams.ingestionKey);
}, [connectionParams, setValue]);
const footer = (
<div className={styles.footerContainer}>
{submitError && (
<Callout
type="error"
size="small"
showIcon
action="dismissible"
onClick={clearSubmitError}
title="Failed to connect GCP account"
testId="gcp-connect-error"
>
{submitError}
</Callout>
)}
<div className={styles.footer}>
<Button
variant="outlined"
color="secondary"
onClick={handleClose}
testId="gcp-cancel-btn"
>
Cancel
</Button>
<Button
variant="solid"
color="primary"
onClick={handleSubmit(connectAccount)}
loading={isLoading}
disabled={isConnectionParamsLoading}
testId="gcp-connect-account-btn"
>
Connect Account
</Button>
</div>
</div>
);
return (
<DrawerWrapper
open={true}
className={styles.setupDrawer}
onOpenChange={(open): void => {
if (!open) {
handleClose();
}
}}
direction="right"
showCloseButton
title="Connect Google Cloud Platform"
width="base"
footer={footer}
drawerHeaderProps={{ className: styles.title }}
>
<FlowSelector value={flow} onChange={setFlow} />
<SetupGuideCallout />
<div className={styles.drawerSection}>
<FieldLabel
htmlFor="gcp-account-name-input"
label="Account Name"
tooltip="A label to identify this group of GCP projects (org ID, billing email, or any descriptive name)"
required
/>
<Controller
name="accountName"
control={control}
rules={{ required: 'Please enter an account name' }}
render={({ field, fieldState }): JSX.Element => (
<>
<Input
id="gcp-account-name-input"
className={styles.fullWidth}
placeholder="e.g. my-org or billing@company.com"
value={field.value}
onChange={(e): void => field.onChange(e.target.value)}
testId="gcp-account-name-input"
/>
{fieldState.error && (
<Typography.Text
as="span"
size="small"
role="alert"
className={styles.fieldError}
>
{fieldState.error.message}
</Typography.Text>
)}
</>
)}
/>
</div>
<div className={styles.drawerSection}>
<FieldLabel
htmlFor="gcp-deployment-project-id-input"
label="Deployment Project ID"
tooltip="The GCP project that hosts your OTel Collector deployment — often separate from the projects you actually monitor"
required
/>
<Controller
name="deploymentProjectId"
control={control}
rules={{ required: 'Please enter the deployment project ID' }}
render={({ field, fieldState }): JSX.Element => (
<>
<Input
id="gcp-deployment-project-id-input"
className={cx(styles.fullWidth, styles.mono)}
placeholder="e.g. my-deployment-project-123"
value={field.value}
onChange={(e): void => field.onChange(e.target.value)}
testId="gcp-deployment-project-id-input"
/>
{fieldState.error && (
<Typography.Text
as="span"
size="small"
role="alert"
className={styles.fieldError}
>
{fieldState.error.message}
</Typography.Text>
)}
</>
)}
/>
</div>
<div className={styles.drawerSection}>
<FieldLabel
htmlFor="gcp-deployment-region-select"
label="Deployment Region"
tooltip="The GCP region where your OTel Collector will be deployed"
required
/>
<Controller
name="deploymentRegion"
control={control}
rules={{ required: 'Please select a region' }}
render={({ field, fieldState }): JSX.Element => (
<>
<ComboboxSimple
id="gcp-deployment-region-select"
className={cx(styles.fullWidth, {
[styles.regionEmpty]: !field.value,
})}
items={REGION_ITEMS}
value={field.value}
onChange={(value): void => field.onChange(value as string)}
placeholder="Select a region..."
inputPlaceholder="Search regions…"
withPortal={false}
testId="gcp-deployment-region-select"
/>
{fieldState.error && (
<Typography.Text
as="span"
size="small"
role="alert"
className={styles.fieldError}
>
{fieldState.error.message}
</Typography.Text>
)}
</>
)}
/>
</div>
<div className={styles.drawerSection}>
<FieldLabel
htmlFor="gcp-project-ids-select"
label="Projects to Monitor"
tooltip="Enter each GCP project ID then press Enter"
required
/>
<Controller
name="projectIds"
control={control}
rules={{
validate: (value): true | string =>
value.length > 0 || 'Please add at least one project ID',
}}
render={({ field, fieldState }): JSX.Element => (
<>
<Select
id="gcp-project-ids-select"
className={cx(styles.fullWidth, styles.projectIdsSelect)}
mode="tags"
value={field.value}
onChange={(value): void => field.onChange(value)}
placeholder="Add project IDs…"
tokenSeparators={[',', ' ']}
notFoundContent={null}
suffixIcon={null}
getPopupContainer={popupContainer}
data-testid="gcp-project-ids-select"
/>
{fieldState.error && (
<Typography.Text
as="span"
size="small"
role="alert"
className={styles.fieldError}
>
{fieldState.error.message}
</Typography.Text>
)}
</>
)}
/>
</div>
<ConnectionSecretsFields
control={control}
isLoading={isConnectionParamsLoading}
connectionParams={connectionParams}
/>
</DrawerWrapper>
);
}
export default CloudAccountSetupDrawer;

View File

@@ -0,0 +1,71 @@
.drawerSurface {
composes: drawerSurface from './shared.module.scss';
}
.drawerSurfaceHead {
composes: drawerSurfaceHead from './shared.module.scss';
}
.drawerSection {
composes: drawerSection from './shared.module.scss';
}
.headLabel {
display: flex;
justify-content: center;
align-items: center;
gap: var(--spacing-2);
}
.mono {
composes: mono from './shared.module.scss';
}
.fieldError {
composes: fieldError from './shared.module.scss';
}
.fullWidth {
width: 100%;
}
.secretsBody {
display: flex;
flex-direction: column;
gap: var(--spacing-6);
}
.skeletonLabel :global(.ant-skeleton-input) {
width: 120px;
min-width: 120px;
height: 14px;
}
.skeletonInput :global(.ant-skeleton-input) {
width: 100%;
min-width: 100%;
height: 36px;
}
.readonlyField {
display: flex;
gap: var(--spacing-2);
align-items: center;
height: 36px;
padding: 0 var(--spacing-2) 0 var(--spacing-4);
background: var(--l2-background);
border: 1px solid var(--l2-border);
border-radius: var(--radius-2);
}
.readonlyValue {
flex: 1 1 auto;
min-width: 0;
overflow: hidden;
// Match the other fields' value text: 13px and the brighter --l1-foreground.
font-size: var(--periscope-font-size-base);
color: var(--l2-foreground);
text-overflow: ellipsis;
white-space: nowrap;
cursor: not-allowed;
}

View File

@@ -0,0 +1,193 @@
import { Lock } from '@signozhq/icons';
import { Input } from '@signozhq/ui/input';
import { toast } from '@signozhq/ui/sonner';
import { Typography } from '@signozhq/ui/typography';
import { Skeleton } from 'antd';
import { CloudintegrationtypesCredentialsDTO } from 'api/generated/services/sigNoz.schemas';
import cx from 'classnames';
import CopyButton from 'periscope/components/CopyButton/CopyButton';
import { Control, Controller } from 'react-hook-form';
import FieldLabel from './FieldLabel';
import { GcpSetupFormValues } from './types';
import { SecretFieldType, validateSecretValue } from './validators';
import styles from './ConnectionSecretsFields.module.scss';
type CredentialField = keyof CloudintegrationtypesCredentialsDTO;
interface FieldConfig {
name: CredentialField;
label: string;
tooltip: string;
placeholder: string;
testId: string;
type: SecretFieldType;
}
const FIELDS: FieldConfig[] = [
{
name: 'sigNozApiUrl',
label: 'SigNoz API URL',
tooltip: 'Base URL of your SigNoz instance the collector reports to',
placeholder: 'https://<tenant>.signoz.cloud',
testId: 'gcp-signoz-api-url-input',
type: 'url',
},
{
name: 'sigNozApiKey',
label: 'SigNoz API Key',
tooltip: 'API key used to authenticate with your SigNoz instance',
placeholder: 'Enter SigNoz API key',
testId: 'gcp-signoz-api-key-input',
type: 'text',
},
{
name: 'ingestionUrl',
label: 'Ingestion URL',
tooltip: 'OTLP ingestion endpoint your OTel Collector sends telemetry to',
placeholder: 'https://ingest.<region>.signoz.cloud',
testId: 'gcp-ingestion-url-input',
type: 'url',
},
{
name: 'ingestionKey',
label: 'Ingestion Key',
tooltip: 'Ingestion key that authorizes telemetry sent to SigNoz',
placeholder: 'Enter ingestion key',
testId: 'gcp-ingestion-key-input',
type: 'text',
},
];
interface ConnectionSecretsFieldsProps {
control: Control<GcpSetupFormValues>;
isLoading: boolean;
connectionParams?: CloudintegrationtypesCredentialsDTO;
}
function ConnectionSecretsFields({
control,
isLoading,
connectionParams,
}: ConnectionSecretsFieldsProps): JSX.Element {
const hasMissingValue = FIELDS.some(
(field) => !connectionParams?.[field.name],
);
return (
<div className={styles.drawerSurface}>
<div className={styles.drawerSurfaceHead}>
<Typography.Text weight="bold" size="base">
Deployment details &amp; ingestion secrets
</Typography.Text>
{!hasMissingValue && (
<div className={styles.headLabel}>
<Lock size={12} />
<Typography.Text as="span" size="small" className={styles.headLabel}>
Auto-filled by SigNoz
</Typography.Text>
</div>
)}
</div>
{isLoading ? (
<div className={styles.secretsBody} data-testid="gcp-secrets-skeleton">
{FIELDS.map((field) => (
<div key={field.name} className={styles.drawerSection}>
<Skeleton.Input active size="small" className={styles.skeletonLabel} />
<Skeleton.Input active block className={styles.skeletonInput} />
</div>
))}
</div>
) : (
<div className={styles.secretsBody}>
{FIELDS.map((field) => {
// Backend-provided values are read-only — the user can't edit them, so
// show a truncated value with a copy button. Missing values (enterprise)
// stay editable inputs with no copy button.
const providedValue = connectionParams?.[field.name];
if (providedValue) {
return (
<div key={field.name} className={styles.drawerSection}>
<FieldLabel
htmlFor={field.testId}
label={field.label}
tooltip={field.tooltip}
/>
<div className={styles.readonlyField}>
<Typography.Text
as="span"
id={field.testId}
className={cx(styles.readonlyValue, styles.mono)}
title={providedValue}
testId={field.testId}
>
{providedValue}
</Typography.Text>
<CopyButton
value={providedValue}
size={12}
ariaLabel={`Copy ${field.label}`}
testId={`${field.testId}-copy`}
onCopy={(): void => {
toast.success(`${field.label} copied to clipboard`, {
position: 'bottom-right',
});
}}
/>
</div>
</div>
);
}
return (
<div key={field.name} className={styles.drawerSection}>
<FieldLabel
htmlFor={field.testId}
label={field.label}
tooltip={field.tooltip}
/>
<Controller
name={field.name}
control={control}
rules={{
validate: (value): true | string =>
validateSecretValue(field.label, field.type, value),
}}
render={({ field: rhfField, fieldState }): JSX.Element => (
<>
<Input
id={field.testId}
className={cx(styles.fullWidth, styles.mono)}
placeholder={field.placeholder}
value={rhfField.value}
onChange={(e): void => rhfField.onChange(e.target.value)}
testId={field.testId}
/>
{fieldState.error && (
<Typography.Text
as="span"
size="small"
role="alert"
className={styles.fieldError}
>
{fieldState.error.message}
</Typography.Text>
)}
</>
)}
/>
</div>
);
})}
</div>
)}
</div>
);
}
ConnectionSecretsFields.defaultProps = {
connectionParams: undefined,
};
export default ConnectionSecretsFields;

View File

@@ -0,0 +1,22 @@
.fieldLabel {
display: inline-flex;
align-items: center;
gap: var(--spacing-2);
}
.required {
composes: required from './shared.module.scss';
}
.infoTrigger {
display: inline-flex;
align-items: center;
color: var(--l3-foreground);
cursor: help;
}
.tooltipContent {
max-width: 240px;
white-space: normal;
word-break: break-word;
}

View File

@@ -0,0 +1,49 @@
import { Info } from '@signozhq/icons';
import { TooltipSimple } from '@signozhq/ui/tooltip';
import styles from './FieldLabel.module.scss';
interface FieldLabelProps {
htmlFor: string;
label: string;
tooltip: string;
required?: boolean;
}
function FieldLabel({
htmlFor,
label,
tooltip,
required,
}: FieldLabelProps): JSX.Element {
return (
<label className={styles.fieldLabel} htmlFor={htmlFor}>
{label}
<TooltipSimple
title={tooltip}
side="top"
tooltipContentProps={{ className: styles.tooltipContent }}
>
<span
className={styles.infoTrigger}
aria-label={`${label} help`}
data-testid={`${htmlFor}-tooltip`}
>
<Info size={12} />
</span>
</TooltipSimple>
{required && (
<span className={styles.required} aria-hidden="true">
*
</span>
)}
</label>
);
}
FieldLabel.defaultProps = {
required: false,
};
export default FieldLabel;

View File

@@ -0,0 +1,84 @@
.drawerSection {
composes: drawerSection from './shared.module.scss';
}
.drawerSurface {
composes: drawerSurface from './shared.module.scss';
}
.drawerSurfaceHead {
composes: drawerSurfaceHead from './shared.module.scss';
}
.flowRadioGroup {
--radio-group-item-border-color: var(--l2-border);
display: flex;
flex-direction: column;
gap: var(--spacing-4);
width: 100%;
.flowRadio {
display: flex;
flex-direction: row;
align-items: flex-start;
justify-content: flex-start;
gap: var(--spacing-5);
width: 100%;
padding: var(--spacing-5) var(--spacing-6);
margin: 0;
border: 1px solid transparent;
border-radius: var(--radius-2);
box-sizing: border-box;
cursor: pointer;
transition:
background-color 0.12s ease,
border-color 0.12s ease;
> button[role='radio'] {
flex: 0 0 16px;
width: 16px;
height: 16px;
margin-top: 3px;
}
> label {
flex: 1 1 auto;
min-width: 0;
display: block;
text-align: left;
cursor: pointer;
font-size: inherit;
font-weight: inherit;
color: inherit;
}
&.flowRadioManual:has(button[data-state='checked']) {
background: color-mix(in srgb, var(--accent-primary) 10%, transparent);
border-color: color-mix(in srgb, var(--accent-primary) 30%, transparent);
}
&:hover {
background: var(--l3-background-hover);
}
&:has(button[disabled]) {
cursor: not-allowed;
opacity: 0.55;
&:hover {
background: var(--l3-background);
}
}
}
}
.flowRadioTitle {
display: flex;
gap: var(--spacing-2);
align-items: center;
}
.flowRadioDesc {
margin-top: var(--spacing-2);
}

View File

@@ -0,0 +1,76 @@
import { Badge } from '@signozhq/ui/badge';
import { RadioGroup, RadioGroupItem } from '@signozhq/ui/radio-group';
import { Typography } from '@signozhq/ui/typography';
import cx from 'classnames';
import { SetupFlow } from './types';
import styles from './FlowSelector.module.scss';
interface FlowSelectorProps {
value: SetupFlow;
onChange: (flow: SetupFlow) => void;
}
function FlowSelector({ value, onChange }: FlowSelectorProps): JSX.Element {
return (
<div className={cx(styles.drawerSection, styles.drawerSurface)}>
<div className={styles.drawerSurfaceHead}>
<Typography.Text weight="bold" size="base">
Connection method
</Typography.Text>
</div>
<RadioGroup
value={value}
onChange={(next): void => onChange(next as SetupFlow)}
className={styles.flowRadioGroup}
>
<RadioGroupItem
value="manual"
containerClassName={cx(styles.flowRadio, styles.flowRadioManual)}
testId="gcp-flow-manual"
>
<div className={styles.flowRadioTitle}>
<Typography.Text weight="semibold" size="base">
Connect Manually
</Typography.Text>
</div>
<Typography.Text
as="p"
size="small"
color="muted"
className={styles.flowRadioDesc}
>
Deploy your own OTel Collector.
</Typography.Text>
</RadioGroupItem>
<RadioGroupItem
value="agent"
containerClassName={styles.flowRadio}
testId="gcp-flow-agent"
disabled
>
<div className={styles.flowRadioTitle}>
<Typography.Text weight="semibold" size="base">
Connect via Agent
</Typography.Text>
<Badge color="robin" variant="default">
Soon
</Badge>
</div>
<Typography.Text
as="p"
size="small"
color="muted"
className={styles.flowRadioDesc}
>
SigNoz deploys and manages the collector for you.
</Typography.Text>
</RadioGroupItem>
</RadioGroup>
</div>
);
}
export default FlowSelector;

View File

@@ -0,0 +1,13 @@
.guideLink {
display: inline-flex;
align-items: center;
gap: var(--spacing-2);
color: var(--callout-primary-title);
font-size: var(--periscope-font-size-base);
text-decoration: none;
&:hover {
color: var(--accent-primary);
text-decoration: underline;
}
}

View File

@@ -0,0 +1,31 @@
import { ArrowUpRight, KeyRound } from '@signozhq/icons';
import { Callout } from '@signozhq/ui/callout';
import { Typography } from '@signozhq/ui/typography';
import styles from './SetupGuideCallout.module.scss';
const GCP_INTEGRATION_DOCS_URL =
'https://signoz.io/docs/integrations/gcp/gcp-integration/';
function SetupGuideCallout(): JSX.Element {
return (
<Callout icon={<KeyRound />} testId="gcp-setup-guide-callout">
<Typography.Text as="span" size="base">
Please go through our GCP integration guide, which covers all prerequisites
service account, IAM roles, and resource setup.
</Typography.Text>
<a
className={styles.guideLink}
href={GCP_INTEGRATION_DOCS_URL}
target="_blank"
rel="noopener noreferrer"
data-testid="gcp-setup-guide-link"
>
GCP integration guide
<ArrowUpRight size={12} />
</a>
</Callout>
);
}
export default SetupGuideCallout;

View File

@@ -0,0 +1,36 @@
.drawerSection {
display: flex;
flex-direction: column;
gap: var(--spacing-3);
}
.drawerSection > label {
font-size: var(--periscope-font-size-normal);
color: var(--l2-foreground);
}
.required {
color: var(--accent-cherry);
}
.fieldError {
font-size: var(--periscope-font-size-small);
color: var(--accent-cherry);
}
.drawerSurface {
padding: var(--spacing-7);
border-radius: 6px;
border: 1px solid var(--l2-border);
}
.drawerSurfaceHead {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: var(--spacing-5);
}
.mono {
font-family: var(--font-family-mono);
}

View File

@@ -0,0 +1,12 @@
export type SetupFlow = 'manual' | 'agent';
export interface GcpSetupFormValues {
accountName: string;
deploymentProjectId: string;
deploymentRegion: string;
projectIds: string[];
sigNozApiUrl: string;
sigNozApiKey: string;
ingestionUrl: string;
ingestionKey: string;
}

View File

@@ -0,0 +1,24 @@
export type SecretFieldType = 'url' | 'text';
export function isValidUrl(value: string): boolean {
try {
return Boolean(new URL(value));
} catch {
return false;
}
}
export function validateSecretValue(
label: string,
type: SecretFieldType,
value: string | undefined,
): true | string {
const trimmed = value?.trim();
if (!trimmed) {
return `Please enter the ${label}`;
}
if (type === 'url' && !isValidUrl(trimmed)) {
return `Please enter a valid URL for ${label}`;
}
return true;
}

View File

@@ -0,0 +1,64 @@
.body {
display: flex;
flex-direction: column;
gap: 17px;
border-radius: 3px;
}
.connectedAccountDetails {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
}
.connectedAccountDetailsTitle {
color: var(--l1-foreground);
font-size: 14px;
font-weight: var(--font-weight-medium);
line-height: var(--line-height-20);
letter-spacing: -0.07px;
}
.accountId {
color: var(--l2-foreground);
font-size: 12px;
line-height: 18px;
letter-spacing: -0.06px;
}
.accountIdValue {
font-family: 'Geist Mono';
font-size: 12px;
font-weight: var(--font-weight-bold);
line-height: 18px;
letter-spacing: -0.06px;
}
.regionSelector {
display: flex;
flex-direction: column;
gap: var(--spacing-2);
}
.regionSelectorTitle {
color: var(--l1-foreground);
font-size: 14px;
font-weight: var(--font-weight-medium);
line-height: var(--line-height-20);
letter-spacing: -0.07px;
}
.regionSelectorDescription {
color: var(--l2-foreground);
font-size: 12px;
line-height: 18px;
letter-spacing: -0.06px;
}
.footer {
display: flex;
flex-direction: row;
gap: var(--spacing-5);
justify-content: space-between;
width: 100%;
}

View File

@@ -0,0 +1,156 @@
import { Dispatch, SetStateAction, useMemo } from 'react';
import { useQueryClient } from 'react-query';
import { Save } from '@signozhq/icons';
import { Button } from '@signozhq/ui/button';
import { DrawerWrapper } from '@signozhq/ui/drawer';
import { Form, Select } from 'antd';
import { invalidateListAccounts } from 'api/generated/services/cloudintegration';
import { INTEGRATION_TYPES } from 'container/Integrations/constants';
import { CloudAccount } from 'container/Integrations/types';
import { useAccountSettingsDrawer } from 'hooks/integration/gcp/useAccountSettingsDrawer';
import RemoveIntegrationAccount from '../../RemoveAccount/RemoveIntegrationAccount';
import styles from './AccountSettingsDrawer.module.scss';
interface AccountSettingsDrawerProps {
onClose: () => void;
account: CloudAccount;
setActiveAccount: Dispatch<SetStateAction<CloudAccount | null>>;
}
function AccountSettingsDrawer({
onClose,
account,
setActiveAccount,
}: AccountSettingsDrawerProps): JSX.Element {
const {
form,
isLoading,
projectIds,
isSaveDisabled,
setProjectIds,
handleSubmit,
handleClose,
} = useAccountSettingsDrawer({ onClose, account, setActiveAccount });
const queryClient = useQueryClient();
const gcpConfig = useMemo(
() => ('project_ids' in account.config ? account.config : null),
[account.config],
);
return (
<DrawerWrapper
open={true}
title="Account Settings"
direction="right"
showCloseButton
onOpenChange={(open): void => {
if (!open) {
handleClose();
}
}}
width="wide"
footer={
<div className={styles.footer}>
<RemoveIntegrationAccount
accountId={account?.id}
onRemoveIntegrationAccountSuccess={(): void => {
void invalidateListAccounts(queryClient, {
cloudProvider: INTEGRATION_TYPES.GCP,
});
setActiveAccount(null);
handleClose();
}}
cloudProvider={INTEGRATION_TYPES.GCP}
/>
<Button
variant="solid"
color="secondary"
disabled={isSaveDisabled}
onClick={handleSubmit}
loading={isLoading}
prefix={<Save size={14} />}
data-testid="gcp-update-account-btn"
>
Update Changes
</Button>
</div>
}
>
<Form
form={form}
layout="vertical"
initialValues={{
projectIds: gcpConfig?.project_ids || [],
}}
>
<div className={styles.body}>
<div className={styles.connectedAccountDetails}>
<div className={styles.connectedAccountDetailsTitle}>
Connected Account details
</div>
<div className={styles.accountId}>
Account Name:{' '}
<span className={styles.accountIdValue}>
{account?.providerAccountId}
</span>
</div>
</div>
{gcpConfig?.deployment_project_id && (
<div className={styles.regionSelector}>
<div className={styles.regionSelectorTitle}>Deployment project ID</div>
<div className={styles.regionSelectorDescription}>
{gcpConfig.deployment_project_id}
</div>
</div>
)}
{gcpConfig?.deployment_region && (
<div className={styles.regionSelector}>
<div className={styles.regionSelectorTitle}>Deployment region</div>
<div className={styles.regionSelectorDescription}>
{gcpConfig.deployment_region}
</div>
</div>
)}
<div className={styles.regionSelector}>
<div className={styles.regionSelectorTitle}>Projects to monitor</div>
<div className={styles.regionSelectorDescription}>
Update the GCP project IDs that should be monitored.
</div>
<Form.Item
name="projectIds"
rules={[
{
required: true,
type: 'array',
min: 1,
message: 'Please add at least one project ID',
},
]}
>
<Select
mode="tags"
value={projectIds}
tokenSeparators={[',']}
onChange={(values): void => {
setProjectIds(values);
form.setFieldValue('projectIds', values);
}}
data-testid="gcp-edit-project-ids-select"
/>
</Form.Item>
</div>
</div>
</Form>
</DrawerWrapper>
);
}
export default AccountSettingsDrawer;

View File

@@ -0,0 +1,230 @@
import { render, screen, waitFor, within } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { toast } from '@signozhq/ui/sonner';
import { TooltipProvider } from '@signozhq/ui/tooltip';
import { server } from 'mocks-server/server';
import { rest, RestRequest } from 'msw';
import MockQueryClientProvider from 'providers/test/MockQueryClientProvider';
import AccountSettingsDrawer from '../EditAccount/AccountSettingsDrawer';
import {
GCP_ACCOUNT_ID,
GCP_ACCOUNT_URL,
GCP_ACCOUNTS_URL,
gcpAccount,
gcpAccountConfig,
listAccountsResponse,
} from './mockData';
// `useAccountSettingsDrawer` imports logEvent by relative path, which the
// jest.config moduleNameMapper (keyed on the `api/common/logEvent` alias) does
// not intercept — so mock the resolved module directly.
jest.mock('../../../../../api/common/logEvent', () => ({
__esModule: true,
default: jest.fn(),
}));
jest.mock('@signozhq/ui/sonner', () => ({
...jest.requireActual('@signozhq/ui/sonner'),
toast: {
success: jest.fn(),
error: jest.fn(),
},
}));
const onClose = jest.fn();
const setActiveAccount = jest.fn();
const renderDrawer = (): void => {
render(
<MockQueryClientProvider>
<TooltipProvider>
<AccountSettingsDrawer
onClose={onClose}
account={gcpAccount}
setActiveAccount={setActiveAccount}
/>
</TooltipProvider>
</MockQueryClientProvider>,
);
};
/** The antd tags Select renders its text input inside the testId wrapper. */
const getProjectIdsInput = (): HTMLElement =>
within(screen.getByTestId('gcp-edit-project-ids-select')).getByRole(
'combobox',
);
/** Each selected tag carries an antd "close" icon that removes it. */
const getProjectIdTagRemoveButtons = (): HTMLElement[] =>
within(screen.getByTestId('gcp-edit-project-ids-select')).queryAllByLabelText(
'close',
);
describe('GCP AccountSettingsDrawer', () => {
let updatePayload: Record<string, unknown> | null;
beforeEach(() => {
updatePayload = null;
server.use(
rest.get(GCP_ACCOUNTS_URL, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(listAccountsResponse)),
),
rest.put(GCP_ACCOUNT_URL, async (req: RestRequest, res, ctx) => {
updatePayload = await req.json();
return res(ctx.status(204));
}),
);
});
it('renders the connected account details and existing project IDs', () => {
renderDrawer();
expect(screen.getByText(gcpAccount.providerAccountId)).toBeInTheDocument();
expect(
screen.getByText(gcpAccountConfig.deployment_project_id),
).toBeInTheDocument();
expect(
screen.getByText(gcpAccountConfig.deployment_region),
).toBeInTheDocument();
gcpAccountConfig.project_ids.forEach((projectId) => {
expect(screen.getByTitle(projectId)).toBeInTheDocument();
});
});
it('keeps save disabled until the project IDs actually change', async () => {
const user = userEvent.setup();
renderDrawer();
expect(screen.getByTestId('gcp-update-account-btn')).toBeDisabled();
await user.type(getProjectIdsInput(), 'project-c,');
await waitFor(() => {
expect(screen.getByTestId('gcp-update-account-btn')).toBeEnabled();
});
});
it('sends the updated project IDs while preserving the immutable deployment fields', async () => {
const user = userEvent.setup();
renderDrawer();
await user.type(getProjectIdsInput(), 'project-c,');
await waitFor(() => {
expect(screen.getByTestId('gcp-update-account-btn')).toBeEnabled();
});
await user.click(screen.getByTestId('gcp-update-account-btn'));
await waitFor(() => {
expect(updatePayload).not.toBeNull();
});
expect(updatePayload).toStrictEqual({
config: {
gcp: {
deploymentRegion: gcpAccountConfig.deployment_region,
deploymentProjectId: gcpAccountConfig.deployment_project_id,
projectIds: ['project-a', 'project-b', 'project-c'],
},
},
});
await waitFor(() => {
expect(setActiveAccount).toHaveBeenCalledWith({
...gcpAccount,
config: {
deployment_region: gcpAccountConfig.deployment_region,
deployment_project_id: gcpAccountConfig.deployment_project_id,
project_ids: ['project-a', 'project-b', 'project-c'],
},
});
});
expect(onClose).toHaveBeenCalledTimes(1);
expect(toast.success).toHaveBeenCalledWith(
'Account settings updated successfully',
expect.anything(),
);
});
it('blocks the update and shows a validation error when every project ID is removed', async () => {
const user = userEvent.setup();
renderDrawer();
// Strip every tag via its remove icon; the list shrinks as we go.
while (getProjectIdTagRemoveButtons().length > 0) {
// eslint-disable-next-line no-await-in-loop
await user.click(getProjectIdTagRemoveButtons()[0]);
}
await waitFor(() => {
expect(screen.getByTestId('gcp-update-account-btn')).toBeEnabled();
});
await user.click(screen.getByTestId('gcp-update-account-btn'));
await waitFor(() => {
expect(
screen.getByText('Please add at least one project ID'),
).toBeInTheDocument();
});
expect(updatePayload).toBeNull();
expect(onClose).not.toHaveBeenCalled();
});
it('surfaces a toast and keeps the drawer open when the update fails', async () => {
server.use(
rest.put(GCP_ACCOUNT_URL, (_req, res, ctx) =>
res(
ctx.status(500),
ctx.json({ status: 'error', error: { message: 'update failed' } }),
),
),
);
const user = userEvent.setup();
renderDrawer();
await user.type(getProjectIdsInput(), 'project-c,');
await waitFor(() => {
expect(screen.getByTestId('gcp-update-account-btn')).toBeEnabled();
});
await user.click(screen.getByTestId('gcp-update-account-btn'));
await waitFor(() => {
expect(toast.error).toHaveBeenCalledWith(
'Failed to update account settings',
expect.anything(),
);
});
expect(setActiveAccount).not.toHaveBeenCalled();
expect(onClose).not.toHaveBeenCalled();
});
it('disconnects the account with the GCP-specific confirmation copy', async () => {
let disconnectedId: string | null = null;
server.use(
rest.delete(`${GCP_ACCOUNTS_URL}/:id`, (req, res, ctx) => {
disconnectedId = req.params.id as string;
return res(ctx.status(204));
}),
);
const user = userEvent.setup();
renderDrawer();
await user.click(screen.getByRole('button', { name: /disconnect/i }));
await expect(
screen.findByText(/manually tear down/i),
).resolves.toBeInTheDocument();
await user.click(screen.getByRole('button', { name: /remove account/i }));
await waitFor(() => {
expect(disconnectedId).toBe(GCP_ACCOUNT_ID);
});
expect(setActiveAccount).toHaveBeenCalledWith(null);
expect(onClose).toHaveBeenCalledTimes(1);
});
});

View File

@@ -0,0 +1,210 @@
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { TooltipProvider } from '@signozhq/ui/tooltip';
import { server } from 'mocks-server/server';
import { rest, RestRequest } from 'msw';
import MockQueryClientProvider from 'providers/test/MockQueryClientProvider';
import CloudAccountSetupDrawer from '../AddNewAccount/CloudAccountSetupDrawer';
import {
checkInResponse,
CLOUD_INTEGRATION_ID,
connectionCredentials,
connectionCredentialsResponse,
createAccountResponse,
GCP_ACCOUNTS_URL,
GCP_CHECK_IN_URL,
GCP_CREDENTIALS_URL,
} from './mockData';
// `useCloudAccountSetupDrawer` imports logEvent by relative path, which the
// jest.config moduleNameMapper (keyed on the `api/common/logEvent` alias) does
// not intercept — so mock the resolved module directly.
jest.mock('../../../../../api/common/logEvent', () => ({
__esModule: true,
default: jest.fn(),
}));
const onClose = jest.fn();
const renderDrawer = (): void => {
render(
<MockQueryClientProvider>
<TooltipProvider>
<CloudAccountSetupDrawer onClose={onClose} />
</TooltipProvider>
</MockQueryClientProvider>,
);
};
describe('GCP CloudAccountSetupDrawer', () => {
let createAccountPayload: Record<string, unknown> | null;
let checkInPayload: Record<string, unknown> | null;
beforeEach(() => {
createAccountPayload = null;
checkInPayload = null;
server.use(
rest.get(GCP_CREDENTIALS_URL, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(connectionCredentialsResponse)),
),
// check_in is registered first — it is a more specific path than
// /accounts and msw matches handlers in registration order.
rest.post(GCP_CHECK_IN_URL, async (req: RestRequest, res, ctx) => {
checkInPayload = await req.json();
return res(ctx.status(200), ctx.json(checkInResponse));
}),
rest.post(GCP_ACCOUNTS_URL, async (req: RestRequest, res, ctx) => {
createAccountPayload = await req.json();
return res(ctx.status(201), ctx.json(createAccountResponse));
}),
);
});
it('renders SigNoz-provided credentials as read-only fields', async () => {
renderDrawer();
await waitFor(() => {
expect(screen.getByTestId('gcp-signoz-api-url-input')).toHaveTextContent(
connectionCredentials.sigNozApiUrl,
);
});
expect(screen.getByTestId('gcp-signoz-api-key-input')).toHaveTextContent(
connectionCredentials.sigNozApiKey,
);
expect(screen.getByTestId('gcp-ingestion-url-input')).toHaveTextContent(
connectionCredentials.ingestionUrl,
);
expect(screen.getByTestId('gcp-ingestion-key-input')).toHaveTextContent(
connectionCredentials.ingestionKey,
);
expect(screen.getByText('Auto-filled by SigNoz')).toBeInTheDocument();
});
it('blocks submission and surfaces validation errors when the form is empty', async () => {
const user = userEvent.setup();
renderDrawer();
await waitFor(() => {
expect(screen.getByTestId('gcp-connect-account-btn')).toBeEnabled();
});
await user.click(screen.getByTestId('gcp-connect-account-btn'));
await waitFor(() => {
expect(screen.getByText('Please enter an account name')).toBeInTheDocument();
});
expect(
screen.getByText('Please enter the deployment project ID'),
).toBeInTheDocument();
expect(screen.getByText('Please select a region')).toBeInTheDocument();
expect(
screen.getByText('Please add at least one project ID'),
).toBeInTheDocument();
expect(createAccountPayload).toBeNull();
expect(onClose).not.toHaveBeenCalled();
});
it('creates the account, checks the agent in, and closes the drawer', async () => {
const user = userEvent.setup();
renderDrawer();
await waitFor(() => {
expect(screen.getByTestId('gcp-connect-account-btn')).toBeEnabled();
});
await user.type(
screen.getByTestId('gcp-account-name-input'),
'billing@company.com',
);
await user.type(
screen.getByTestId('gcp-deployment-project-id-input'),
'my-deployment-project-123',
);
await user.click(screen.getByTestId('gcp-deployment-region-select'));
await user.click(await screen.findByText('Mumbai (asia-south1)'));
const projectIdsInput = document.querySelector(
'#gcp-project-ids-select',
) as HTMLInputElement;
await user.type(projectIdsInput, 'project-a,project-b,');
await user.click(screen.getByTestId('gcp-connect-account-btn'));
await waitFor(() => {
expect(createAccountPayload).not.toBeNull();
});
expect(createAccountPayload).toStrictEqual({
config: {
gcp: {
deploymentRegion: 'asia-south1',
deploymentProjectId: 'my-deployment-project-123',
projectIds: ['project-a', 'project-b'],
},
},
// Backend-provided credentials win over anything in the form.
credentials: connectionCredentials,
});
await waitFor(() => {
expect(checkInPayload).toStrictEqual({
providerAccountId: 'billing@company.com',
cloudIntegrationId: CLOUD_INTEGRATION_ID,
data: {},
});
});
await waitFor(() => {
expect(onClose).toHaveBeenCalledTimes(1);
});
});
it('shows the backend error inline when account creation fails', async () => {
server.use(
rest.post(GCP_ACCOUNTS_URL, (_req, res, ctx) =>
res(
ctx.status(400),
ctx.json({
status: 'error',
error: { message: 'deployment project id is not accessible' },
}),
),
),
);
const user = userEvent.setup();
renderDrawer();
await waitFor(() => {
expect(screen.getByTestId('gcp-connect-account-btn')).toBeEnabled();
});
await user.type(screen.getByTestId('gcp-account-name-input'), 'my-org');
await user.type(
screen.getByTestId('gcp-deployment-project-id-input'),
'my-deployment-project-123',
);
await user.click(screen.getByTestId('gcp-deployment-region-select'));
await user.click(await screen.findByText('Mumbai (asia-south1)'));
const projectIdsInput = document.querySelector(
'#gcp-project-ids-select',
) as HTMLInputElement;
await user.type(projectIdsInput, 'project-a,');
await user.click(screen.getByTestId('gcp-connect-account-btn'));
await waitFor(() => {
expect(screen.getByTestId('gcp-connect-error')).toHaveTextContent(
'deployment project id is not accessible',
);
});
expect(checkInPayload).toBeNull();
expect(onClose).not.toHaveBeenCalled();
});
});

View File

@@ -0,0 +1,65 @@
import { CloudintegrationtypesCredentialsDTO } from 'api/generated/services/sigNoz.schemas';
import {
CloudAccount,
GCPCloudAccountConfig,
} from 'container/Integrations/types';
export const GCP_CREDENTIALS_URL =
'http://localhost/api/v1/cloud_integrations/gcp/credentials';
export const GCP_ACCOUNTS_URL =
'http://localhost/api/v1/cloud_integrations/gcp/accounts';
export const GCP_CHECK_IN_URL =
'http://localhost/api/v1/cloud_integrations/gcp/accounts/check_in';
export const CLOUD_INTEGRATION_ID = 'ci-gcp-1234';
export const GCP_ACCOUNT_ID = 'acc-gcp-1';
export const GCP_ACCOUNT_URL = `${GCP_ACCOUNTS_URL}/${GCP_ACCOUNT_ID}`;
export const gcpAccountConfig: GCPCloudAccountConfig = {
deployment_region: 'asia-south1',
deployment_project_id: 'my-deployment-project-123',
project_ids: ['project-a', 'project-b'],
};
export const gcpAccount: CloudAccount = {
id: GCP_ACCOUNT_ID,
cloud_account_id: 'gcp-cloud-1',
providerAccountId: 'billing@company.com',
config: gcpAccountConfig,
status: { integration: { last_heartbeat_ts_ms: 1_700_000_000_000 } },
};
export const listAccountsResponse = {
status: 'success',
data: { accounts: [] },
};
/**
* Credentials the backend hands out on SigNoz Cloud. When present the drawer
* renders them read-only and sends them back verbatim on submit.
*/
export const connectionCredentials: CloudintegrationtypesCredentialsDTO = {
sigNozApiUrl: 'https://tenant.signoz.cloud',
sigNozApiKey: 'signoz-api-key-abc',
ingestionUrl: 'https://ingest.us.signoz.cloud',
ingestionKey: 'ingestion-key-xyz',
};
export const connectionCredentialsResponse = {
status: 'success',
data: connectionCredentials,
};
export const createAccountResponse = {
status: 'success',
data: {
id: CLOUD_INTEGRATION_ID,
connectionArtifact: {},
},
};
export const checkInResponse = {
status: 'success',
data: {},
};

View File

@@ -60,6 +60,44 @@ function RemoveIntegrationAccount({
setIsModalOpen(false);
};
let modalDescription: JSX.Element;
if (cloudProvider === INTEGRATION_TYPES.AWS) {
modalDescription = (
<>
Removing this account will remove all components created for sending
telemetry to SigNoz in your AWS account within the next ~15 minutes
(cloudformation stacks named signoz-integration-telemetry-collection in
enabled regions). <br />
<br />
After that, you can delete the cloudformation stack that was created
manually when connecting this account.
</>
);
} else if (cloudProvider === INTEGRATION_TYPES.GCP) {
modalDescription = (
<>
Removing this account will stop SigNoz from monitoring it. <br />
<br />
Since you manage the GCP resources yourself, remember to manually tear down
the OTel collector and Pub/Sub resources you created for this integration if
you no longer need them.
</>
);
} else {
modalDescription = (
<>
Removing this account will remove all components created for sending
telemetry to SigNoz in your Azure subscription within the next ~15 minutes
(deployment stack named signoz-integration-telemetry will be deleted
automatically). <br />
<br />
After that, you have to manually delete &apos;signoz-integration&apos;
deployment stack that was created while connecting this account (Takes ~20
minutes to delete).
</>
);
}
return (
<div className="remove-integration-account-container">
<Button
@@ -84,28 +122,7 @@ function RemoveIntegrationAccount({
loading: isRemoveIntegrationLoading,
}}
>
{cloudProvider === INTEGRATION_TYPES.AWS ? (
<>
Removing this account will remove all components created for sending
telemetry to SigNoz in your AWS account within the next ~15 minutes
(cloudformation stacks named signoz-integration-telemetry-collection in
enabled regions). <br />
<br />
After that, you can delete the cloudformation stack that was created
manually when connecting this account.
</>
) : (
<>
Removing this account will remove all components created for sending
telemetry to SigNoz in your Azure subscription within the next ~15 minutes
(deployment stack named signoz-integration-telemetry will be deleted
automatically). <br />
<br />
After that, you have to manually delete &apos;signoz-integration&apos;
deployment stack that was created while connecting this account (Takes ~20
minutes to delete).
</>
)}
{modalDescription}
</Modal>
</div>
);

View File

@@ -47,3 +47,27 @@ export function mapAccountDtoToAzureCloudAccount(
providerAccountId: account.providerAccountId,
};
}
export function mapAccountDtoToGcpCloudAccount(
account: CloudintegrationtypesAccountDTO,
): IntegrationCloudAccount | null {
if (!account.providerAccountId) {
return null;
}
return {
id: account.id,
cloud_account_id: account.id,
config: {
deployment_region: account.config?.gcp?.deploymentRegion ?? '',
deployment_project_id: account.config?.gcp?.deploymentProjectId ?? '',
project_ids: account.config?.gcp?.projectIds ?? [],
},
status: {
integration: {
last_heartbeat_ts_ms: account.agentReport?.timestampMillis ?? 0,
},
},
providerAccountId: account.providerAccountId,
};
}

View File

@@ -13,8 +13,8 @@ import { ArrowLeft, MoveUpRight, RotateCw } from '@signozhq/icons';
import awwSnapUrl from '@/assets/Icons/awwSnap.svg';
import CloudIntegration from '../CloudIntegration/CloudIntegration';
import { INTEGRATION_TYPES } from '../constants';
import { IntegrationType } from '../types';
import { INTEGRATION_TYPES } from '../constants';
import { handleContactSupport } from '../utils';
import IntegrationDetailContent from './IntegrationDetailContent';
import IntegrationDetailHeader from './IntegrationDetailHeader';
@@ -24,6 +24,12 @@ import { getConnectionStatesFromConnectionStatus } from './utils';
import './IntegrationDetailPage.styles.scss';
const cloudIntegrationTypeById: Record<string, IntegrationType> = {
[INTEGRATION_TYPES.AWS]: IntegrationType.AWS_SERVICES,
[INTEGRATION_TYPES.AZURE]: IntegrationType.AZURE_SERVICES,
[INTEGRATION_TYPES.GCP]: IntegrationType.GCP_SERVICES,
};
// eslint-disable-next-line sonarjs/cognitive-complexity
function IntegrationDetailPage(): JSX.Element {
const history = useHistory();
@@ -55,19 +61,8 @@ function IntegrationDetailPage(): JSX.Element {
),
);
if (
integrationId === INTEGRATION_TYPES.AWS ||
integrationId === INTEGRATION_TYPES.AZURE
) {
return (
<CloudIntegration
type={
integrationId === INTEGRATION_TYPES.AWS
? IntegrationType.AWS_SERVICES
: IntegrationType.AZURE_SERVICES
}
/>
);
if (integrationId && cloudIntegrationTypeById[integrationId]) {
return <CloudIntegration type={cloudIntegrationTypeById[integrationId]} />;
}
return (

View File

@@ -1,7 +1,8 @@
import awsDarkLogo from '@/assets/Logos/aws-dark.svg';
import azureOpenaiLogo from '@/assets/Logos/azure-openai.svg';
import gcpLogo from '@/assets/Logos/gcp.svg';
import { AzureRegion } from './types';
import { AzureRegion, GCPRegion } from './types';
export const INTEGRATION_TELEMETRY_EVENTS = {
INTEGRATIONS_LIST_VISITED: 'Integrations Page: Visited the list page',
@@ -21,6 +22,7 @@ export const INTEGRATION_TELEMETRY_EVENTS = {
export const INTEGRATION_TYPES = {
AWS: 'aws',
AZURE: 'azure',
GCP: 'gcp',
};
export const AWS_INTEGRATION = {
@@ -53,7 +55,26 @@ export const AZURE_INTEGRATION = {
is_new: true,
};
export const ONE_CLICK_INTEGRATIONS = [AWS_INTEGRATION, AZURE_INTEGRATION];
export const GCP_INTEGRATION = {
id: INTEGRATION_TYPES.GCP,
title: 'Google Cloud Platform',
description: 'Setup for GCP monitoring with SigNoz',
author: {
name: 'SigNoz',
email: 'integrations@signoz.io',
homepage: 'https://signoz.io',
},
icon: gcpLogo,
icon_alt: 'gcp-logo',
is_installed: false,
is_new: true,
};
export const ONE_CLICK_INTEGRATIONS = [
AWS_INTEGRATION,
AZURE_INTEGRATION,
GCP_INTEGRATION,
];
export const AZURE_REGIONS: AzureRegion[] = [
{
@@ -165,3 +186,66 @@ export const AZURE_REGIONS: AzureRegion[] = [
{ label: 'West US 2', value: 'westus2', geography: 'United States' },
{ label: 'West US 3', value: 'westus3', geography: 'United States' },
];
// Source of truth: pkg/types/cloudintegrationtypes/regions.go (GCP regions).
export const GCP_REGIONS: GCPRegion[] = [
{ label: 'Johannesburg', value: 'africa-south1', geography: 'Africa' },
{ label: 'Changhua County', value: 'asia-east1', geography: 'APAC' },
{ label: 'Hong Kong', value: 'asia-east2', geography: 'APAC' },
{ label: 'Tokyo', value: 'asia-northeast1', geography: 'APAC' },
{ label: 'Osaka', value: 'asia-northeast2', geography: 'APAC' },
{ label: 'Seoul', value: 'asia-northeast3', geography: 'APAC' },
{ label: 'Mumbai', value: 'asia-south1', geography: 'APAC' },
{ label: 'Delhi', value: 'asia-south2', geography: 'APAC' },
{ label: 'Singapore', value: 'asia-southeast1', geography: 'APAC' },
{ label: 'Jakarta', value: 'asia-southeast2', geography: 'APAC' },
{ label: 'Bangkok', value: 'asia-southeast3', geography: 'APAC' },
{ label: 'Sydney', value: 'australia-southeast1', geography: 'APAC' },
{ label: 'Melbourne', value: 'australia-southeast2', geography: 'APAC' },
{ label: 'Warsaw', value: 'europe-central2', geography: 'Europe' },
{ label: 'Hamina', value: 'europe-north1', geography: 'Europe' },
{ label: 'Stockholm', value: 'europe-north2', geography: 'Europe' },
{ label: 'Madrid', value: 'europe-southwest1', geography: 'Europe' },
{ label: 'St. Ghislain', value: 'europe-west1', geography: 'Europe' },
{ label: 'London', value: 'europe-west2', geography: 'Europe' },
{ label: 'Frankfurt', value: 'europe-west3', geography: 'Europe' },
{ label: 'Eemshaven', value: 'europe-west4', geography: 'Europe' },
{ label: 'Zurich', value: 'europe-west6', geography: 'Europe' },
{ label: 'Milan', value: 'europe-west8', geography: 'Europe' },
{ label: 'Paris', value: 'europe-west9', geography: 'Europe' },
{ label: 'Berlin', value: 'europe-west10', geography: 'Europe' },
{ label: 'Turin', value: 'europe-west12', geography: 'Europe' },
{ label: 'Doha', value: 'me-central1', geography: 'Middle East' },
{ label: 'Dammam', value: 'me-central2', geography: 'Middle East' },
{ label: 'Tel Aviv', value: 'me-west1', geography: 'Middle East' },
{
label: 'Montréal',
value: 'northamerica-northeast1',
geography: 'North America',
},
{
label: 'Toronto',
value: 'northamerica-northeast2',
geography: 'North America',
},
{
label: 'Querétaro',
value: 'northamerica-south1',
geography: 'North America',
},
{
label: 'São Paulo',
value: 'southamerica-east1',
geography: 'South America',
},
{ label: 'Santiago', value: 'southamerica-west1', geography: 'South America' },
{ label: 'Council Bluffs', value: 'us-central1', geography: 'North America' },
{ label: 'Moncks Corner', value: 'us-east1', geography: 'North America' },
{ label: 'Ashburn', value: 'us-east4', geography: 'North America' },
{ label: 'Columbus', value: 'us-east5', geography: 'North America' },
{ label: 'Dallas', value: 'us-south1', geography: 'North America' },
{ label: 'The Dalles', value: 'us-west1', geography: 'North America' },
{ label: 'Los Angeles', value: 'us-west2', geography: 'North America' },
{ label: 'Salt Lake City', value: 'us-west3', geography: 'North America' },
{ label: 'Las Vegas', value: 'us-west4', geography: 'North America' },
];

View File

@@ -6,6 +6,7 @@ import {
export enum IntegrationType {
AWS_SERVICES = 'aws',
AZURE_SERVICES = 'azure',
GCP_SERVICES = 'gcp',
}
interface LogField {
@@ -87,7 +88,10 @@ export interface ServiceData {
export interface CloudAccount {
id: string;
cloud_account_id: string;
config: AzureCloudAccountConfig | AWSCloudAccountConfig;
config:
| AzureCloudAccountConfig
| AWSCloudAccountConfig
| GCPCloudAccountConfig;
status: AccountStatus | IServiceStatus;
providerAccountId: string;
}
@@ -97,6 +101,12 @@ export interface AzureCloudAccountConfig {
resource_groups: string[];
}
export interface GCPCloudAccountConfig {
deployment_region: string;
deployment_project_id: string;
project_ids: string[];
}
export interface AccountStatus {
integration: IntegrationStatus;
}
@@ -111,6 +121,12 @@ export interface AzureRegion {
value: string;
}
export interface GCPRegion {
label: string;
geography: string;
value: string;
}
export interface UpdateServiceConfigPayload {
cloud_account_id: string;
config: AzureServicesConfig;

View File

@@ -0,0 +1,11 @@
.explorer {
display: flex;
flex-direction: column;
gap: var(--spacing-4);
padding: var(--spacing-2) var(--spacing-0);
}
.placeholder {
color: var(--l2-foreground);
font-size: var(--periscope-font-size-base);
}

View File

@@ -0,0 +1,14 @@
import styles from './Explorer.module.scss';
// Shell for the AI Observability Explorer tab. Owns the
// /ai-observability/explorer route and is intentionally empty for now: the
// query builder + results surface land in a follow-up.
function Explorer(): JSX.Element {
return (
<div className={styles.explorer} data-testid="llm-observability-explorer">
<div className={styles.placeholder}>Explorer coming soon.</div>
</div>
);
}
export default Explorer;

View File

@@ -44,6 +44,7 @@ describe('LLMObservability (integration)', () => {
expect(screen.getByTestId('llm-observability-overview')).toBeInTheDocument();
expect(screen.getByTestId('llm-overview-dashboard')).toBeInTheDocument();
expect(screen.getByRole('tab', { name: 'Overview' })).toBeInTheDocument();
expect(screen.getByRole('tab', { name: 'Explorer' })).toBeInTheDocument();
expect(
screen.getByRole('tab', { name: 'Model pricing' }),
).toBeInTheDocument();
@@ -78,6 +79,27 @@ describe('LLMObservability (integration)', () => {
);
});
it('navigates to the explorer route when the Explorer tab is clicked', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
render(<LLMObservability />, undefined, {
initialRoute: ROUTES.AI_OBSERVABILITY_OVERVIEW,
});
await user.click(screen.getByRole('tab', { name: 'Explorer' }));
expect(safeNavigateMock).toHaveBeenCalledWith(
ROUTES.AI_OBSERVABILITY_EXPLORER,
);
});
it('renders the explorer panel on the explorer route', () => {
render(<LLMObservability />, undefined, {
initialRoute: ROUTES.AI_OBSERVABILITY_EXPLORER,
});
expect(screen.getByTestId('llm-observability-explorer')).toBeInTheDocument();
});
it('renders the attribute mapping page on the attribute mapping route', () => {
render(<LLMObservability />, undefined, {
initialRoute: ROUTES.AI_OBSERVABILITY_ATTRIBUTE_MAPPING,

View File

@@ -5,10 +5,12 @@ import ROUTES from 'constants/routes';
import { useSafeNavigate } from 'hooks/useSafeNavigate';
import LLMObservabilityAttributeMapping from '../AttributeMapping/LLMObservabilityAttributeMapping';
import Explorer from '../Explorer/Explorer';
import Overview from '../Overview/Overview';
import LLMObservabilityModelPricing from '../Settings/ModelPricing/LLMObservabilityModelPricing';
const OVERVIEW_KEY = ROUTES.AI_OBSERVABILITY_OVERVIEW;
const EXPLORER_KEY = ROUTES.AI_OBSERVABILITY_EXPLORER;
const CONFIGURATION_KEY = ROUTES.AI_OBSERVABILITY_CONFIGURATION;
const ATTRIBUTE_MAPPING_KEY = ROUTES.AI_OBSERVABILITY_ATTRIBUTE_MAPPING;
@@ -31,6 +33,8 @@ export function useLLMObservabilityTabs(): UseLLMObservabilityTabsResult {
activeTab = CONFIGURATION_KEY;
} else if (pathname.startsWith(ATTRIBUTE_MAPPING_KEY)) {
activeTab = ATTRIBUTE_MAPPING_KEY;
} else if (pathname.startsWith(EXPLORER_KEY)) {
activeTab = EXPLORER_KEY;
}
const onTabChange = useCallback(
@@ -46,6 +50,11 @@ export function useLLMObservabilityTabs(): UseLLMObservabilityTabsResult {
label: 'Overview',
children: <Overview />,
},
{
key: EXPLORER_KEY,
label: 'Explorer',
children: <Explorer />,
},
{
key: CONFIGURATION_KEY,
label: 'Model pricing',

View File

@@ -206,6 +206,7 @@ export const routesToSkip = [
ROUTES.AI_OBSERVABILITY_OVERVIEW,
ROUTES.AI_OBSERVABILITY_CONFIGURATION,
ROUTES.AI_OBSERVABILITY_ATTRIBUTE_MAPPING,
ROUTES.AI_OBSERVABILITY_EXPLORER,
];
export const routesToDisable = [ROUTES.LOGS_EXPLORER, ROUTES.LIVE_LOGS];

View File

@@ -10,7 +10,8 @@ import {
export function isOneClickIntegration(integrationId: string): boolean {
return (
integrationId === INTEGRATION_TYPES.AWS ||
integrationId === INTEGRATION_TYPES.AZURE
integrationId === INTEGRATION_TYPES.AZURE ||
integrationId === INTEGRATION_TYPES.GCP
);
}

View File

@@ -39,8 +39,12 @@ export function useAccountSettingsModal({
}: UseAccountSettingsModalProps): UseAccountSettingsModal {
const [form] = Form.useForm();
const { mutate: updateAccount, isLoading } = useUpdateAccount();
// `account.config` is the shared per-provider union (Azure | AWS | GCP).
// Narrow to Azure by `resource_groups` (Azure-only) rather than
// `deployment_region`, which GCP also has — so it no longer identifies
// Azure uniquely.
const accountConfig = useMemo(
() => ('deployment_region' in account.config ? account.config : null),
() => ('resource_groups' in account.config ? account.config : null),
[account.config],
);
const [resourceGroups, setResourceGroups] = useState<string[]>(

View File

@@ -0,0 +1,148 @@
import {
Dispatch,
SetStateAction,
useCallback,
useEffect,
useMemo,
useState,
} from 'react';
import { toast } from '@signozhq/ui/sonner';
import { Form } from 'antd';
import { FormInstance } from 'antd/lib';
import { useUpdateAccount } from 'api/generated/services/cloudintegration';
import { INTEGRATION_TYPES } from 'container/Integrations/constants';
import { CloudAccount } from 'container/Integrations/types';
import { isEqual } from 'lodash-es';
import logEvent from '../../../api/common/logEvent';
interface UseAccountSettingsDrawerProps {
onClose: () => void;
account: CloudAccount;
setActiveAccount: Dispatch<SetStateAction<CloudAccount | null>>;
}
interface UseAccountSettingsDrawer {
form: FormInstance;
isLoading: boolean;
projectIds: string[];
isSaveDisabled: boolean;
setProjectIds: Dispatch<SetStateAction<string[]>>;
handleSubmit: () => Promise<void>;
handleClose: () => void;
}
export function useAccountSettingsDrawer({
onClose,
account,
setActiveAccount,
}: UseAccountSettingsDrawerProps): UseAccountSettingsDrawer {
const [form] = Form.useForm();
const { mutate: updateAccount, isLoading } = useUpdateAccount();
const accountConfig = useMemo(
() => ('project_ids' in account.config ? account.config : null),
[account.config],
);
const [projectIds, setProjectIds] = useState<string[]>(
accountConfig?.project_ids || [],
);
useEffect(() => {
if (!accountConfig) {
return;
}
form.setFieldsValue({
projectIds: accountConfig.project_ids,
});
setProjectIds(accountConfig.project_ids);
}, [accountConfig, form]);
const handleSubmit = useCallback(async (): Promise<void> => {
try {
const values = await form.validateFields();
if (!accountConfig) {
return;
}
updateAccount(
{
pathParams: {
cloudProvider: INTEGRATION_TYPES.GCP,
id: account?.id || '',
},
data: {
config: {
gcp: {
// Deployment region & project ID are immutable in the UI, but the
// Updatable GCP DTO requires all three fields to be sent.
deploymentRegion: accountConfig.deployment_region,
deploymentProjectId: accountConfig.deployment_project_id,
projectIds: values.projectIds || [],
},
},
},
},
{
onSuccess: () => {
const nextConfig = {
deployment_region: accountConfig.deployment_region,
deployment_project_id: accountConfig.deployment_project_id,
project_ids: values.projectIds || [],
};
setActiveAccount({
...account,
config: nextConfig,
});
onClose();
toast.success('Account settings updated successfully', {
position: 'bottom-right',
});
void logEvent('GCP Integration: Account settings updated', {
cloudAccountId: account.cloud_account_id,
deploymentRegion: nextConfig.deployment_region,
projectIds: nextConfig.project_ids,
});
},
onError: (error) => {
toast.error('Failed to update account settings', {
description: error?.message,
position: 'bottom-right',
});
},
},
);
} catch (error) {
console.error('Form submission failed:', error);
}
}, [form, updateAccount, account, accountConfig, setActiveAccount, onClose]);
const isSaveDisabled = useMemo(() => {
if (!accountConfig) {
return true;
}
return isEqual(
[...(projectIds || [])].sort(),
[...accountConfig.project_ids].sort(),
);
}, [accountConfig, projectIds]);
const handleClose = useCallback(() => {
onClose();
}, [onClose]);
return {
form,
isLoading,
projectIds,
isSaveDisabled,
setProjectIds,
handleSubmit,
handleClose,
};
}

View File

@@ -0,0 +1,165 @@
import { useCallback, useState } from 'react';
import { useQueryClient } from 'react-query';
import { toast } from '@signozhq/ui/sonner';
import {
CreateAccountMutationResult,
GetConnectionCredentialsQueryResult,
invalidateListAccounts,
useAgentCheckIn,
useCreateAccount,
useGetConnectionCredentials,
} from 'api/generated/services/cloudintegration';
import {
CloudintegrationtypesCredentialsDTO,
CloudintegrationtypesPostableAccountDTO,
RenderErrorResponseDTO,
} from 'api/generated/services/sigNoz.schemas';
import { ErrorType } from 'api/generatedAPIInstance';
import { INTEGRATION_TYPES } from 'container/Integrations/constants';
import { GcpSetupFormValues } from 'container/Integrations/CloudIntegration/GoogleCloudPlatform/AddNewAccount/types';
import useAxiosError from 'hooks/useAxiosError';
import { toAPIError } from 'utils/errorUtils';
import logEvent from '../../../api/common/logEvent';
interface UseCloudAccountSetupDrawerProps {
onClose: () => void;
}
interface UseCloudAccountSetupDrawer {
isLoading: boolean;
connectAccount: (values: GcpSetupFormValues) => Promise<void>;
handleClose: () => void;
connectionParams?: CloudintegrationtypesCredentialsDTO;
isConnectionParamsLoading: boolean;
submitError: string | null;
clearSubmitError: () => void;
}
export function useCloudAccountSetupDrawer({
onClose,
}: UseCloudAccountSetupDrawerProps): UseCloudAccountSetupDrawer {
const queryClient = useQueryClient();
const [isLoading, setIsLoading] = useState(false);
const [submitError, setSubmitError] = useState<string | null>(null);
const clearSubmitError = useCallback((): void => {
setSubmitError(null);
}, []);
const { mutateAsync: createAccount } = useCreateAccount();
const { mutateAsync: checkIn } = useAgentCheckIn();
const handleError = useAxiosError();
const { data: connectionParams, isLoading: isConnectionParamsLoading } =
useGetConnectionCredentials<GetConnectionCredentialsQueryResult>(
{
cloudProvider: INTEGRATION_TYPES.GCP,
},
{
query: {
onError: handleError,
},
},
);
const handleClose = useCallback((): void => {
onClose();
}, [onClose]);
const handleConnectionSuccess = useCallback(
(payload: {
cloudIntegrationId: string;
providerAccountId: string;
}): void => {
void logEvent('GCP Integration: Account connected', {
cloudIntegrationId: payload.cloudIntegrationId,
providerAccountId: payload.providerAccountId,
});
toast.success('GCP account connected successfully', {
position: 'bottom-right',
});
void invalidateListAccounts(queryClient, {
cloudProvider: INTEGRATION_TYPES.GCP,
});
handleClose();
},
[handleClose, queryClient],
);
const connectAccount = useCallback(
async (values: GcpSetupFormValues): Promise<void> => {
try {
setIsLoading(true);
setSubmitError(null);
const payload: CloudintegrationtypesPostableAccountDTO = {
config: {
gcp: {
deploymentRegion: values.deploymentRegion,
deploymentProjectId: values.deploymentProjectId,
projectIds: values.projectIds || [],
},
},
credentials: {
// Cloud users can't edit these — the backend-provided credentials are
// authoritative. Enterprise users have no backend defaults and enter
// their own (validated non-empty), so their form values are used.
ingestionUrl: connectionParams?.data?.ingestionUrl || values.ingestionUrl,
ingestionKey: connectionParams?.data?.ingestionKey || values.ingestionKey,
sigNozApiUrl: connectionParams?.data?.sigNozApiUrl || values.sigNozApiUrl,
sigNozApiKey: connectionParams?.data?.sigNozApiKey || values.sigNozApiKey,
},
};
// Step 1: create the integration account.
const createResponse: CreateAccountMutationResult = await createAccount({
pathParams: { cloudProvider: INTEGRATION_TYPES.GCP },
data: payload,
});
const cloudIntegrationId = createResponse.data.id;
const providerAccountId = values.accountName;
void logEvent('GCP Integration: Account created', {
id: cloudIntegrationId,
});
// Step 2: mimic the agent by checking in from the frontend (manual flow).
await checkIn({
pathParams: { cloudProvider: INTEGRATION_TYPES.GCP },
data: {
providerAccountId,
cloudIntegrationId,
data: {},
},
});
handleConnectionSuccess({ cloudIntegrationId, providerAccountId });
} catch (error) {
// Surface the backend's message inline in the drawer instead of a
// generic failure string.
const message = toAPIError(
error as ErrorType<RenderErrorResponseDTO>,
'Failed to connect GCP account',
).getErrorMessage();
setSubmitError(message);
} finally {
setIsLoading(false);
}
},
[connectionParams, createAccount, checkIn, handleConnectionSuccess],
);
return {
isLoading,
connectAccount,
handleClose,
connectionParams: connectionParams?.data as
| CloudintegrationtypesCredentialsDTO
| undefined,
isConnectionParamsLoading,
submitError,
clearSubmitError,
};
}

View File

@@ -16,6 +16,8 @@ export interface CopyButtonProps {
/** Extra class merged onto the button. */
className?: string;
testId?: string;
/** Called after the copy is triggered (e.g. to show a toast). */
onCopy?: () => void;
}
/**
@@ -29,6 +31,7 @@ function CopyButton({
ariaLabel = 'Copy',
className,
testId,
onCopy,
}: CopyButtonProps): JSX.Element {
const { copyToClipboard, isCopied } = useCopyButton();
@@ -36,8 +39,9 @@ function CopyButton({
(e: MouseEvent<HTMLButtonElement>): void => {
e.stopPropagation();
copyToClipboard(value);
onCopy?.();
},
[copyToClipboard, value],
[copyToClipboard, value, onCopy],
);
const stackStyle: CSSProperties = { width: size, height: size };
@@ -65,6 +69,7 @@ CopyButton.defaultProps = {
ariaLabel: 'Copy',
className: undefined,
testId: undefined,
onCopy: undefined,
};
export default CopyButton;

View File

@@ -151,6 +151,7 @@ export const routePermission: Record<keyof typeof ROUTES, ROLES[]> = {
AI_OBSERVABILITY_ATTRIBUTE_MAPPING: ['ADMIN', 'EDITOR', 'VIEWER'],
AI_OBSERVABILITY_BASE: ['ADMIN', 'EDITOR', 'VIEWER'],
AI_OBSERVABILITY_OVERVIEW: ['ADMIN', 'EDITOR', 'VIEWER'],
AI_OBSERVABILITY_EXPLORER: ['ADMIN', 'EDITOR', 'VIEWER'],
AI_OBSERVABILITY_CONFIGURATION: ['ADMIN', 'EDITOR', 'VIEWER'],
};

View File

@@ -8,8 +8,6 @@ import (
// Config holds the configuration for config.
type Config struct {
// Address is the TCP address the API server listens on, in the form "host:port".
Address string `mapstructure:"address"`
Timeout Timeout `mapstructure:"timeout"`
Logging Logging `mapstructure:"logging"`
}
@@ -34,7 +32,6 @@ func NewConfigFactory() factory.ConfigFactory {
func newConfig() factory.Config {
return &Config{
Address: "0.0.0.0:8080",
Timeout: Timeout{
Default: 60 * time.Second,
Max: 600 * time.Second,

View File

@@ -13,7 +13,6 @@ import (
)
func TestNewWithEnvProvider(t *testing.T) {
t.Setenv("SIGNOZ_APISERVER_ADDRESS", "0.0.0.0:9090")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_DEFAULT", "70s")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_MAX", "700s")
t.Setenv("SIGNOZ_APISERVER_TIMEOUT_EXCLUDED__ROUTES", "/excluded1,/excluded2")
@@ -39,7 +38,6 @@ func TestNewWithEnvProvider(t *testing.T) {
require.NoError(t, err)
expected := &Config{
Address: "0.0.0.0:9090",
Timeout: Timeout{
Default: 70 * time.Second,
Max: 700 * time.Second,

View File

@@ -3,16 +3,17 @@ package flagger
import "github.com/SigNoz/signoz/pkg/types/featuretypes"
var (
FeatureUseSpanMetrics = featuretypes.MustNewName("use_span_metrics")
FeatureKafkaSpanEval = featuretypes.MustNewName("kafka_span_eval")
FeatureHideRootUser = featuretypes.MustNewName("hide_root_user")
FeatureGetMetersFromZeus = featuretypes.MustNewName("get_meters_from_zeus")
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureUseMeterReporter = featuretypes.MustNewName("use_meter_reporter")
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureEnableAIObservability = featuretypes.MustNewName("enable_ai_observability")
FeatureEnableMetricsReduction = featuretypes.MustNewName("enable_metrics_reduction")
FeatureUseSpanMetrics = featuretypes.MustNewName("use_span_metrics")
FeatureKafkaSpanEval = featuretypes.MustNewName("kafka_span_eval")
FeatureHideRootUser = featuretypes.MustNewName("hide_root_user")
FeatureGetMetersFromZeus = featuretypes.MustNewName("get_meters_from_zeus")
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureUseMeterReporter = featuretypes.MustNewName("use_meter_reporter")
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureEnableAIObservability = featuretypes.MustNewName("enable_ai_observability")
FeatureEnableMetricsReduction = featuretypes.MustNewName("enable_metrics_reduction")
FeatureUsePrometheusClickhouseV2 = featuretypes.MustNewName("use_prometheus_clickhouse_v2")
FeatureResolveSemconvFamilies = featuretypes.MustNewName("resolve_semconv_families")
)
func MustNewRegistry() featuretypes.Registry {
@@ -97,6 +98,14 @@ func MustNewRegistry() featuretypes.Registry {
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureResolveSemconvFamilies,
Kind: featuretypes.KindBoolean,
Stage: featuretypes.StageExperimental,
Description: "Controls whether trace queries resolve a semantic-convention name to all the spellings of its family",
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
)
if err != nil {
panic(err)

View File

@@ -40,7 +40,10 @@ func (c *conditionBuilder) ConditionFor(
return nil, nil, err
}
keys, warning := querybuilder.ResolveKeys(key, querybuilder.MatchingFieldKeys(key, fieldKeys))
// Rule state history fields have no family support, so every logical field
// is single-member and flattens losslessly to its physical key.
resolved, warning := querybuilder.ResolveLogicalFields(key, querybuilder.MatchingLogicalFields(ctx, orgID, nil, key, fieldKeys))
keys := querybuilder.SingleKeys(resolved)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)

View File

@@ -64,6 +64,23 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, _ valuer.UUID, _, _ uint64,
return []*schema.Column{col}, nil
}
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
// Real columns always exist; labels are checked for key membership.
func (m *fieldMapper) ExistsFor(ctx context.Context, _ valuer.UUID, _, _ uint64, key *telemetrytypes.TelemetryFieldKey, exists bool) (string, error) {
col, err := m.getColumn(ctx, key)
if err != nil {
return "", err
}
if col.Name != "labels" || key.Name == "labels" {
return "true", nil
}
pred := fmt.Sprintf("has(JSONExtractKeys(labels), '%s')", strings.ReplaceAll(key.Name, "'", "\\'"))
if exists {
return pred, nil
}
return "not " + pred, nil
}
func (m *fieldMapper) ColumnExpressionFor(ctx context.Context, orgID valuer.UUID, tsStart, tsEnd uint64, field *telemetrytypes.TelemetryFieldKey, _ telemetrytypes.FieldDataType, _ map[string][]*telemetrytypes.TelemetryFieldKey) (string, error) {
colName, err := m.FieldFor(ctx, orgID, tsStart, tsEnd, field)
if err != nil {

View File

@@ -95,7 +95,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
s := &Server{
config: config,
signoz: signoz,
httpHostPort: config.APIServer.Address,
httpHostPort: constants.HTTPHostPort,
unavailableChannel: make(chan healthcheck.Status),
}
@@ -218,7 +218,7 @@ func (s *Server) initListeners() error {
var err error
publicHostPort := s.httpHostPort
if publicHostPort == "" {
return fmt.Errorf("apiserver.address is required")
return fmt.Errorf("constants.HTTPHostPort is required")
}
s.httpConn, err = net.Listen("tcp", publicHostPort)

View File

@@ -11,7 +11,11 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
)
const OpAmpWsEndpoint = "0.0.0.0:4320" // address for opamp websocket
const (
HTTPHostPort = "0.0.0.0:8080" // Address to serve http (query service)
PrivateHostPort = "0.0.0.0:8085" // Address to server internal services like alert manager
OpAmpWsEndpoint = "0.0.0.0:4320" // address for opamp websocket
)
const MaxAllowedPointsInTimeSeries = 300

View File

@@ -0,0 +1,64 @@
package querybuilder
import (
"context"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/semconv"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
// semconvFamiliesEnabled evaluates the resolve_semconv_families flag for the
// org. A nil flagger means off, so a caller without family support stays
// literal by default.
func semconvFamiliesEnabled(ctx context.Context, orgID valuer.UUID, fl flagger.Flagger) bool {
if fl == nil {
return false
}
return fl.BooleanOrEmpty(ctx, flagger.FeatureResolveSemconvFamilies, featuretypes.NewFlaggerEvaluationContext(orgID))
}
// ExpandKeySelectorsForFamilies adds selectors for the other members of each
// semantic-convention family that a selector names. The metadata fetched for
// a query then contains each spelling that MatchingLogicalFields can group.
// This function is the prefetch of the resolution layer: statement builders
// call it after they derive the selectors, and the metadata store stays
// family-blind (autocomplete responses keep the literal spelling that the
// user typed). It does nothing when the resolve_semconv_families flag is off
// for the org. Only trace selectors expand today, because that matches the
// family support. Fuzzy (search-style) selectors never expand.
func ExpandKeySelectorsForFamilies(ctx context.Context, orgID valuer.UUID, fl flagger.Flagger, selectors []*telemetrytypes.FieldKeySelector) []*telemetrytypes.FieldKeySelector {
if !semconvFamiliesEnabled(ctx, orgID, fl) {
return selectors
}
out := selectors
seen := make(map[string]bool, len(selectors))
for _, selector := range selectors {
seen[selector.Name] = true
}
for _, selector := range selectors {
if selector.Signal != telemetrytypes.SignalTraces ||
selector.SelectorMatchType == telemetrytypes.FieldSelectorMatchTypeFuzzy {
continue
}
members := semconv.Members(semconv.KindAttribute, telemetrytypes.FieldKeySelector{
Name: selector.Name,
Signal: selector.Signal,
FieldContext: selector.FieldContext,
})
for _, member := range members {
if seen[member] {
continue
}
seen[member] = true
expanded := *selector
expanded.Name = member
out = append(out, &expanded)
}
}
return out
}

View File

@@ -21,24 +21,25 @@ const (
hasTokenFunctionDocURL = "https://signoz.io/docs/userguide/functions-reference/#hastoken-function"
)
// ResolveKeys picks which matching field keys a filter term builds conditions for.
// With 0 or 1 match it returns the input unchanged and no warning. When a name is
// ambiguous it returns a warning; a resource+attribute mix defaults to the resource
// keys (the common intent), noted in the warning.
func ResolveKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeysForName []*telemetrytypes.TelemetryFieldKey) ([]*telemetrytypes.TelemetryFieldKey, string) {
if len(fieldKeysForName) <= 1 {
return fieldKeysForName, ""
// ResolveLogicalFields picks which logical fields a filter term builds conditions
// for. With 0 or 1 field it returns the input unchanged and no warning. When a
// name is ambiguous (several logical fields — a family is one field and never
// ambiguous with itself) it returns a warning; a resource+attribute mix defaults
// to the resource fields (the common intent), noted in the warning.
func ResolveLogicalFields(field *telemetrytypes.TelemetryFieldKey, logicalFields []*telemetrytypes.LogicalField) ([]*telemetrytypes.LogicalField, string) {
if len(logicalFields) <= 1 {
return logicalFields, ""
}
warning := fmt.Sprintf(
"Key `%s` is ambiguous, found %d different combinations of field context / data type: %v.",
field.Name,
len(fieldKeysForName),
fieldKeysForName,
len(logicalFields),
logicalFields,
)
hasResource, hasAttribute := false, false
for _, item := range fieldKeysForName {
for _, item := range logicalFields {
switch item.FieldContext {
case telemetrytypes.FieldContextResource:
hasResource = true
@@ -49,18 +50,40 @@ func ResolveKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeysForName []*te
// when there is both resource and attribute context, default to resource only
if hasResource && hasAttribute {
filteredKeys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(fieldKeysForName))
for _, item := range fieldKeysForName {
filtered := make([]*telemetrytypes.LogicalField, 0, len(logicalFields))
for _, item := range logicalFields {
if item.FieldContext == telemetrytypes.FieldContextResource {
filteredKeys = append(filteredKeys, item)
filtered = append(filtered, item)
}
}
fieldKeysForName = filteredKeys
logicalFields = filtered
warning += " " + "Using `resource` context by default. To query attributes explicitly, " +
fmt.Sprintf("use the fully qualified name (e.g., 'attribute.%s')", field.Name)
}
return fieldKeysForName, warning
return logicalFields, warning
}
// WrapAsLogicalFields wraps physical keys (candidate or synthesized) as
// single-member logical fields addressed by the requested spelling.
func WrapAsLogicalFields(requestedName string, keys []*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.LogicalField {
fields := make([]*telemetrytypes.LogicalField, 0, len(keys))
for _, key := range keys {
fields = append(fields, telemetrytypes.SingleLogicalField(requestedName, key))
}
return fields
}
// SingleKeys flattens logical fields to their single members. It is the
// adapter for signals whose fields are single-member by construction (every
// signal without family support); their condition builders keep compiling per
// physical key.
func SingleKeys(fields []*telemetrytypes.LogicalField) []*telemetrytypes.TelemetryFieldKey {
keys := make([]*telemetrytypes.TelemetryFieldKey, 0, len(fields))
for _, field := range fields {
keys = append(keys, field.Single())
}
return keys
}
// NewKeyNotFoundError builds the error a condition builder returns when a filter term

View File

@@ -0,0 +1,94 @@
package querybuilder
import (
"context"
"fmt"
"strings"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
// The two functions below are the only place family expressions are built.
// They compose exclusively from the mapper's per-key primitives (FieldFor,
// ExistsFor), so every member honors its own storage: materialized columns,
// evolutions, and JSON plans ride the member keys, and a signal supports
// families the moment its primitives are correct.
// LogicalValueExpr returns the value expression for a resolved logical field:
// the member's own expression for a single-member field, and a current-first
// merge across the members' expressions for a family.
func LogicalValueExpr(
ctx context.Context,
orgID valuer.UUID,
tsStart, tsEnd uint64,
fm qbtypes.FieldMapper,
logical *telemetrytypes.LogicalField,
) (string, error) {
if !logical.IsFamily() {
return fm.FieldFor(ctx, orgID, tsStart, tsEnd, logical.Single())
}
memberExprs := make([]string, 0, len(logical.Members))
for _, member := range logical.Members {
expr, err := fm.FieldFor(ctx, orgID, tsStart, tsEnd, member)
if err != nil {
return "", err
}
memberExprs = append(memberExprs, expr)
}
if logical.FieldDataType == telemetrytypes.FieldDataTypeString {
// The trailing '' keeps single-key semantics for rows without any
// member: string maps read '' for an absent key, and negative
// operators must keep including such rows (see AddDefaultExistsFilter).
values := make([]string, 0, len(memberExprs))
for _, expr := range memberExprs {
values = append(values, fmt.Sprintf("NULLIF(%s, '')", expr))
}
return "COALESCE(" + strings.Join(values, ", ") + ", '')", nil
}
// Numeric and boolean maps return zero for an absent key. If a family of
// either type is enabled, this tail must become zero too.
branches := make([]string, 0, len(logical.Members)*2)
for i, member := range logical.Members {
guard, err := fm.ExistsFor(ctx, orgID, tsStart, tsEnd, member, true)
if err != nil {
return "", err
}
branches = append(branches, guard, memberExprs[i])
}
return "multiIf(" + strings.Join(branches, ", ") + ", NULL)", nil
}
// LogicalExistsExpr returns the existence predicate for a resolved logical
// field: the member's own predicate for a single-member field, presence of
// any member for a family.
func LogicalExistsExpr(
ctx context.Context,
orgID valuer.UUID,
tsStart, tsEnd uint64,
fm qbtypes.FieldMapper,
logical *telemetrytypes.LogicalField,
exists bool,
) (string, error) {
if !logical.IsFamily() {
return fm.ExistsFor(ctx, orgID, tsStart, tsEnd, logical.Single(), exists)
}
guards := make([]string, 0, len(logical.Members))
for _, member := range logical.Members {
guard, err := fm.ExistsFor(ctx, orgID, tsStart, tsEnd, member, true)
if err != nil {
return "", err
}
guards = append(guards, guard)
}
combined := "(" + strings.Join(guards, " OR ") + ")"
if exists {
return combined, nil
}
return "NOT " + combined, nil
}

View File

@@ -0,0 +1,95 @@
package querybuilder
import (
"context"
"testing"
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// stubFieldMapper provides just the two per-key primitives the shared
// composition builds on; the remaining FieldMapper methods are unused here.
type stubFieldMapper struct{}
func (stubFieldMapper) FieldFor(_ context.Context, _ valuer.UUID, _, _ uint64, key *telemetrytypes.TelemetryFieldKey) (string, error) {
return "value(" + key.Name + ")", nil
}
func (stubFieldMapper) ExistsFor(_ context.Context, _ valuer.UUID, _, _ uint64, key *telemetrytypes.TelemetryFieldKey, exists bool) (string, error) {
if exists {
return "has(" + key.Name + ")", nil
}
return "NOT has(" + key.Name + ")", nil
}
func (stubFieldMapper) ColumnFor(context.Context, valuer.UUID, uint64, uint64, *telemetrytypes.TelemetryFieldKey) ([]*schema.Column, error) {
return nil, qbtypes.ErrColumnNotFound
}
func (stubFieldMapper) ColumnExpressionFor(context.Context, valuer.UUID, uint64, uint64, *telemetrytypes.TelemetryFieldKey, telemetrytypes.FieldDataType, map[string][]*telemetrytypes.TelemetryFieldKey) (string, error) {
return "", qbtypes.ErrColumnNotFound
}
func (stubFieldMapper) CandidateKeys(context.Context, valuer.UUID, *telemetrytypes.TelemetryFieldKey, any, map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
return nil
}
func stringFamily(names ...string) *telemetrytypes.LogicalField {
members := make([]*telemetrytypes.TelemetryFieldKey, 0, len(names))
for _, name := range names {
members = append(members, &telemetrytypes.TelemetryFieldKey{Name: name, FieldDataType: telemetrytypes.FieldDataTypeString})
}
return &telemetrytypes.LogicalField{Name: names[0], FieldDataType: telemetrytypes.FieldDataTypeString, Members: members}
}
func TestLogicalValueExprSingleMemberDelegatesToFieldFor(t *testing.T) {
logical := telemetrytypes.SingleLogicalField("a", &telemetrytypes.TelemetryFieldKey{Name: "a"})
expr, err := LogicalValueExpr(context.Background(), valuer.UUID{}, 0, 0, stubFieldMapper{}, logical)
require.NoError(t, err)
assert.Equal(t, "value(a)", expr)
}
func TestLogicalValueExprStringFamilyMergesCurrentFirst(t *testing.T) {
expr, err := LogicalValueExpr(context.Background(), valuer.UUID{}, 0, 0, stubFieldMapper{}, stringFamily("current", "old"))
require.NoError(t, err)
// The trailing '' preserves keyless-row semantics for negative operators.
assert.Equal(t, "COALESCE(NULLIF(value(current), ''), NULLIF(value(old), ''), '')", expr)
}
func TestLogicalValueExprNumericFamilyGuardsEveryMember(t *testing.T) {
logical := &telemetrytypes.LogicalField{
Name: "current",
FieldDataType: telemetrytypes.FieldDataTypeNumber,
Members: []*telemetrytypes.TelemetryFieldKey{
{Name: "current", FieldDataType: telemetrytypes.FieldDataTypeNumber},
{Name: "old", FieldDataType: telemetrytypes.FieldDataTypeNumber},
},
}
expr, err := LogicalValueExpr(context.Background(), valuer.UUID{}, 0, 0, stubFieldMapper{}, logical)
require.NoError(t, err)
assert.Equal(t, "multiIf(has(current), value(current), has(old), value(old), NULL)", expr)
}
func TestLogicalExistsExprSingleMemberDelegatesToExistsFor(t *testing.T) {
logical := telemetrytypes.SingleLogicalField("a", &telemetrytypes.TelemetryFieldKey{Name: "a"})
expr, err := LogicalExistsExpr(context.Background(), valuer.UUID{}, 0, 0, stubFieldMapper{}, logical, false)
require.NoError(t, err)
assert.Equal(t, "NOT has(a)", expr)
}
func TestLogicalExistsExprFamilyIsAnyMemberPresence(t *testing.T) {
family := stringFamily("current", "old")
expr, err := LogicalExistsExpr(context.Background(), valuer.UUID{}, 0, 0, stubFieldMapper{}, family, true)
require.NoError(t, err)
assert.Equal(t, "(has(current) OR has(old))", expr)
expr, err = LogicalExistsExpr(context.Background(), valuer.UUID{}, 0, 0, stubFieldMapper{}, family, false)
require.NoError(t, err)
assert.Equal(t, "NOT (has(current) OR has(old))", expr)
}

View File

@@ -0,0 +1,195 @@
package querybuilder
import (
"context"
"testing"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// familiesOn returns a flagger with resolve_semconv_families on.
func familiesOn(t *testing.T) flagger.Flagger {
return flaggertest.WithBooleanFlags(t, map[string]bool{
flagger.FeatureResolveSemconvFamilies.String(): true,
})
}
func traceKey(name string, ctx telemetrytypes.FieldContext) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: ctx,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
}
func memberNames(logical *telemetrytypes.LogicalField) []string {
names := make([]string, 0, len(logical.Members))
for _, member := range logical.Members {
names = append(names, member.Name)
}
return names
}
// The deployment.environment(.name) family (enabled in pkg/semconv) drives the
// grouping tests below.
// With the resolve_semconv_families flag off, matches stay single-member and
// selectors stay literal, even when the metadata map has both spellings.
func TestFamiliesOffByDefault(t *testing.T) {
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment.name": {traceKey("deployment.environment.name", telemetrytypes.FieldContextResource)},
"deployment.environment": {traceKey("deployment.environment", telemetrytypes.FieldContextResource)},
}
fields := MatchingLogicalFields(context.Background(), valuer.UUID{}, flaggertest.New(t), &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}, fieldKeys)
require.Len(t, fields, 1)
assert.False(t, fields[0].IsFamily())
assert.Equal(t, []string{"deployment.environment.name"}, memberNames(fields[0]))
selectors := []*telemetrytypes.FieldKeySelector{
{Name: "deployment.environment.name", Signal: telemetrytypes.SignalTraces, SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact},
}
assert.Len(t, ExpandKeySelectorsForFamilies(context.Background(), valuer.UUID{}, flaggertest.New(t), selectors), 1)
}
func TestMatchingLogicalFieldsGroupsFamilyMembers(t *testing.T) {
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment.name": {traceKey("deployment.environment.name", telemetrytypes.FieldContextResource)},
"deployment.environment": {traceKey("deployment.environment", telemetrytypes.FieldContextResource)},
}
for _, requested := range []string{"deployment.environment.name", "deployment.environment"} {
fields := MatchingLogicalFields(context.Background(), valuer.UUID{}, familiesOn(t), &telemetrytypes.TelemetryFieldKey{Name: requested}, fieldKeys)
require.Len(t, fields, 1, "a family is one logical field, requested via %s", requested)
logical := fields[0]
assert.Equal(t, requested, logical.Name, "response identity is the requested spelling")
assert.Equal(t, telemetrytypes.FieldContextResource, logical.FieldContext)
assert.True(t, logical.IsFamily())
assert.Equal(t, []string{"deployment.environment.name", "deployment.environment"}, memberNames(logical),
"members are current-first regardless of the requested spelling")
}
}
// Member precedence is the family's current-first order, not lookup arrival
// order: a current-name key found only under its context-prefixed spelling
// arrives in the second lookup pass yet must still sort first.
func TestMatchingLogicalFieldsOrdersMembersByFamilyRank(t *testing.T) {
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment": {traceKey("deployment.environment", telemetrytypes.FieldContextResource)},
"resource.deployment.environment.name": {traceKey("resource.deployment.environment.name", telemetrytypes.FieldContextResource)},
}
fields := MatchingLogicalFields(context.Background(), valuer.UUID{}, familiesOn(t), &telemetrytypes.TelemetryFieldKey{
Name: "deployment.environment.name",
FieldContext: telemetrytypes.FieldContextResource,
}, fieldKeys)
require.Len(t, fields, 1)
assert.Equal(t, []string{"resource.deployment.environment.name", "deployment.environment"}, memberNames(fields[0]))
}
// Non-trace signals have no family support: the requested spelling stays
// literal, and a family member name never pulls in its siblings.
func TestMatchingLogicalFieldsKeepsLogsLiteral(t *testing.T) {
logsKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalLogs,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
}
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment.name": {logsKey("deployment.environment.name")},
"deployment.environment": {logsKey("deployment.environment")},
}
fields := MatchingLogicalFields(context.Background(), valuer.UUID{}, familiesOn(t), &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}, fieldKeys)
require.Len(t, fields, 1)
assert.False(t, fields[0].IsFamily())
assert.Equal(t, []string{"deployment.environment.name"}, memberNames(fields[0]))
}
// A family and a genuine same-name collision stack cleanly: the family stays
// one logical field, the collision adds another, and resource preference keeps
// the family as a unit.
func TestResolveLogicalFieldsKeepsFamilyThroughAmbiguity(t *testing.T) {
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment.name": {
traceKey("deployment.environment.name", telemetrytypes.FieldContextResource),
traceKey("deployment.environment.name", telemetrytypes.FieldContextAttribute),
},
"deployment.environment": {traceKey("deployment.environment", telemetrytypes.FieldContextResource)},
}
requested := &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}
fields := MatchingLogicalFields(context.Background(), valuer.UUID{}, familiesOn(t), requested, fieldKeys)
require.Len(t, fields, 2, "resource family + attribute collision")
resolved, warning := ResolveLogicalFields(requested, fields)
assert.NotEmpty(t, warning)
require.Len(t, resolved, 1)
assert.Equal(t, telemetrytypes.FieldContextResource, resolved[0].FieldContext)
assert.Equal(t, []string{"deployment.environment.name", "deployment.environment"}, memberNames(resolved[0]))
}
// Members of a family with different data types never merge: the identity
// (signal, context, data type) separates them into distinct logical fields.
func TestMatchingLogicalFieldsNeverMergesAcrossDataTypes(t *testing.T) {
numberKey := traceKey("deployment.environment", telemetrytypes.FieldContextResource)
numberKey.FieldDataType = telemetrytypes.FieldDataTypeNumber
fieldKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment.name": {traceKey("deployment.environment.name", telemetrytypes.FieldContextResource)},
"deployment.environment": {numberKey},
}
fields := MatchingLogicalFields(context.Background(), valuer.UUID{}, familiesOn(t), &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}, fieldKeys)
require.Len(t, fields, 2)
for _, logical := range fields {
assert.False(t, logical.IsFamily())
}
}
func TestExpandKeySelectorsForFamilies(t *testing.T) {
selectors := []*telemetrytypes.FieldKeySelector{
{Name: "deployment.environment.name", Signal: telemetrytypes.SignalTraces, SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact},
{Name: "service.name", Signal: telemetrytypes.SignalTraces, SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact},
{Name: "deployment.environment.name", Signal: telemetrytypes.SignalLogs, SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact},
}
expanded := ExpandKeySelectorsForFamilies(context.Background(), valuer.UUID{}, familiesOn(t), selectors)
names := make([]string, 0, len(expanded))
for _, selector := range expanded {
names = append(names, selector.Name)
}
assert.Equal(t, []string{
"deployment.environment.name",
"service.name",
"deployment.environment.name",
"deployment.environment",
}, names, "one sibling selector for the trace family member; logs and non-family names untouched")
sibling := expanded[len(expanded)-1]
assert.Equal(t, telemetrytypes.SignalTraces, sibling.Signal)
assert.Equal(t, telemetrytypes.FieldSelectorMatchTypeExact, sibling.SelectorMatchType)
}
func TestExpandKeySelectorsForFamiliesDeduplicatesAndSkipsFuzzy(t *testing.T) {
both := []*telemetrytypes.FieldKeySelector{
{Name: "deployment.environment.name", Signal: telemetrytypes.SignalTraces, SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact},
{Name: "deployment.environment", Signal: telemetrytypes.SignalTraces, SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact},
}
assert.Len(t, ExpandKeySelectorsForFamilies(context.Background(), valuer.UUID{}, familiesOn(t), both), 2, "both spellings already referenced")
fuzzy := []*telemetrytypes.FieldKeySelector{
{Name: "deployment.environment.name", Signal: telemetrytypes.SignalTraces, SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeFuzzy},
}
assert.Len(t, ExpandKeySelectorsForFamilies(context.Background(), valuer.UUID{}, familiesOn(t), fuzzy), 1, "fuzzy (search-style) selectors never expand")
}

View File

@@ -9,7 +9,9 @@ import (
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
grammar "github.com/SigNoz/signoz/pkg/parser/filterquery/grammar"
"github.com/SigNoz/signoz/pkg/semconv"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -27,6 +29,7 @@ const stringMatchingOperatorDocURL = "https://signoz.io/docs/userguide/operators
type filterExpressionVisitor struct {
context context.Context
orgID valuer.UUID
fl flagger.Flagger
fieldMapper qbtypes.FieldMapper
conditionBuilder qbtypes.ConditionBuilder
warnings []string
@@ -48,8 +51,11 @@ type filterExpressionVisitor struct {
}
type FilterExprVisitorOpts struct {
Context context.Context
OrgID valuer.UUID
Context context.Context
OrgID valuer.UUID
// Flagger evaluates the resolve_semconv_families flag during resolution.
// A nil Flagger keeps resolution literal.
Flagger flagger.Flagger
Logger *slog.Logger
FieldMapper qbtypes.FieldMapper
ConditionBuilder qbtypes.ConditionBuilder
@@ -68,6 +74,7 @@ func newFilterExpressionVisitor(opts FilterExprVisitorOpts) *filterExpressionVis
return &filterExpressionVisitor{
context: opts.Context,
orgID: opts.OrgID,
fl: opts.Flagger,
fieldMapper: opts.FieldMapper,
conditionBuilder: opts.ConditionBuilder,
fieldKeys: opts.FieldKeys,
@@ -360,7 +367,7 @@ func (v *filterExpressionVisitor) VisitPrimary(ctx *grammar.PrimaryContext) any
return ErrorConditionLiteral
}
}
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.TelemetryFieldKey{v.fullTextColumn}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(searchText))
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.LogicalField{telemetrytypes.SingleLogicalField(v.fullTextColumn.Name, v.fullTextColumn)}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(searchText))
if !ok {
return ErrorConditionLiteral
}
@@ -379,7 +386,7 @@ func (v *filterExpressionVisitor) VisitPrimary(ctx *grammar.PrimaryContext) any
// VisitComparison handles all comparison operators.
func (v *filterExpressionVisitor) VisitComparison(ctx *grammar.ComparisonContext) any {
key := v.Visit(ctx.Key()).(*telemetrytypes.TelemetryFieldKey)
matching := MatchingFieldKeys(key, v.fieldKeys)
matching := MatchingLogicalFields(v.context, v.orgID, v.fl, key, v.fieldKeys)
// Handle EXISTS specially
if ctx.EXISTS() != nil {
@@ -675,7 +682,7 @@ func (v *filterExpressionVisitor) VisitFullText(ctx *grammar.FullTextContext) an
v.errors = append(v.errors, "full text search is not supported")
return ErrorConditionLiteral
}
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.TelemetryFieldKey{v.fullTextColumn}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(text))
conds, ok := v.buildConditions(v.fullTextColumn, []*telemetrytypes.LogicalField{telemetrytypes.SingleLogicalField(v.fullTextColumn.Name, v.fullTextColumn)}, qbtypes.FilterOperatorRegexp, FormatFullTextSearch(text))
if !ok {
return ErrorConditionLiteral
}
@@ -730,7 +737,7 @@ func (v *filterExpressionVisitor) VisitFunctionCall(ctx *grammar.FunctionCallCon
return ErrorConditionLiteral
}
conds, ok := v.buildConditions(key, MatchingFieldKeys(key, v.fieldKeys), operator, value)
conds, ok := v.buildConditions(key, MatchingLogicalFields(v.context, v.orgID, v.fl, key, v.fieldKeys), operator, value)
if !ok {
return ErrorConditionLiteral
}
@@ -922,7 +929,7 @@ func (v *filterExpressionVisitor) VisitKey(ctx *grammar.KeyContext) any {
// buildConditions invokes the condition builder for a filter term, folding its
// warnings/errors into visitor state; returns false if an error was recorded.
func (v *filterExpressionVisitor) buildConditions(key *telemetrytypes.TelemetryFieldKey, matching []*telemetrytypes.TelemetryFieldKey, op qbtypes.FilterOperator, value any) ([]string, bool) {
func (v *filterExpressionVisitor) buildConditions(key *telemetrytypes.TelemetryFieldKey, matching []*telemetrytypes.LogicalField, op qbtypes.FilterOperator, value any) ([]string, bool) {
conds, warns, err := v.conditionBuilder.ConditionFor(v.context, v.orgID, v.startNs, v.endNs, key, v.fieldKeys, qbtypes.ConditionBuilderOptions{SkipResourceFilter: v.skipResourceFilter}, op, value, v.builder)
if err != nil {
_, _, _, _, errURL, _ := errors.Unwrapb(err)
@@ -979,30 +986,158 @@ func assignIfEmpty(s *string, value string) {
}
}
// MatchingFieldKeys returns the field keys from the map that match the given key,
// honoring any context/data type the user specified.
func MatchingFieldKeys(field *telemetrytypes.TelemetryFieldKey, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.TelemetryFieldKey {
fieldKeysForName := []*telemetrytypes.TelemetryFieldKey{}
// match by name; keep items whose context and data type match (unspecified matches any)
for _, item := range fieldKeys[field.Name] {
if (field.FieldContext == telemetrytypes.FieldContextUnspecified || field.FieldContext == item.FieldContext) &&
(field.FieldDataType == telemetrytypes.FieldDataTypeUnspecified || field.FieldDataType == item.FieldDataType) {
fieldKeysForName = append(fieldKeysForName, item)
}
// familyMemberNames returns the physical spellings to look up for the
// referenced key: the semantic-convention family members (current-first) when
// the resolve_semconv_families flag is on for the org and the key can resolve
// to traces, else just the requested name. Only trace field mappers understand
// families today; logs and metrics keep the requested spelling until theirs
// land.
func familyMemberNames(ctx context.Context, orgID valuer.UUID, fl flagger.Flagger, field *telemetrytypes.TelemetryFieldKey) []string {
if !semconvFamiliesEnabled(ctx, orgID, fl) {
return []string{field.Name}
}
// A context may have been split off a name that legitimately contained it (e.g.
// `attribute.key`); also look up the context-prefixed name so both readings resolve.
if field.FieldContext != telemetrytypes.FieldContextUnspecified {
contextPrefixedFieldName := fmt.Sprintf("%s.%s", field.FieldContext.StringValue(), field.Name)
for _, item := range fieldKeys[contextPrefixedFieldName] {
// Context already matched via the lookup key; only data type needs checking.
if field.FieldDataType == telemetrytypes.FieldDataTypeUnspecified || item.FieldDataType == field.FieldDataType {
fieldKeysForName = append(fieldKeysForName, item)
}
}
if field.Signal != telemetrytypes.SignalUnspecified && field.Signal != telemetrytypes.SignalTraces {
return []string{field.Name}
}
return fieldKeysForName
return semconv.Members(semconv.KindAttribute, telemetrytypes.FieldKeySelector{
Name: field.Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: field.FieldContext,
})
}
// MatchingLogicalFields resolves the referenced key against the metadata map
// into logical fields, honoring any context/data type the user specified.
//
// Physical keys that are members of one semantic-convention family (traces
// only today) group into a single logical field per (signal, context, data
// type) identity, members ordered current-first. Every other matching key
// becomes its own single-member logical field. Ambiguity is therefore the
// length of the returned slice, and a family is never ambiguous with itself.
// Members alias the metadata map entries; nothing is copied or mutated.
//
// Family grouping only happens when the resolve_semconv_families flag is on
// for the org. A nil flagger means off: every match then stays a
// single-member logical field.
func MatchingLogicalFields(ctx context.Context, orgID valuer.UUID, fl flagger.Flagger, field *telemetrytypes.TelemetryFieldKey, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.LogicalField {
members := familyMemberNames(ctx, orgID, fl, field)
matches := collectMemberMatches(field, members, fieldKeys)
return groupIntoLogicalFields(field.Name, len(members) > 1, matches)
}
// memberMatch pairs a metadata entry with the family rank of the member name
// it matched under. The stored name of a context-prefixed match differs from
// the member name, so the rank must travel with the match.
type memberMatch struct {
key *telemetrytypes.TelemetryFieldKey
rank int
}
// matchesRequestedIdentity reports whether the entry fits the context and data
// type that the request specified; unspecified matches any. A context-prefixed
// lookup already matched the context through the lookup key itself.
func matchesRequestedIdentity(field, item *telemetrytypes.TelemetryFieldKey, contextMatched bool) bool {
if !contextMatched && field.FieldContext != telemetrytypes.FieldContextUnspecified && field.FieldContext != item.FieldContext {
return false
}
if field.FieldDataType != telemetrytypes.FieldDataTypeUnspecified && field.FieldDataType != item.FieldDataType {
return false
}
return true
}
// inFamilyScope reports whether a match found under a sibling member name is
// legitimate: the entry must be trace metadata, and the member must be in the
// family of the requested name for the entry's context. A member lookup can
// otherwise find a same-named field in a scope where the family does not
// apply.
func inFamilyScope(field, item *telemetrytypes.TelemetryFieldKey, memberName string) bool {
if item.Signal != telemetrytypes.SignalTraces {
return false
}
return slices.Contains(semconv.Members(semconv.KindAttribute, telemetrytypes.FieldKeySelector{
Name: field.Name,
Signal: telemetrytypes.SignalTraces,
FieldContext: item.FieldContext,
}), memberName)
}
// collectMemberMatches finds the metadata entries for every member spelling:
// first under the member names, then under their context-prefixed spellings
// (a context can be a legitimate part of a stored name, e.g. `attribute.key`).
func collectMemberMatches(field *telemetrytypes.TelemetryFieldKey, members []string, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey) []memberMatch {
matches := make([]memberMatch, 0)
collect := func(lookupName string, rank int, memberName string, contextMatched bool) {
for _, item := range fieldKeys[lookupName] {
if !matchesRequestedIdentity(field, item, contextMatched) {
continue
}
if memberName != field.Name && !inFamilyScope(field, item, memberName) {
continue
}
matches = append(matches, memberMatch{key: item, rank: rank})
}
}
for rank, member := range members {
collect(member, rank, member, false)
}
if field.FieldContext != telemetrytypes.FieldContextUnspecified {
for rank, member := range members {
collect(fmt.Sprintf("%s.%s", field.FieldContext.StringValue(), member), rank, member, true)
}
}
return matches
}
// groupIntoLogicalFields turns matches into logical fields. Trace entries in
// family mode group by their (signal, context, data type) identity; every
// other entry becomes its own single-member field. Members sort by family
// rank at the end: precedence is a property of the family, not of the order
// in which the lookups found the members.
func groupIntoLogicalFields(requestedName string, familyMode bool, matches []memberMatch) []*telemetrytypes.LogicalField {
fields := make([]*telemetrytypes.LogicalField, 0, len(matches))
groups := make(map[string]*telemetrytypes.LogicalField)
ranks := make(map[*telemetrytypes.TelemetryFieldKey]int)
for _, match := range matches {
if !familyMode || match.key.Signal != telemetrytypes.SignalTraces {
fields = append(fields, telemetrytypes.SingleLogicalField(requestedName, match.key))
continue
}
identity := match.key.Signal.StringValue() + ";" + match.key.FieldContext.StringValue() + ";" + match.key.FieldDataType.StringValue()
group, ok := groups[identity]
if !ok {
group = &telemetrytypes.LogicalField{
Name: requestedName,
Signal: match.key.Signal,
FieldContext: match.key.FieldContext,
FieldDataType: match.key.FieldDataType,
}
groups[identity] = group
fields = append(fields, group)
}
if groupHasMemberNamed(group, match.key.Name) {
continue
}
ranks[match.key] = match.rank
group.Members = append(group.Members, match.key)
}
for _, logical := range fields {
slices.SortStableFunc(logical.Members, func(a, b *telemetrytypes.TelemetryFieldKey) int {
return ranks[a] - ranks[b]
})
}
return fields
}
func groupHasMemberNamed(group *telemetrytypes.LogicalField, name string) bool {
for _, member := range group.Members {
if member.Name == name {
return true
}
}
return false
}

View File

@@ -588,9 +588,11 @@ func TestVisitKey(t *testing.T) {
// VisitKey only parses; the condition builder matches, resolves ambiguity
// and decides not-found handling. Replay that here against the generic
// builder behavior (error unless the key is ignored).
matching := MatchingFieldKeys(key, tt.fieldKeys)
keys, warning := ResolveKeys(key, matching)
// builder behavior (error unless the key is ignored). The test maps carry
// no signal, so every logical field is single-member and flattens losslessly.
matching := MatchingLogicalFields(context.Background(), valuer.UUID{}, nil, key, tt.fieldKeys)
resolved, warning := ResolveLogicalFields(key, matching)
keys := SingleKeys(resolved)
var gotErrors []string
var gotMainErrURL, gotMainWrnURL string
@@ -766,7 +768,8 @@ func (b *resourceConditionBuilder) ConditionFor(
return nil, nil, nil
}
keys, warning := ResolveKeys(key, MatchingFieldKeys(key, fieldKeys))
resolved, warning := ResolveLogicalFields(key, MatchingLogicalFields(context.Background(), valuer.UUID{}, nil, key, fieldKeys))
keys := SingleKeys(resolved)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)
@@ -808,7 +811,8 @@ func (b *conditionBuilder) ConditionFor(
return []string{fmt.Sprintf("%s_cond", key.Name)}, nil, nil
}
keys, warning := ResolveKeys(key, MatchingFieldKeys(key, fieldKeys))
resolved, warning := ResolveLogicalFields(key, MatchingLogicalFields(context.Background(), valuer.UUID{}, nil, key, fieldKeys))
keys := SingleKeys(resolved)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)

View File

@@ -5,6 +5,7 @@ import (
"fmt"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -14,12 +15,15 @@ import (
type defaultConditionBuilder struct {
fm qbtypes.FieldMapper
// fl evaluates the resolve_semconv_families flag during resolution.
// A nil flagger keeps resolution literal.
fl flagger.Flagger
}
var _ qbtypes.ConditionBuilder = (*defaultConditionBuilder)(nil)
func NewConditionBuilder(fm qbtypes.FieldMapper) *defaultConditionBuilder {
return &defaultConditionBuilder{fm: fm}
func NewConditionBuilder(fm qbtypes.FieldMapper, fl flagger.Flagger) *defaultConditionBuilder {
return &defaultConditionBuilder{fm: fm, fl: fl}
}
func valueForIndexFilter(op qbtypes.FilterOperator, key *telemetrytypes.TelemetryFieldKey, value any) any {
@@ -44,10 +48,74 @@ func keyIndexFilter(key *telemetrytypes.TelemetryFieldKey) any {
return fmt.Sprintf(`%%%s%%`, key.Name)
}
// The three helpers below take the members of one logical field. With a single
// member they render exactly the pre-family shapes; a family widens key/value
// index hints to any-member and presence to any-member (all-absent when negated).
func keyIndexCondition(sb *sqlbuilder.SelectBuilder, column string, members []*telemetrytypes.TelemetryFieldKey) string {
conditions := make([]string, 0, len(members))
for _, member := range members {
conditions = append(conditions, sb.Like(column, keyIndexFilter(member)))
}
if len(conditions) == 1 {
return conditions[0]
}
return sb.Or(conditions...)
}
func valueIndexCondition(
sb *sqlbuilder.SelectBuilder,
column string,
members []*telemetrytypes.TelemetryFieldKey,
op qbtypes.FilterOperator,
value any,
caseInsensitive bool,
) string {
conditions := make([]string, 0, len(members))
for _, member := range members {
patterns := valueForIndexFilter(op, member, value)
switch values := patterns.(type) {
case []string:
for _, pattern := range values {
conditions = append(conditions, sb.Like(column, pattern))
}
default:
if caseInsensitive {
conditions = append(conditions, sb.ILike(column, values))
} else {
conditions = append(conditions, sb.Like(column, values))
}
}
}
if len(conditions) == 1 {
return conditions[0]
}
return sb.Or(conditions...)
}
func memberPresenceCondition(sb *sqlbuilder.SelectBuilder, column string, members []*telemetrytypes.TelemetryFieldKey, exists bool) string {
conditions := make([]string, 0, len(members))
for _, member := range members {
field := fmt.Sprintf("simpleJSONHas(%s, '%s')", column, member.Name)
if exists {
conditions = append(conditions, sb.E(field, true))
} else {
conditions = append(conditions, sb.NE(field, true))
}
}
if exists {
if len(conditions) == 1 {
return conditions[0]
}
return sb.Or(conditions...)
}
return sb.And(conditions...)
}
// SkipResourceFilter is not applicable here: the fingerprint table only stores resource attributes.
func (b *defaultConditionBuilder) ConditionFor(
ctx context.Context,
_ valuer.UUID,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
@@ -57,7 +125,7 @@ func (b *defaultConditionBuilder) ConditionFor(
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
matches := querybuilder.MatchingFieldKeys(key, fieldKeys)
matches := querybuilder.MatchingLogicalFields(ctx, orgID, b.fl, key, fieldKeys)
// has/hasAny/hasAll/hasToken are logs-body-only functions; they never apply to the
// resource fingerprint table, so skip them (the main query still evaluates them).
@@ -65,21 +133,21 @@ func (b *defaultConditionBuilder) ConditionFor(
return nil, nil, nil
}
keys, warning := querybuilder.ResolveKeys(key, matches)
logicalFields, warning := querybuilder.ResolveLogicalFields(key, matches)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)
}
conds := make([]string, 0, len(keys))
for _, k := range keys {
// the resource fingerprint table only stores resource attributes; keys from
conds := make([]string, 0, len(logicalFields))
for _, logical := range logicalFields {
// the resource fingerprint table only stores resource attributes; fields from
// any other context contribute no condition and are omitted. An empty result
// (including an unknown key) lets the caller skip this filter entirely.
if k.FieldContext != telemetrytypes.FieldContextResource {
if logical.FieldContext != telemetrytypes.FieldContextResource {
continue
}
cond, err := b.conditionForKey(ctx, startNs, endNs, k, op, value, sb)
cond, err := b.conditionForLogicalField(ctx, orgID, startNs, endNs, logical, op, value, sb)
if err != nil {
return nil, nil, err
}
@@ -88,11 +156,12 @@ func (b *defaultConditionBuilder) ConditionFor(
return conds, warnings, nil
}
func (b *defaultConditionBuilder) conditionForKey(
func (b *defaultConditionBuilder) conditionForLogicalField(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
logical *telemetrytypes.LogicalField,
op qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
@@ -102,7 +171,7 @@ func (b *defaultConditionBuilder) conditionForKey(
// as we store resource values as string
formattedValue := querybuilder.FormatValueForContains(value)
columns, err := b.fm.ColumnFor(ctx, valuer.UUID{}, startNs, endNs, key)
columns, err := b.fm.ColumnFor(ctx, orgID, startNs, endNs, logical.Single())
if err != nil {
return "", err
}
@@ -115,10 +184,12 @@ func (b *defaultConditionBuilder) conditionForKey(
// as we have not changed the resource column in the resource fingerprint table.
column := columns[0]
keyIdxFilter := sb.Like(column.Name, keyIndexFilter(key))
valueForIndexFilter := valueForIndexFilter(op, key, value)
members := logical.Members
isFamily := logical.IsFamily()
keyIdxFilter := keyIndexCondition(sb, column.Name, members)
singleValueIndexFilter := valueForIndexFilter(op, members[0], value)
fieldName, err := b.fm.FieldFor(ctx, valuer.UUID{}, startNs, endNs, key)
fieldName, err := querybuilder.LogicalValueExpr(ctx, orgID, startNs, endNs, b.fm, logical)
if err != nil {
return "", err
}
@@ -128,12 +199,17 @@ func (b *defaultConditionBuilder) conditionForKey(
return sb.And(
sb.E(fieldName, formattedValue),
keyIdxFilter,
sb.Like(column.Name, valueForIndexFilter),
valueIndexCondition(sb, column.Name, members, op, value, false),
), nil
case qbtypes.FilterOperatorNotEqual:
if isFamily {
// A negated value-index hint would drop rows where another member
// holds the value; the fingerprint scan is small enough without it.
return sb.NE(fieldName, formattedValue), nil
}
return sb.And(
sb.NE(fieldName, formattedValue),
sb.NotLike(column.Name, valueForIndexFilter),
sb.NotLike(column.Name, singleValueIndexFilter),
), nil
case qbtypes.FilterOperatorGreaterThan:
return sb.And(sb.GT(fieldName, formattedValue), keyIdxFilter), nil
@@ -148,7 +224,7 @@ func (b *defaultConditionBuilder) conditionForKey(
return sb.And(
sb.ILike(fieldName, formattedValue),
keyIdxFilter,
sb.ILike(column.Name, valueForIndexFilter),
valueIndexCondition(sb, column.Name, members, op, value, true),
), nil
case qbtypes.FilterOperatorNotLike, qbtypes.FilterOperatorNotILike:
// no index filter: as cannot apply `not contains x%y` as y can be somewhere else
@@ -185,13 +261,11 @@ func (b *defaultConditionBuilder) conditionForKey(
inConditions = append(inConditions, sb.E(fieldName, querybuilder.FormatValueForContains(v)))
}
mainCondition := sb.Or(inConditions...)
valConditions := make([]string, 0, len(values))
if valuesForIndexFilter, ok := valueForIndexFilter.([]string); ok {
for _, v := range valuesForIndexFilter {
valConditions = append(valConditions, sb.Like(column.Name, v))
}
}
mainCondition = sb.And(mainCondition, keyIdxFilter, sb.Or(valConditions...))
mainCondition = sb.And(
mainCondition,
keyIdxFilter,
valueIndexCondition(sb, column.Name, members, op, value, false),
)
return mainCondition, nil
case qbtypes.FilterOperatorNotIn:
@@ -204,8 +278,13 @@ func (b *defaultConditionBuilder) conditionForKey(
notInConditions = append(notInConditions, sb.NE(fieldName, querybuilder.FormatValueForContains(v)))
}
mainCondition := sb.And(notInConditions...)
if isFamily {
// A negated value-index hint would drop rows where another member
// holds the value; the fingerprint scan is small enough without it.
return mainCondition, nil
}
valConditions := make([]string, 0, len(values))
if valuesForIndexFilter, ok := valueForIndexFilter.([]string); ok {
if valuesForIndexFilter, ok := singleValueIndexFilter.([]string); ok {
for _, v := range valuesForIndexFilter {
valConditions = append(valConditions, sb.NotLike(column.Name, v))
}
@@ -215,13 +294,11 @@ func (b *defaultConditionBuilder) conditionForKey(
case qbtypes.FilterOperatorExists:
return sb.And(
sb.E(fmt.Sprintf("simpleJSONHas(%s, '%s')", column.Name, key.Name), true),
memberPresenceCondition(sb, column.Name, members, true),
keyIdxFilter,
), nil
case qbtypes.FilterOperatorNotExists:
return sb.And(
sb.NE(fmt.Sprintf("simpleJSONHas(%s, '%s')", column.Name, key.Name), true),
), nil
return memberPresenceCondition(sb, column.Name, members, false), nil
case qbtypes.FilterOperatorRegexp:
return sb.And(
@@ -237,7 +314,7 @@ func (b *defaultConditionBuilder) conditionForKey(
return sb.And(
sb.ILike(fieldName, fmt.Sprintf(`%%%s%%`, formattedValue)),
keyIdxFilter,
sb.ILike(column.Name, valueForIndexFilter),
valueIndexCondition(sb, column.Name, members, op, value, true),
), nil
case qbtypes.FilterOperatorNotContains:
// no index filter: as cannot apply `not contains x%y` as y can be somewhere else

View File

@@ -2,6 +2,7 @@ package resourcefilter
import (
"context"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"testing"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -201,7 +202,7 @@ func TestConditionBuilder(t *testing.T) {
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
conditionBuilder := NewConditionBuilder(fm, flaggertest.New(t))
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()

View File

@@ -0,0 +1,92 @@
package resourcefilter
import (
"context"
"testing"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func familyFieldKeys() map[string][]*telemetrytypes.TelemetryFieldKey {
newKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
}
return map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment.name": {newKey("deployment.environment.name")},
"deployment.environment": {newKey("deployment.environment")},
}
}
func familyConditionSQL(t *testing.T, op qbtypes.FilterOperator, value any) (string, []any) {
t.Helper()
cb := NewConditionBuilder(NewFieldMapper(), flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureResolveSemconvFamilies.String(): true}))
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(context.Background(), valuer.UUID{}, 0, 0,
&telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"},
familyFieldKeys(), qbtypes.ConditionBuilderOptions{}, op, value, sb)
require.NoError(t, err)
require.Len(t, conds, 1)
sb.Where(conds...)
return sb.BuildWithFlavor(sqlbuilder.ClickHouse)
}
const familyValueExpr = "COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '')"
func TestFamilyEqualWidensIndexHintsToAnyMember(t *testing.T) {
sql, args := familyConditionSQL(t, qbtypes.FilterOperatorEqual, "production")
assert.Contains(t, sql, familyValueExpr+" = ?")
// key hint: either member name may appear in the labels JSON
assert.Contains(t, sql, "(labels LIKE ? OR labels LIKE ?)")
assert.Contains(t, args, "%deployment.environment.name%")
assert.Contains(t, args, "%deployment.environment%")
assert.Contains(t, args, `%deployment.environment.name":"production%`)
assert.Contains(t, args, `%deployment.environment":"production%`)
}
func TestFamilyNotEqualDropsNegatedValueHint(t *testing.T) {
sql, args := familyConditionSQL(t, qbtypes.FilterOperatorNotEqual, "production")
assert.Contains(t, sql, familyValueExpr+" <> ?")
// A negated per-member value hint would drop rows where the other member
// holds the value, so the family form carries no index hints at all.
assert.NotContains(t, sql, "NOT LIKE")
assert.Equal(t, []any{"production"}, args)
}
func TestFamilyExistsIsAnyMemberPresence(t *testing.T) {
sql, _ := familyConditionSQL(t, qbtypes.FilterOperatorExists, nil)
assert.Contains(t, sql, "(simpleJSONHas(labels, 'deployment.environment.name') = ? OR simpleJSONHas(labels, 'deployment.environment') = ?)")
sql, _ = familyConditionSQL(t, qbtypes.FilterOperatorNotExists, nil)
assert.Contains(t, sql, "(simpleJSONHas(labels, 'deployment.environment.name') <> ? AND simpleJSONHas(labels, 'deployment.environment') <> ?)")
}
// With only one member in metadata the SQL keeps the exact pre-family shape,
// including the negated value hint on !=.
func TestSingleMemberShapesUnchanged(t *testing.T) {
cb := NewConditionBuilder(NewFieldMapper(), flaggertest.WithBooleanFlags(t, map[string]bool{flagger.FeatureResolveSemconvFamilies.String(): true}))
soloKeys := map[string][]*telemetrytypes.TelemetryFieldKey{
"deployment.environment.name": familyFieldKeys()["deployment.environment.name"],
}
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(context.Background(), valuer.UUID{}, 0, 0,
&telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"},
soloKeys, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorNotEqual, "production", sb)
require.NoError(t, err)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, "simpleJSONExtractString(labels, 'deployment.environment.name') <> ?")
assert.Contains(t, sql, "labels NOT LIKE ?")
assert.NotContains(t, sql, "COALESCE")
}

View File

@@ -71,6 +71,33 @@ func (m *defaultFieldMapper) FieldFor(
return columns[0].Name, nil
}
// ExistsFor reports key presence in the fingerprint labels JSON. Only resource
// context keys have a presence notion here; anything else is a real column and
// always present.
func (m *defaultFieldMapper) ExistsFor(
ctx context.Context,
_ valuer.UUID,
tsStart, tsEnd uint64,
key *telemetrytypes.TelemetryFieldKey,
exists bool,
) (string, error) {
columns, err := m.getColumn(ctx, tsStart, tsEnd, key)
if err != nil {
return "", err
}
if key.FieldContext != telemetrytypes.FieldContextResource {
if exists {
return "true", nil
}
return "false", nil
}
pred := fmt.Sprintf("simpleJSONHas(%s, '%s')", columns[0].Name, key.Name)
if exists {
return pred, nil
}
return "NOT " + pred, nil
}
func (m *defaultFieldMapper) ColumnExpressionFor(
ctx context.Context,
orgID valuer.UUID,

View File

@@ -47,7 +47,7 @@ func New[T any](
) *resourceFilterStatementBuilder[T] {
set := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/statementbuilder/resourcefilter")
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
cb := NewConditionBuilder(fm, fl)
return &resourceFilterStatementBuilder[T]{
logger: set.Logger(),
dbName: dbName,
@@ -99,7 +99,7 @@ func (b *resourceFilterStatementBuilder[T]) Build(
q.Select("fingerprint")
q.From(fmt.Sprintf("%s.%s", b.dbName, b.tableName))
keySelectors := b.getKeySelectors(query)
keySelectors := querybuilder.ExpandKeySelectorsForFamilies(ctx, orgID, b.flagger, b.getKeySelectors(query))
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, keySelectors)
if err != nil {
return nil, err
@@ -164,6 +164,7 @@ func (b *resourceFilterStatementBuilder[T]) addConditions(
filterWhereClause, err := querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Flagger: b.flagger,
Logger: b.logger,
FieldMapper: b.fieldMapper,
ConditionBuilder: b.conditionBuilder,

View File

@@ -38,6 +38,7 @@ type scopedTraceStatementBuilder struct {
scope TraceScope
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
fl flagger.Flagger
}
var _ qbtypes.StatementBuilder[qbtypes.TraceAggregation] = (*scopedTraceStatementBuilder)(nil)
@@ -59,8 +60,8 @@ func NewFactory(
if err != nil {
return nil, err
}
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
return NewScopedTraceStatementBuilder(settings, metadataStore, fm, cb, scope, traceStmtBuilder, fl), nil
},
)
@@ -98,6 +99,7 @@ func NewScopedTraceStatementBuilder(
scope: scope,
traceStmtBuilder: traceStmtBuilder,
resourceFilterStmtBuilder: resourceFilterStmtBuilder,
fl: fl,
}
}
@@ -266,7 +268,7 @@ func (b *scopedTraceStatementBuilder) fetchKeys(ctx context.Context, orgID value
SelectorMatchType: telemetrytypes.FieldSelectorMatchTypeExact,
})
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, querybuilder.ExpandKeySelectorsForFamilies(ctx, orgID, b.fl, selectors))
return keys, err
}
@@ -442,13 +444,14 @@ func (b *scopedTraceStatementBuilder) resolveSpanPredicate(ctx context.Context,
for i := range selectors {
selectors[i].Signal = telemetrytypes.SignalTraces
}
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, selectors)
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, querybuilder.ExpandKeySelectorsForFamilies(ctx, orgID, b.fl, selectors))
if err != nil {
return "", nil, "", err
}
prepared, err := querybuilder.PrepareWhereClause(expr, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Flagger: b.fl,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,

View File

@@ -31,6 +31,7 @@ type traceQueryStatementBuilder struct {
cb qbtypes.ConditionBuilder
resourceFilterResolver *resourcefilter.ResourceFingerprintResolver[qbtypes.TraceAggregation]
aggExprRewriter qbtypes.AggExprRewriter
fl flagger.Flagger
skipResourceFingerprintEnabled bool
}
@@ -47,8 +48,8 @@ func NewFactory(
return factory.NewProviderFactory(
factory.MustNewName("traces"),
func(_ context.Context, settings factory.ProviderSettings, cfg statementbuilder.Config) (qbtypes.StatementBuilder[qbtypes.TraceAggregation], error) {
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, fm, cb, fl)
return NewTraceQueryStatementBuilder(
settings, metadataStore, fm, cb, aggExprRewriter, telemetryStore, fl,
@@ -91,6 +92,7 @@ func NewTraceQueryStatementBuilder(
cb: conditionBuilder,
resourceFilterResolver: resourceFilterResolver,
aggExprRewriter: aggExprRewriter,
fl: flagger,
skipResourceFingerprintEnabled: skipResourceFingerprintEnable,
}
}
@@ -119,7 +121,7 @@ func (b *traceQueryStatementBuilder) Build(
// We modify SelectFields above (injecting default fields), and those default
// fields can carry keys that need evolutions, so fetch keys after that.
keySelectors := getKeySelectors(query)
keySelectors := querybuilder.ExpandKeySelectorsForFamilies(ctx, orgID, b.fl, getKeySelectors(query))
keys, _, err := b.metadataStore.GetKeysMulti(ctx, orgID, keySelectors)
if err != nil {
@@ -795,6 +797,7 @@ func (b *traceQueryStatementBuilder) addFilterCondition(
preparedWhereClause, err = querybuilder.PrepareWhereClause(query.Filter.Expression, querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: orgID,
Flagger: b.fl,
Logger: b.logger,
FieldMapper: b.fm,
ConditionBuilder: b.cb,

View File

@@ -8,6 +8,7 @@ import (
cmock "github.com/SigNoz/clickhouse-go-mock"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
@@ -376,8 +377,8 @@ func TestStatementBuilder(t *testing.T) {
}
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -677,8 +678,8 @@ func TestStatementBuilderListQuery(t *testing.T) {
}
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -804,8 +805,8 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = c.keysMap
if mockMetadataStore.KeysMap == nil {
@@ -879,8 +880,8 @@ func TestStatementBuilderGroupByResourceEvolution(t *testing.T) {
}
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -1046,8 +1047,8 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
}
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -1684,8 +1685,8 @@ func newSkipResourceFingerprintBuilder(
t.Helper()
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
@@ -1714,8 +1715,8 @@ func TestStatementBuilderGroupByUnseenKey(t *testing.T) {
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -1756,8 +1757,8 @@ func TestStatementBuilderAggregationUnseenKey(t *testing.T) {
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
@@ -1787,3 +1788,72 @@ func TestStatementBuilderAggregationUnseenKey(t *testing.T) {
assert.Contains(t, q.Query, "attributes_number['error.type']")
assert.Contains(t, q.Query, "attributes_bool['error.type']")
}
// TestStatementBuilderSemconvFamilies builds the same family-member filter
// with the resolve_semconv_families flag on and off. On: the resource filter
// merges both spellings and widens the index hints to any member. Off: the
// query uses only the requested spelling, so users see no change.
func TestStatementBuilderSemconvFamilies(t *testing.T) {
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
query := qbtypes.QueryBuilderQuery[qbtypes.TraceAggregation]{
Signal: telemetrytypes.SignalTraces,
StepInterval: qbtypes.Step{Duration: 30 * time.Second},
Aggregations: []qbtypes.TraceAggregation{{Expression: "count()"}},
Filter: &qbtypes.Filter{
Expression: "deployment.environment.name = 'production'",
},
}
cases := []struct {
name string
flag bool
expected qbtypes.Statement
}{
{
name: "flag on merges both spellings",
flag: true,
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (COALESCE(NULLIF(simpleJSONExtractString(labels, 'deployment.environment.name'), ''), NULLIF(simpleJSONExtractString(labels, 'deployment.environment'), ''), '') = ? AND (labels LIKE ? OR labels LIKE ?) AND (labels LIKE ? OR labels LIKE ?)) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY __result_0 DESC",
Args: []any{"production", "%deployment.environment.name%", "%deployment.environment%", "%deployment.environment.name\":\"production%", "%deployment.environment\":\"production%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
},
{
name: "flag off keeps the literal spelling",
flag: false,
expected: qbtypes.Statement{
Query: "WITH __resource_filter AS (SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE (simpleJSONExtractString(labels, 'deployment.environment.name') = ? AND labels LIKE ? AND labels LIKE ?) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint) SELECT count() AS __result_0 FROM signoz_traces.distributed_signoz_index_v3 WHERE resource_fingerprint GLOBAL IN (SELECT fingerprint FROM __resource_filter) AND timestamp >= ? AND timestamp < ? AND ts_bucket_start >= ? AND ts_bucket_start <= ? ORDER BY __result_0 DESC",
Args: []any{"production", "%deployment.environment.name%", "%deployment.environment.name\":\"production%", uint64(1747945619), uint64(1747983448), "1747947419000000000", "1747983448000000000", uint64(1747945619), uint64(1747983448)},
},
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
fl := flaggertest.WithBooleanFlags(t, map[string]bool{
flagger.FeatureResolveSemconvFamilies.String(): c.flag,
})
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
fm,
cb,
aggExprRewriter,
nil,
fl,
false,
100000,
)
q, err := statementBuilder.Build(context.Background(), valuer.UUID{}, 1747947419000, 1747983448000, qbtypes.RequestTypeScalar, query, nil)
require.NoError(t, err)
require.Equal(t, c.expected.Query, q.Query)
require.Equal(t, c.expected.Args, q.Args)
})
}
}

View File

@@ -212,7 +212,7 @@ func (b *traceOperatorCTEBuilder) buildQueryCTE(ctx context.Context, queryName s
return cteName, nil
}
keySelectors := getKeySelectors(*query)
keySelectors := querybuilder.ExpandKeySelectorsForFamilies(ctx, b.orgID, b.stmtBuilder.fl, getKeySelectors(*query))
b.stmtBuilder.logger.DebugContext(ctx, "Key selectors for query", slog.String("query_name", queryName), slog.Any("key_selectors", keySelectors))
keys, _, err := b.stmtBuilder.metadataStore.GetKeysMulti(ctx, b.orgID, keySelectors)
if err != nil {
@@ -265,6 +265,7 @@ func (b *traceOperatorCTEBuilder) buildQueryCTE(ctx context.Context, queryName s
querybuilder.FilterExprVisitorOpts{
Context: ctx,
OrgID: b.orgID,
Flagger: b.stmtBuilder.fl,
Logger: b.stmtBuilder.logger,
FieldMapper: b.stmtBuilder.fm,
ConditionBuilder: b.stmtBuilder.cb,
@@ -442,7 +443,7 @@ func (b *traceOperatorCTEBuilder) buildFinalQuery(ctx context.Context, selectFro
}
}
keySelectors := b.getKeySelectors()
keySelectors := querybuilder.ExpandKeySelectorsForFamilies(ctx, b.orgID, b.stmtBuilder.fl, b.getKeySelectors())
keys, _, err := b.stmtBuilder.metadataStore.GetKeysMulti(ctx, b.orgID, keySelectors)
if err != nil {
return nil, err

View File

@@ -20,8 +20,8 @@ func newTestTraceOperatorStatementBuilder(t *testing.T) *traceOperatorStatementB
t.Helper()
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)

View File

@@ -25,6 +25,7 @@ type traceOperatorStatementBuilder struct {
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation]
aggExprRewriter qbtypes.AggExprRewriter
fl flagger.Flagger
}
var _ qbtypes.TraceOperatorStatementBuilder = (*traceOperatorStatementBuilder)(nil)
@@ -41,8 +42,8 @@ func NewOperatorFactory(
return factory.NewProviderFactory(
factory.MustNewName("traceoperator"),
func(_ context.Context, settings factory.ProviderSettings, cfg statementbuilder.Config) (qbtypes.TraceOperatorStatementBuilder, error) {
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, fm, cb, fl)
traceStmtBuilder := NewTraceQueryStatementBuilder(
settings, metadataStore, fm, cb, aggExprRewriter, telemetryStore, fl,
@@ -85,6 +86,7 @@ func NewTraceOperatorStatementBuilder(
traceStmtBuilder: traceStmtBuilder,
resourceFilterStmtBuilder: resourceFilterStmtBuilder,
aggExprRewriter: aggExprRewriter,
fl: flagger,
}
}

View File

@@ -21,8 +21,9 @@ import (
func TestTraceTimeRangeOptimization(t *testing.T) {
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
fm := tracestelemetryschema.NewFieldMapper()
cb := tracestelemetryschema.NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := tracestelemetryschema.NewFieldMapper(fl)
cb := tracestelemetryschema.NewConditionBuilder(fm, fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = tracestelemetryschema.BuildCompleteFieldKeyMap(releaseTime)
@@ -39,7 +40,6 @@ func TestTraceTimeRangeOptimization(t *testing.T) {
Signal: telemetrytypes.SignalTraces,
}}
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, fl)
statementBuilder := NewTraceQueryStatementBuilder(

View File

@@ -38,8 +38,11 @@ func (c *conditionBuilder) ConditionFor(
return nil, nil, err
}
// an unknown key simply yields no condition rather than an error.
keys, warning := querybuilder.ResolveKeys(key, querybuilder.MatchingFieldKeys(key, fieldKeys))
// an unknown key simply yields no condition rather than an error. Metadata
// fields have no family support, so every logical field is single-member
// and flattens losslessly to its physical key.
resolved, warning := querybuilder.ResolveLogicalFields(key, querybuilder.MatchingLogicalFields(ctx, orgID, nil, key, fieldKeys))
keys := querybuilder.SingleKeys(resolved)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)

View File

@@ -58,6 +58,19 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, _ valuer.UUID, tsStart, tsE
return columns, nil
}
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
func (m *fieldMapper) ExistsFor(ctx context.Context, _ valuer.UUID, tsStart, tsEnd uint64, key *telemetrytypes.TelemetryFieldKey, exists bool) (string, error) {
columns, err := m.getColumn(ctx, tsStart, tsEnd, key)
if err != nil {
return "", err
}
pred := fmt.Sprintf("mapContains(%s, '%s')", columns[0].Name, key.Name)
if exists {
return pred, nil
}
return "NOT " + pred, nil
}
func (m *fieldMapper) FieldFor(ctx context.Context, _ valuer.UUID, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (string, error) {
columns, err := m.getColumn(ctx, startNs, endNs, key)
if err != nil {

View File

@@ -139,7 +139,10 @@ func (c *conditionBuilder) ConditionFor(
return nil, nil, err
}
keys, warning := querybuilder.ResolveKeys(key, querybuilder.MatchingFieldKeys(key, fieldKeys))
// Audit fields have no family support, so every logical field is
// single-member and flattens losslessly to its physical key.
resolved, warning := querybuilder.ResolveLogicalFields(key, querybuilder.MatchingLogicalFields(ctx, orgID, nil, key, fieldKeys))
keys := querybuilder.SingleKeys(resolved)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)

View File

@@ -97,6 +97,19 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, _ valuer.UUID, _, _ uint64,
return m.getColumn(ctx, key)
}
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
func (m *fieldMapper) ExistsFor(ctx context.Context, orgID valuer.UUID, tsStart, tsEnd uint64, key *telemetrytypes.TelemetryFieldKey, exists bool) (string, error) {
fieldExpression, err := m.FieldFor(ctx, orgID, tsStart, tsEnd, key)
if err != nil {
return "", err
}
columns, err := m.getColumn(ctx, key)
if err != nil {
return "", err
}
return querybuilder.ExistsExpression(columns, key, tsStart, tsEnd, fieldExpression, exists)
}
func (m *fieldMapper) ColumnExpressionFor(
ctx context.Context,
orgID valuer.UUID,

View File

@@ -452,7 +452,7 @@ func (c *conditionBuilder) ConditionFor(
value any,
sb *sqlbuilder.SelectBuilder,
) ([]string, []string, error) {
matches := querybuilder.MatchingFieldKeys(key, fieldKeys)
matches := querybuilder.MatchingLogicalFields(ctx, orgID, nil, key, fieldKeys)
skipResourceFilter := options.SkipResourceFilter
// search() resolves its own (optional) scope; handle it before key resolution.
@@ -460,7 +460,10 @@ func (c *conditionBuilder) ConditionFor(
return c.conditionForSearch(ctx, orgID, key, value, sb)
}
keys, warning := querybuilder.ResolveKeys(key, matches)
// Logs fields have no family support yet, so every logical field is
// single-member and flattens losslessly to its physical key.
resolved, warning := querybuilder.ResolveLogicalFields(key, matches)
keys := querybuilder.SingleKeys(resolved)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)

View File

@@ -279,7 +279,7 @@ func (m *fieldMapper) ColumnExpressionFor(
}
var stmts []string
for _, key := range candidates {
guard, err := m.existsExpressionFor(ctx, orgID, tsStart, tsEnd, key, true)
guard, err := m.ExistsFor(ctx, orgID, tsStart, tsEnd, key, true)
if err != nil {
return "", err
}
@@ -308,7 +308,7 @@ func (m *fieldMapper) ColumnExpressionFor(
if !m.membershipGuarded(ctx, orgID, tsStart, tsEnd, candidates[0]) {
return m.FieldFor(ctx, orgID, tsStart, tsEnd, candidates[0])
}
guard, err := m.existsExpressionFor(ctx, orgID, tsStart, tsEnd, candidates[0], true)
guard, err := m.ExistsFor(ctx, orgID, tsStart, tsEnd, candidates[0], true)
if err != nil {
return "", err
}
@@ -326,7 +326,7 @@ func (m *fieldMapper) ColumnExpressionFor(
var stmts []string
for _, key := range candidates {
guard, err := m.existsExpressionFor(ctx, orgID, tsStart, tsEnd, key, true)
guard, err := m.ExistsFor(ctx, orgID, tsStart, tsEnd, key, true)
if err != nil {
return "", err
}
@@ -569,7 +569,8 @@ func (m *fieldMapper) membershipGuarded(ctx context.Context, orgID valuer.UUID,
return columnType == schema.ColumnTypeEnumMap || columnType == schema.ColumnTypeEnumJSON
}
func (m *fieldMapper) existsExpressionFor(
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
func (m *fieldMapper) ExistsFor(
ctx context.Context,
orgID valuer.UUID,
tsStart, tsEnd uint64,

View File

@@ -162,7 +162,9 @@ func (c *conditionBuilder) ConditionFor(
return nil, nil, err
}
keys := querybuilder.MatchingFieldKeys(key, fieldKeys)
// Metric labels have no family support, so every logical field is
// single-member and flattens losslessly to its physical key.
keys := querybuilder.SingleKeys(querybuilder.MatchingLogicalFields(ctx, orgID, nil, key, fieldKeys))
var warnings []string
if len(keys) == 0 {
if _, isColumn := timeSeriesV4Columns[key.Name]; isColumn {

View File

@@ -97,6 +97,18 @@ func (m *fieldMapper) ColumnFor(ctx context.Context, _ valuer.UUID, tsStart, tsE
return m.getColumn(ctx, tsStart, tsEnd, key)
}
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
// Intrinsic fields always exist; labels are checked for key membership.
func (m *fieldMapper) ExistsFor(_ context.Context, _ valuer.UUID, _, _ uint64, key *telemetrytypes.TelemetryFieldKey, exists bool) (string, error) {
if slices.Contains(IntrinsicFields, key.Name) {
return "true", nil
}
if exists {
return fmt.Sprintf("has(JSONExtractKeys(labels), '%s')", key.Name), nil
}
return fmt.Sprintf("not has(JSONExtractKeys(labels), '%s')", key.Name), nil
}
func (m *fieldMapper) ColumnExpressionFor(
ctx context.Context,
orgID valuer.UUID,

View File

@@ -9,6 +9,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -19,12 +20,15 @@ import (
type conditionBuilder struct {
fm qbtypes.FieldMapper
// fl evaluates the resolve_semconv_families flag during resolution.
// A nil flagger keeps resolution literal.
fl flagger.Flagger
}
var _ qbtypes.ConditionBuilder = (*conditionBuilder)(nil)
func NewConditionBuilder(fm qbtypes.FieldMapper) *conditionBuilder {
return &conditionBuilder{fm: fm}
func NewConditionBuilder(fm qbtypes.FieldMapper, fl flagger.Flagger) *conditionBuilder {
return &conditionBuilder{fm: fm, fl: fl}
}
func (c *conditionBuilder) conditionFor(
@@ -32,7 +36,7 @@ func (c *conditionBuilder) conditionFor(
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
logical *telemetrytypes.LogicalField,
operator qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
@@ -42,13 +46,13 @@ func (c *conditionBuilder) conditionFor(
value = querybuilder.FormatValueForContains(value)
}
fieldExpression, err := c.fm.FieldFor(ctx, orgID, startNs, endNs, key)
fieldExpression, err := querybuilder.LogicalValueExpr(ctx, orgID, startNs, endNs, c.fm, logical)
if err != nil {
return "", err
}
// TODO(srikanthccv): maybe extend this to every possible attribute
if key.Name == "duration_nano" || key.Name == "durationNano" { // QoL improvement
if logical.Name == "duration_nano" || logical.Name == "durationNano" { // QoL improvement
switch v := value.(type) {
case string:
if duration, err := time.ParseDuration(v); err == nil {
@@ -65,7 +69,7 @@ func (c *conditionBuilder) conditionFor(
}
}
fieldExpression, value = querybuilder.DataTypeCollisionHandledFieldName(key, value, fieldExpression, operator)
fieldExpression, value = querybuilder.DataTypeCollisionHandledFieldName(logical.Single(), value, fieldExpression, operator)
// regular operators
switch operator {
@@ -154,11 +158,7 @@ func (c *conditionBuilder) conditionFor(
// in the query builder, `exists` and `not exists` are used for
// key membership checks, so depending on the column type, the condition changes
case qbtypes.FilterOperatorExists, qbtypes.FilterOperatorNotExists:
columns, err := c.fm.ColumnFor(ctx, orgID, startNs, endNs, key)
if err != nil {
return "", err
}
pred, err := querybuilder.ExistsExpression(columns, key, startNs, endNs, fieldExpression, operator == qbtypes.FilterOperatorExists)
pred, err := querybuilder.LogicalExistsExpr(ctx, orgID, startNs, endNs, c.fm, logical, operator == qbtypes.FilterOperatorExists)
if err != nil {
return "", err
}
@@ -210,10 +210,10 @@ func (c *conditionBuilder) ConditionFor(
return nil, nil, err
}
matches := querybuilder.MatchingFieldKeys(key, fieldKeys)
matches := querybuilder.MatchingLogicalFields(ctx, orgID, c.fl, key, fieldKeys)
skipResourceFilter := options.SkipResourceFilter
keys, warning := querybuilder.ResolveKeys(key, matches)
logicalFields, warning := querybuilder.ResolveLogicalFields(key, matches)
var warnings []string
if warning != "" {
warnings = append(warnings, warning)
@@ -221,10 +221,10 @@ func (c *conditionBuilder) ConditionFor(
// A bare key that names a real column filters on the column too — first. When metadata
// only knows the name under other contexts, prepend the column and keep metadata matches
// only where their type is consistent with it (a corrupt entry can't degrade the column).
if key.FieldContext == telemetrytypes.FieldContextUnspecified && len(keys) > 0 {
if key.FieldContext == telemetrytypes.FieldContextUnspecified && len(logicalFields) > 0 {
hasColumn := false
for _, k := range keys {
if k.FieldContext == telemetrytypes.FieldContextSpan {
for _, logical := range logicalFields {
if logical.FieldContext == telemetrytypes.FieldContextSpan {
hasColumn = true
break
}
@@ -232,49 +232,49 @@ func (c *conditionBuilder) ConditionFor(
if !hasColumn {
probe := telemetrytypes.NewTelemetryFieldKey(key.Name, telemetrytypes.FieldContextSpan, key.FieldDataType)
if cols, colErr := c.fm.ColumnFor(ctx, orgID, startNs, endNs, probe); colErr == nil && len(cols) > 0 {
combined := make([]*telemetrytypes.TelemetryFieldKey, 0, len(keys)+1)
combined = append(combined, probe)
for _, k := range keys {
if columnMatchesDataType(cols[0], k.FieldDataType) {
combined = append(combined, k)
combined := make([]*telemetrytypes.LogicalField, 0, len(logicalFields)+1)
combined = append(combined, telemetrytypes.SingleLogicalField(key.Name, probe))
for _, logical := range logicalFields {
if columnMatchesDataType(cols[0], logical.FieldDataType) {
combined = append(combined, logical)
}
}
keys = combined
logicalFields = combined
}
}
}
synthesized := false
if len(keys) == 0 {
if len(logicalFields) == 0 {
// Not in metadata. CandidateKeys resolves it: fold contexts (span/trace) get the
// metadata map so it can honor a real column, correct to a stripped-name metadata
// match, or synthesize; strict contexts pass nil and keep their synthesize path.
keys = c.fm.CandidateKeys(ctx, orgID, key, value, candidateLookupKeys(key, fieldKeys))
if len(keys) == 0 {
logicalFields = querybuilder.WrapAsLogicalFields(key.Name, c.fm.CandidateKeys(ctx, orgID, key, value, candidateLookupKeys(key, fieldKeys)))
if len(logicalFields) == 0 {
return nil, warnings, querybuilder.NewKeyNotFoundError(key.Name)
}
synthesized = true
warnings = append(warnings, querybuilder.NewKeyNotFoundWarning(key.Name))
}
// When a resource sub-query already covers the term, drop resource keys from the main
// When a resource sub-query already covers the term, drop resource fields from the main
// query. Synthesized keys are exempt: the sub-query skips keys absent from metadata.
if skipResourceFilter && !synthesized {
filtered := make([]*telemetrytypes.TelemetryFieldKey, 0, len(keys))
for _, k := range keys {
if k.FieldContext != telemetrytypes.FieldContextResource {
filtered = append(filtered, k)
filtered := make([]*telemetrytypes.LogicalField, 0, len(logicalFields))
for _, logical := range logicalFields {
if logical.FieldContext != telemetrytypes.FieldContextResource {
filtered = append(filtered, logical)
}
}
if len(filtered) == 0 {
return nil, warnings, nil
}
keys = filtered
logicalFields = filtered
}
conds := make([]string, 0, len(keys))
for _, k := range keys {
cond, err := c.conditionForKey(ctx, orgID, startNs, endNs, k, operator, value, sb)
conds := make([]string, 0, len(logicalFields))
for _, logical := range logicalFields {
cond, err := c.conditionForLogicalField(ctx, orgID, startNs, endNs, logical, operator, value, sb)
if err != nil {
return nil, nil, err
}
@@ -283,28 +283,28 @@ func (c *conditionBuilder) ConditionFor(
return conds, warnings, nil
}
func (c *conditionBuilder) conditionForKey(
func (c *conditionBuilder) conditionForLogicalField(
ctx context.Context,
orgID valuer.UUID,
startNs uint64,
endNs uint64,
key *telemetrytypes.TelemetryFieldKey,
logical *telemetrytypes.LogicalField,
operator qbtypes.FilterOperator,
value any,
sb *sqlbuilder.SelectBuilder,
) (string, error) {
if c.isSpanScopeField(key.Name) {
return c.buildSpanScopeCondition(key, operator, value, startNs)
if c.isSpanScopeField(logical.Name) {
return c.buildSpanScopeCondition(logical.Single(), operator, value, startNs)
}
condition, err := c.conditionFor(ctx, orgID, startNs, endNs, key, operator, value, sb)
condition, err := c.conditionFor(ctx, orgID, startNs, endNs, logical, operator, value, sb)
if err != nil {
return "", err
}
if operator.AddDefaultExistsFilter() {
// skip adding exists filter for intrinsic fields
field, _ := c.fm.FieldFor(ctx, orgID, startNs, endNs, key)
field, _ := c.fm.FieldFor(ctx, orgID, startNs, endNs, logical.Single())
if slices.Contains(maps.Keys(IntrinsicFields), field) ||
slices.Contains(maps.Keys(IntrinsicFieldsDeprecated), field) ||
slices.Contains(maps.Keys(CalculatedFields), field) ||
@@ -312,7 +312,7 @@ func (c *conditionBuilder) conditionForKey(
return condition, nil
}
existsCondition, err := c.conditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorExists, nil, sb)
existsCondition, err := c.conditionFor(ctx, orgID, startNs, endNs, logical, qbtypes.FilterOperatorExists, nil, sb)
if err != nil {
return "", err
}

View File

@@ -2,6 +2,7 @@ package tracestelemetryschema
import (
"context"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"testing"
"time"
@@ -288,8 +289,8 @@ func TestConditionFor(t *testing.T) {
},
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
fm := NewFieldMapper(flaggertest.New(t))
conditionBuilder := NewConditionBuilder(fm, flaggertest.New(t))
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
@@ -375,8 +376,8 @@ func TestConditionForResourceWithEvolution(t *testing.T) {
},
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
fm := NewFieldMapper(flaggertest.New(t))
conditionBuilder := NewConditionBuilder(fm, flaggertest.New(t))
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
@@ -395,8 +396,8 @@ func TestConditionForResourceWithEvolution(t *testing.T) {
// user input and queries anyway, emitting a warning instead of failing.
func TestConditionForSynthesizedKeys(t *testing.T) {
ctx := context.Background()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := NewFieldMapper(flaggertest.New(t))
cb := NewConditionBuilder(fm, flaggertest.New(t))
// no metadata matches -> the builder must synthesize from user input
var noMatches map[string][]*telemetrytypes.TelemetryFieldKey

View File

@@ -0,0 +1,183 @@
package tracestelemetryschema
import (
"context"
"fmt"
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// familyFixture returns the deployment.environment(.name) family keys as trace
// resource attributes (with the canonical evolution timeline), a metadata map
// holding them, and a time range inside the JSON-column window.
// familyFlagOn returns a flagger with resolve_semconv_families on.
func familyFlagOn(t *testing.T) flagger.Flagger {
return flaggertest.WithBooleanFlags(t, map[string]bool{
flagger.FeatureResolveSemconvFamilies.String(): true,
})
}
func familyFixture() (current, old *telemetrytypes.TelemetryFieldKey, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey, startNs, endNs uint64) {
releaseTime := time.Date(2025, 5, 22, 22, 0, 0, 0, time.UTC)
newKey := func(name string) *telemetrytypes.TelemetryFieldKey {
return &telemetrytypes.TelemetryFieldKey{
Name: name,
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
Evolutions: MockEvolutionData(releaseTime),
}
}
current = newKey("deployment.environment.name")
old = newKey("deployment.environment")
fieldKeys = map[string][]*telemetrytypes.TelemetryFieldKey{
current.Name: {current},
old.Name: {old},
}
return current, old, fieldKeys, uint64(1747947419000000000), uint64(1747983448000000000)
}
// memberValueExprs returns each member's own FieldFor output; family
// expressions must be exactly the composition of these.
func memberValueExprs(t *testing.T, fm qbtypes.FieldMapper, startNs, endNs uint64, members ...*telemetrytypes.TelemetryFieldKey) []string {
t.Helper()
exprs := make([]string, 0, len(members))
for _, member := range members {
expr, err := fm.FieldFor(context.Background(), valuer.UUID{}, startNs, endNs, member)
require.NoError(t, err)
exprs = append(exprs, expr)
}
return exprs
}
func TestConditionForFamilyMergesMembersCurrentFirst(t *testing.T) {
current, old, fieldKeys, startNs, endNs := familyFixture()
fl := familyFlagOn(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
// The requested spelling is the old name; precedence must still be
// current-first.
requested := &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment"}
sb := sqlbuilder.NewSelectBuilder()
conds, warnings, err := cb.ConditionFor(context.Background(), valuer.UUID{}, startNs, endNs, requested, fieldKeys, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "production", sb)
require.NoError(t, err)
assert.Empty(t, warnings, "a family is one logical field, never ambiguous with itself")
require.Len(t, conds, 1)
exprs := memberValueExprs(t, fm, startNs, endNs, current, old)
family := fmt.Sprintf("COALESCE(NULLIF(%s, ''), NULLIF(%s, ''), '')", exprs[0], exprs[1])
sb.Where(conds...)
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, family+" = ?")
// Equal adds the default exists filter: presence of any member.
assert.Contains(t, sql, fmt.Sprintf("(%s IS NOT NULL OR %s IS NOT NULL)", exprs[0], exprs[1]))
assert.Equal(t, []any{"production"}, args)
}
func TestConditionForFamilyNegativeKeepsKeylessRows(t *testing.T) {
current, old, fieldKeys, startNs, endNs := familyFixture()
fl := familyFlagOn(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
requested := &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(context.Background(), valuer.UUID{}, startNs, endNs, requested, fieldKeys, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorNotEqual, "production", sb)
require.NoError(t, err)
require.Len(t, conds, 1)
exprs := memberValueExprs(t, fm, startNs, endNs, current, old)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
// The trailing '' makes rows without any member read '' (single-key map
// semantics), so `!=` keeps including them; no exists filter is added.
assert.Contains(t, sql, fmt.Sprintf("COALESCE(NULLIF(%s, ''), NULLIF(%s, ''), '') <> ?", exprs[0], exprs[1]))
assert.NotContains(t, sql, "IS NOT NULL OR")
}
func TestConditionForFamilyExists(t *testing.T) {
current, old, fieldKeys, startNs, endNs := familyFixture()
fl := familyFlagOn(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
requested := &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(context.Background(), valuer.UUID{}, startNs, endNs, requested, fieldKeys, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorNotExists, nil, sb)
require.NoError(t, err)
require.Len(t, conds, 1)
exprs := memberValueExprs(t, fm, startNs, endNs, current, old)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, fmt.Sprintf("NOT (%s IS NOT NULL OR %s IS NOT NULL)", exprs[0], exprs[1]))
}
// With the flag off, both spellings can be in the metadata map and the
// condition still uses only the requested key. Users see no change until the
// flag is on.
func TestConditionForFamilyOffByDefault(t *testing.T) {
current, _, fieldKeys, startNs, endNs := familyFixture()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
requested := &telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(context.Background(), valuer.UUID{}, startNs, endNs, requested, fieldKeys, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "production", sb)
require.NoError(t, err)
require.Len(t, conds, 1)
exprs := memberValueExprs(t, fm, startNs, endNs, current)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, exprs[0]+" = ?")
assert.NotContains(t, sql, "COALESCE")
}
// A single-member key's condition is byte-identical to the pre-family shape:
// composition only appears when metadata proves a second member.
func TestConditionForSingleMemberIsUnchanged(t *testing.T) {
current, _, _, startNs, endNs := familyFixture()
fl := familyFlagOn(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
soloKeys := map[string][]*telemetrytypes.TelemetryFieldKey{current.Name: {current}}
requested := &telemetrytypes.TelemetryFieldKey{Name: current.Name}
sb := sqlbuilder.NewSelectBuilder()
conds, _, err := cb.ConditionFor(context.Background(), valuer.UUID{}, startNs, endNs, requested, soloKeys, qbtypes.ConditionBuilderOptions{}, qbtypes.FilterOperatorEqual, "production", sb)
require.NoError(t, err)
require.Len(t, conds, 1)
exprs := memberValueExprs(t, fm, startNs, endNs, current)
sb.Where(conds...)
sql, _ := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, exprs[0]+" = ?")
assert.NotContains(t, sql, "COALESCE")
}
func TestColumnExpressionForFamilyGroupBy(t *testing.T) {
current, old, fieldKeys, startNs, endNs := familyFixture()
fm := NewFieldMapper(familyFlagOn(t))
expr, err := fm.ColumnExpressionFor(context.Background(), valuer.UUID{}, startNs, endNs,
&telemetrytypes.TelemetryFieldKey{Name: "deployment.environment.name"}, telemetrytypes.FieldDataTypeString, fieldKeys)
require.NoError(t, err)
exprs := memberValueExprs(t, fm, startNs, endNs, current, old)
family := fmt.Sprintf("COALESCE(NULLIF(%s, ''), NULLIF(%s, ''), '')", exprs[0], exprs[1])
guard := fmt.Sprintf("(%s IS NOT NULL OR %s IS NOT NULL)", exprs[0], exprs[1])
assert.Equal(t, fmt.Sprintf("multiIf(%s, %s, NULL)", guard, family), expr)
}

View File

@@ -7,6 +7,7 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -159,12 +160,16 @@ var (
}
)
type fieldMapper struct{}
type fieldMapper struct {
// fl evaluates the resolve_semconv_families flag during resolution.
// A nil flagger keeps resolution literal.
fl flagger.Flagger
}
var _ qbtypes.FieldMapper = (*fieldMapper)(nil)
func NewFieldMapper() *fieldMapper {
return &fieldMapper{}
func NewFieldMapper(fl flagger.Flagger) *fieldMapper {
return &fieldMapper{fl: fl}
}
func (m *fieldMapper) getColumn(
@@ -336,6 +341,68 @@ func (m *fieldMapper) resolveColumnExprs(
return exprs, existExprs, columns, nil
}
// logicalForResolvedColumn upgrades a directly-resolvable key (the FieldFor
// probe succeeded) to its family when the metadata map proves membership;
// otherwise the key stays a single-member logical field.
func (m *fieldMapper) logicalForResolvedColumn(ctx context.Context, orgID valuer.UUID, field *telemetrytypes.TelemetryFieldKey, keys map[string][]*telemetrytypes.TelemetryFieldKey) *telemetrytypes.LogicalField {
for _, logical := range querybuilder.MatchingLogicalFields(ctx, orgID, m.fl, field, keys) {
if logical.IsFamily() &&
logical.FieldContext == field.FieldContext &&
(field.FieldDataType == telemetrytypes.FieldDataTypeUnspecified || logical.FieldDataType == field.FieldDataType) {
return logical
}
}
return telemetrytypes.SingleLogicalField(field.Name, field)
}
// upgradeToFamilies swaps single-member candidates for their family when the
// metadata map proves membership. Candidate order and every non-family
// candidate stay exactly as the legacy flow produced them; sibling candidates
// of an already-emitted family are dropped rather than duplicated.
func (m *fieldMapper) upgradeToFamilies(ctx context.Context, orgID valuer.UUID, field *telemetrytypes.TelemetryFieldKey, candidates []*telemetrytypes.LogicalField, keys map[string][]*telemetrytypes.TelemetryFieldKey) []*telemetrytypes.LogicalField {
var families []*telemetrytypes.LogicalField
for _, logical := range querybuilder.MatchingLogicalFields(ctx, orgID, m.fl, field, keys) {
if logical.IsFamily() {
families = append(families, logical)
}
}
if len(families) == 0 {
return candidates
}
out := make([]*telemetrytypes.LogicalField, 0, len(candidates))
emitted := make(map[*telemetrytypes.LogicalField]bool)
for _, candidate := range candidates {
var family *telemetrytypes.LogicalField
for _, fam := range families {
if fam.FieldContext != candidate.FieldContext || fam.FieldDataType != candidate.FieldDataType {
continue
}
memberOfFamily := candidate.Single().Name == field.Name
for _, member := range fam.Members {
if member.Name == candidate.Single().Name {
memberOfFamily = true
break
}
}
if memberOfFamily {
family = fam
break
}
}
if family == nil {
out = append(out, candidate)
continue
}
if emitted[family] {
continue
}
emitted[family] = true
out = append(out, family)
}
return out
}
// ColumnExpressionFor returns the bare (unaliased) SQL expression for the field, resolving
// unknown keys via CandidateKeys and wrapping guardable columns with exists-guard multiIfs
// so an absent key yields NULL.
@@ -348,18 +415,23 @@ func (m *fieldMapper) ColumnExpressionFor(
keys map[string][]*telemetrytypes.TelemetryFieldKey,
) (string, error) {
// Resolve the candidate column(s).
var candidates []*telemetrytypes.TelemetryFieldKey
// Resolve the candidate logical field(s).
var candidates []*telemetrytypes.LogicalField
switch _, err := m.FieldFor(ctx, orgID, startNs, endNs, field); {
case err == nil:
candidates = []*telemetrytypes.TelemetryFieldKey{field}
// A directly-resolvable key upgrades to its family when the metadata
// map proves membership; otherwise it stays single-member.
candidates = []*telemetrytypes.LogicalField{m.logicalForResolvedColumn(ctx, orgID, field, keys)}
case errors.Is(err, qbtypes.ErrColumnNotFound):
// column (when the bare name is one) plus metadata matches, else synthesized
// type-variant keys.
candidates = m.CandidateKeys(ctx, orgID, field, nil, keys)
if len(candidates) == 0 {
// The legacy candidate flow, unchanged: column (when the bare name is
// one) plus metadata matches, else synthesized type-variant keys. The
// family step below only swaps candidates for their family; it never
// changes candidate order or non-family behavior.
raw := m.CandidateKeys(ctx, orgID, field, nil, keys)
if len(raw) == 0 {
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.NewSuggestionsOnLevenshteinDistance(field.Name, errors.NounKeys, maps.Keys(keys))...)
}
candidates = m.upgradeToFamilies(ctx, orgID, field, querybuilder.WrapAsLogicalFields(field.Name, raw), keys)
default:
return "", err
}
@@ -373,21 +445,21 @@ func (m *fieldMapper) ColumnExpressionFor(
dummyValue = 0.0
}
stmts := make([]string, 0, len(candidates)*2)
for _, key := range candidates {
value, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
for _, logical := range candidates {
value, err := querybuilder.LogicalValueExpr(ctx, orgID, startNs, endNs, m, logical)
if err != nil {
return "", err
}
guard, err := m.existsExpressionFor(ctx, orgID, startNs, endNs, key, true)
guard, err := querybuilder.LogicalExistsExpr(ctx, orgID, startNs, endNs, m, logical, true)
if err != nil {
return "", err
}
coerced := value
// a time column keeps its native type; coercing it would yield seconds
if temporal, err := m.columnIsTemporal(ctx, startNs, endNs, key); err != nil {
if temporal, err := m.logicalIsTemporal(ctx, startNs, endNs, logical); err != nil {
return "", err
} else if !temporal {
coerced, _ = querybuilder.DataTypeCollisionHandledFieldName(key, dummyValue, value, qbtypes.FilterOperatorUnknown)
coerced, _ = querybuilder.DataTypeCollisionHandledFieldName(logical.Single(), dummyValue, value, qbtypes.FilterOperatorUnknown)
}
stmts = append(stmts, guard, coerced)
}
@@ -395,13 +467,14 @@ func (m *fieldMapper) ColumnExpressionFor(
}
if len(candidates) == 1 {
value, err := m.FieldFor(ctx, orgID, startNs, endNs, candidates[0])
logical := candidates[0]
value, err := querybuilder.LogicalValueExpr(ctx, orgID, startNs, endNs, m, logical)
if err != nil {
return "", err
}
exprs, existExprs, _, _ := m.resolveColumnExprs(ctx, startNs, endNs, candidates[0])
if len(exprs) == 1 && len(existExprs) == 1 {
guard, err := m.existsExpressionFor(ctx, orgID, startNs, endNs, candidates[0], true)
exprs, existExprs, _, _ := m.resolveColumnExprs(ctx, startNs, endNs, logical.Single())
if !logical.IsFamily() && len(exprs) == 1 && len(existExprs) == 1 {
guard, err := querybuilder.LogicalExistsExpr(ctx, orgID, startNs, endNs, m, logical, true)
if err != nil {
return "", err
}
@@ -413,12 +486,12 @@ func (m *fieldMapper) ColumnExpressionFor(
// Multiple candidates (collision / synth): multiIf picks the first that exists,
// stringified so branches share a type.
args := make([]string, 0, len(candidates))
for _, key := range candidates {
value, err := m.FieldFor(ctx, orgID, startNs, endNs, key)
for _, logical := range candidates {
value, err := querybuilder.LogicalValueExpr(ctx, orgID, startNs, endNs, m, logical)
if err != nil {
return "", err
}
guard, err := m.existsExpressionFor(ctx, orgID, startNs, endNs, key, true)
guard, err := querybuilder.LogicalExistsExpr(ctx, orgID, startNs, endNs, m, logical, true)
if err != nil {
return "", err
}
@@ -427,6 +500,15 @@ func (m *fieldMapper) ColumnExpressionFor(
return fmt.Sprintf("multiIf(%s, NULL)", strings.Join(args, ", ")), nil
}
// logicalIsTemporal reports whether the logical field resolves to a single time
// column. A family is attribute-backed and never temporal.
func (m *fieldMapper) logicalIsTemporal(ctx context.Context, startNs, endNs uint64, logical *telemetrytypes.LogicalField) (bool, error) {
if logical.IsFamily() {
return false, nil
}
return m.columnIsTemporal(ctx, startNs, endNs, logical.Single())
}
// columnIsTemporal reports whether key resolves to a single time column, after evolution
// selection. Multiple columns mean an attribute-map union, which is never temporal.
func (m *fieldMapper) columnIsTemporal(ctx context.Context, startNs, endNs uint64, key *telemetrytypes.TelemetryFieldKey) (bool, error) {
@@ -522,7 +604,8 @@ func (m *fieldMapper) CandidateKeys(ctx context.Context, _ valuer.UUID, field *t
return nil
}
func (m *fieldMapper) existsExpressionFor(
// ExistsFor implements the per-key existence primitive of qbtypes.FieldMapper.
func (m *fieldMapper) ExistsFor(
ctx context.Context,
orgID valuer.UUID,
tsStart, tsEnd uint64,

View File

@@ -2,6 +2,7 @@ package tracestelemetryschema
import (
"context"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"testing"
"time"
@@ -107,7 +108,7 @@ func TestGetFieldKeyName(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fm := NewFieldMapper()
fm := NewFieldMapper(flaggertest.New(t))
result, err := fm.FieldFor(ctx, valuer.UUID{}, uint64(time.Date(2024, 6, 1, 0, 0, 0, 0, time.UTC).UnixNano()), uint64(time.Date(2024, 6, 5, 0, 0, 0, 0, time.UTC).UnixNano()), &tc.key)
if tc.expectedError != nil {
@@ -195,7 +196,7 @@ func TestFieldForResourceWithEvolution(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fm := NewFieldMapper()
fm := NewFieldMapper(flaggertest.New(t))
result, err := fm.FieldFor(ctx, valuer.UUID{}, tc.tsStart, tc.tsEnd, &tc.key)
require.NoError(t, err)
assert.Equal(t, tc.expectedResult, result)
@@ -255,7 +256,7 @@ func TestColumnExpressionForTemporalColumn(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fm := NewFieldMapper()
fm := NewFieldMapper(flaggertest.New(t))
result, err := fm.ColumnExpressionFor(ctx, valuer.UUID{}, tsStart, tsEnd, &tc.key, tc.requiredDataType, nil)
require.NoError(t, err)
assert.Equal(t, tc.expectedResult, result)
@@ -283,7 +284,7 @@ func TestColumnExpressionForTimestampAttributeCollision(t *testing.T) {
},
}
fm := NewFieldMapper()
fm := NewFieldMapper(flaggertest.New(t))
t.Run("bare timestamp resolves to the intrinsic column alone", func(t *testing.T) {
bare := telemetrytypes.TelemetryFieldKey{Name: "timestamp"}

View File

@@ -2,6 +2,7 @@ package tracestelemetryschema
import (
"context"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"testing"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
@@ -14,8 +15,8 @@ import (
func TestSpanScopeFilterExpression(t *testing.T) {
// Test that span scope fields work in filter expressions
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := NewFieldMapper(flaggertest.New(t))
cb := NewConditionBuilder(fm, flaggertest.New(t))
tests := []struct {
name string
@@ -128,8 +129,8 @@ func TestSpanScopeWithResourceFilter(t *testing.T) {
// For now, just verify the expression parses correctly
// In a real implementation, we'd need to check that the resource filter
// is properly skipped when span scope fields are present
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := NewFieldMapper(flaggertest.New(t))
cb := NewConditionBuilder(fm, flaggertest.New(t))
// Prepare field keys for the test
fieldKeys := make(map[string][]*telemetrytypes.TelemetryFieldKey)

Some files were not shown because too many files have changed in this diff Show More