Compare commits

..

19 Commits

Author SHA1 Message Date
grandwizard28
4f400da770 test(callbackauthn): pin the auth domain POST/GET roundtrip contract
Clients like the terraform provider land state via a follow-up GET after
every write, so the responses must round-trip posted values exactly. The
parametrized cases pin that contract per kind, including the server-side
defaulting (attribute/claim mappings, zero-value scalars) and role-name
normalization (EDITOR -> signoz-editor, null groupMappings).
2026-08-08 17:58:32 +05:30
grandwizard28
de3ba9d20c test(callbackauthn): cover the google authn flow end to end
A wiremock container impersonates Google's OIDC provider: it joins the
test network as accounts.google.com and serves HTTPS with a certificate
issued by a new integration CA, which every signoz container now trusts
via SSL_CERT_FILE, since the google callback authn hardcodes Google's
issuer and fully verifies the RS256 id_token against the served JWKS.
Stubs are installed per test with a pre-signed token for the identity
under test. The domain tests also sweep leftover domains so reruns
against a reused stack stay green.
2026-08-08 17:45:22 +05:30
grandwizard28
fe447e7f35 fix(authtypes): mark validator-required config fields as required in the schema 2026-08-08 17:12:19 +05:30
grandwizard28
94ce3e51f1 fix(apiserver): move auth domain endpoints to /api/v2/auth_domains
The request and response shapes changed, so the endpoints move to a new
version instead of breaking /api/v1/domains in place; the v1 routes are
removed.
2026-08-08 17:02:21 +05:30
grandwizard28
a5ddafc5db fix(authtypes): rename google_auth kind to google and saml ssoUrl to location
The kind follows the provider name; the SAML field follows the Location
attribute of the SingleSignOnService element, consistent with entityId
and certificate. Persisted rows are untouched: StorableAuthDomainConfig
translates the legacy google_auth value on read and keeps writing it.
2026-08-08 16:52:28 +05:30
grandwizard28
54e2caaac2 docs(contributing): explain envelope placement and tagging-style rationale 2026-08-08 16:21:18 +05:30
grandwizard28
9313e7bea5 docs(contributing): document the kind/spec envelope for sum types 2026-08-08 16:12:35 +05:30
grandwizard28
6b58b6fcae fix(tests): update auth domain payloads to kind/spec envelope 2026-08-08 16:11:22 +05:30
grandwizard28
cc5d3574a4 fix(frontend): adopt kind/spec auth domain payload in org settings
Regenerates the orval client (AuthtypesAuthDomainConfigDTO is now a
discriminated union) and updates the AuthDomain container, toggle, list
and tests to the new enabled/config/roleMapping root shape and the
renamed SAML spec keys.
2026-08-08 16:07:58 +05:30
grandwizard28
6175ff3fc2 chore(openapi): regenerate spec for auth domain kind/spec envelope 2026-08-08 16:00:21 +05:30
grandwizard28
db7ec08969 fix(authtypes): restructure auth domain payload into a kind/spec envelope
The auth domain config previously carried the discriminator (ssoType) and
the per-provider payloads as sibling fields, which cannot be expressed as
an OpenAPI discriminated union. AuthDomainConfig is now a kind/spec
envelope; ssoEnabled and roleMapping move to the root as enabled and
roleMapping. The persisted shape is unchanged: StorableAuthDomainConfig
keeps the legacy keys and conversions happen at the type boundary.
2026-08-08 15:59:32 +05:30
Nikhil Soni
a81c7d3f97 fix(sqlmigration): skip saved views with dangling org_id in migration 109 (#12469)
## Summary
- `restructureSavedViewSpec` (migration 109) bulk-inserts legacy
`saved_views` rows into the new `saved_view` table, which has an
enforced `org_id -> organizations(id)` FK (sqlite runs with
`foreign_keys=ON`).
- Some tenants hit `constraint failed: FOREIGN KEY constraint failed
(787)` on startup because a row's `org_id` didn't match any live
organization -- e.g. an org deleted before the old table had a cascading
FK, or an install where `org_id` was never backfilled
(`015_update_dashboards_savedviews` only backfills it when there's
exactly one org).
- Fix: fetch live organization IDs up front inside the same transaction,
and skip (with a `WarnContext` log, counted in `skipped`) any row whose
non-empty `org_id` isn't among them -- same treatment as the existing
empty-`org_id` skip.

Followup to https://github.com/SigNoz/signoz/pull/12342

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-08 07:03:40 +00:00
Tushar Vats
b2ff5ef99c fix(querier): use collector-stamped insert time for last_observed stats (#12455)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
The `telemetry.*.last_observed` stats took `max()` over client-supplied
event-time columns, so a single row with a skewed or corrupt timestamp
(a 2050-dated log, a `2^32−1`-second span) poisoned them indefinitely.

### What
- Traces/logs `last_observed` now reads `max(inserted_at)` — the
collector-stamped insert time added in SigNoz/signoz-otel-collector#875;
metrics reads `inserted_at_unix_milli` (metrics migration 1007).
- Each signal checks `hasColumnInTable` first and falls back to the
previous expression, so tenants without the schema migration keep
today's behavior and switch over automatically.

### Notes
- `created_at` is unusable here: pre-migration rows evaluate its
`now64(3)` default at read time, so `max(created_at)` always reads as
"now".
- Pre-migration rows read `inserted_at` as epoch, which `max()` ignores;
the all-old case lands on the existing `Unix() != 0` skip-guard.
- Future-dated garbage never TTLs out (TTL is keyed on the event
timestamp), which is why the old stat stayed wrong once poisoned.

### Testing
- Expressions validated against `clickhouse local`, including garbage
rows (`2^64−1`, `9.3e18` ns) and empty/pre-migration tables.
- `go build`, `go vet`, golangci-lint clean.

Fixes https://github.com/SigNoz/engineering-pod/issues/5864
2026-08-07 17:35:38 +00:00
Tushar Vats
58c21637a1 fix(tests): deflake SSO login tests — wait until the browser has left the idp after keycloak login (#12399)
Deflakes the SSO login tests (`callbackauthn` and `basepath`). They all
share the `idp_login` fixture, and after it clicked Keycloak's login
button it could hand control back to the test too early — in two
different ways ([example CI
failure](https://github.com/SigNoz/signoz/actions/runs/30898802502/job/91957986941)).

### What

The fixture used to wait for the login button to disappear and treat
that as "login is done". Two things go wrong with that:

1. **The page can vanish while we're looking at it.** Asking "is the
button still visible?" takes two round-trips to the browser: find
`kc-login`, then ask whether it's displayed. If Keycloak's redirect
lands between the two, the second call is asking about a node that no
longer exists. Selenium normally recognises that as a stale element and
quietly retries — but Keycloak → SigNoz is a *same-site* hop
(`localhost` → `localhost`), where the renderer survives the swap and
that detection can miss. The raw chromedriver error (`Node with given id
does not belong to the document`) then escapes and fails the test.
That's the CI failure above.

2. **The button disappearing doesn't mean login finished.** It only
means we left the login *page*. In the SAML flow Keycloak next serves a
small auto-submitting page — still on the IdP — and *that* POST is what
actually creates the user in SigNoz. So a test could go looking for the
user before SigNoz had ever seen the callback, and fail with `User ...
not found`. Reproduces locally on `test_idp_initiated_saml_authn`.

The wait now checks what the tests actually need: **the browser has left
the IdP host** (the hostname in the URL changed) *and* the login button
is gone.

### Guardrails

- Nothing is held across the navigation — the button is looked up fresh
on every poll with `find_elements`, so "gone" is simply an empty list,
never a question asked of a dying node.
- Any browser error during a poll is read as "still navigating, try
again" instead of failing the wait.
- The wait sits through everything that's still on Keycloak (the
`login-actions` hops, the SAML interstitial) and passes only once SigNoz
has handled the callback and redirected — so the user exists by the time
the test asserts on it.
- Wrong credentials still fail loudly: Keycloak re-renders the login
form on its own host, so the wait times out exactly as before.

One change, in the shared fixture — the OIDC and SAML flows in both
`callbackauthn` and `basepath` all go through it.

### Notes

- Failure 1 needs the redirect to land inside a ~2–5 ms window of a poll
that only runs every 500 ms, so it's effectively a loaded-CI-runner
lottery — which is why it's rare and CI-only. Failure 2 shows up
locally.
- Unrelated to the PR it fired on (#12382, query-builder only); the
identical SAML test passed in the same run.

### Testing

- Reproduced failure 1 outside pytest, with a probe driving real
headless **Chrome for Testing 151.0.7922.71** (the exact build from the
CI log) through a click → POST → same-site redirect that mimics the
Keycloak login flow, with server think-time near the 500 ms poll
boundary. Both waits ran verbatim, at their real polling rate:

  | Post-click wait | Logins | Failures |
  |---|---|---|
| old (`EC.invisibility_of_element`) | 400 | **3 × the exact CI
inspector error** |
  | new (left-the-IdP check) | 400 | **0** |

- Cross-checked the mechanism against the selenium 4.40 source with a
stubbed driver: the detached-node error does escape the old wait (it
only catches stale/not-found), while the new one absorbs it and passes
on the next poll. A bad-credentials control times out on both old and
new, so failure detection isn't weakened.
- Ran the full suites locally on the final fixture, with a fresh sqlite
+ wal store per suite (matching the failing CI leg): `basepath` 6/6, and
all 36 SSO/domain tests in `callbackauthn` — including
`test_idp_initiated_saml_authn`, which flaked with `User not found` on
the old wait in the same setup. (The one local non-pass,
`test_apply_license`, is unrelated: it asserts on wiremock's request
journal and the reused license-mock container is never reset between
runs — CI gets a fresh mock.)
- `make py-fmt` / `make py-lint` clean.

Fixes https://github.com/SigNoz/engineering-pod/issues/5850
2026-08-07 17:08:04 +00:00
Pandey
80fd5cc38a chore(deps): upgrade tests project dependencies (#12463)
#### Description

- `uv lock --upgrade` across the tests project: pytest 9.0.3→9.1.1, ruff
0.15.11→0.16.2, selenium 4.43→4.46, numpy 2.4.4→2.5.1, uvicorn
0.46→0.52.1, testcontainers 4.14.2→4.15.0, requests, sqlalchemy,
websockets, and the rest of the transitive set (zstandard dropped as no
longer required).
- Ignore `PLR0917` (too-many-positional-arguments), newly enforced by
ruff 0.16 — muted alongside the other `PLR09xx` complexity rules the
project already ignores (193 pre-existing hits, all in test/fixture
signatures).

#### Additional Information

- `py-fmt` (no reformats), `py-lint` (clean), and full integration-test
collection (1782 tests) pass on the upgraded toolchain. Runtime
verification against the docker stack was not run.
2026-08-07 16:59:13 +00:00
Pandey
fa05a73aef chore: convert lifecycle-free fixture-factories to plain functions (#12462)
#### Description

- Add the fixture-vs-function rule to `.claude/rules/pytest.md`: a
fixture earns its indirection only by owning setup/teardown (`yield` +
cleanup) or provisioning a resource; a stateless action or lookup is a
plain importable function in the matching `tests/fixtures/` module
taking `signoz`/`token` as ordinary arguments.
- Apply it to the three fixture-factories introduced in #12460 that have
no lifecycle: `delete_all_dashboards` (renamed from
`wipe_all_dashboards`) and `run_query_case` are now plain functions,
their modules deregistered from `pytest_plugins`, and all call sites
updated.
- Generalize `Metrics.load_from_file` with a `label_substitutions`
parameter (placeholder rewriting, e.g. `__START_TIME__` → runtime ISO
string) and drop the bespoke `load_pods_metrics`, which duplicated the
base-time rebase logic — `02_pods.py` now loads JSONL the same way as
every other inframonitoring suite file.

Follow-up promised in
https://github.com/SigNoz/signoz/pull/12460#discussion_r3737099384.
2026-08-07 16:53:07 +00:00
Pandey
38cc4d2bea chore: add pytest conventions rule and apply it across integration tests (#12460)
#### Description

- Add `.claude/rules/pytest.md` with conventions for the Python
integration suite. The lead rule: **no `_`-prefixed helper functions in
test modules** — inline the logic; repetition across tests is cheaper
than indirection; genuinely shared machinery becomes a fixture. Fixtures
live in `tests/fixtures/` only, never under `integration/tests/` — with
one exception: SigNoz-level fixtures (a suite spinning up SigNoz with
different envs via `create_signoz`/`create_migrator`) always belong in
that suite's `conftest.py`. Plus: fixture-factory over indirect
parametrization, skip at collection, config via explicit `--flags`,
snake_case parametrize ids, and collection gotchas (`python_files`
prefix matching, `--import-mode=importlib`).
- Apply the no-`_helper` rule across `tests/integration`: all 40
module-level `_` helpers eliminated in dashboard, inframonitoring,
promqlconformance, querier_json_body, querierlogs, queriermetrics, and
queriertraces. Pure transforms and request wrappers were inlined at
their call sites; case-table verifier callables became data flags with
inline branches; the two 115-line resource-evolution mega-helpers folded
into parametrized tests; shared machinery moved to `tests/fixtures/` as
fixture-factories (`wipe_all_dashboards`, `load_pods_metrics`,
`run_query_case`) registered via `pytest_plugins`.
- Apply the py-comments rule across `tests/`: drop module docstrings
that restate the filename, relocate the ones carrying real constraints
next to the code they constrain, delete function/class docstrings that
restate the identifier, trim narrated steps and Args/Returns
boilerplate.
- Fix camelCase parametrize ids in `queriermetrics/01_fill.py`
(`fillGaps`/`fillZero` → `fill_gaps`/`fill_zero`).

#### Additional Information

- All 629 tests in the touched suites collect cleanly;
`py-fmt`/`py-lint`/compileall pass. Runtime verification against the
docker stack was not run.
2026-08-07 16:14:11 +00:00
Srikanth Chekuri
e0b278e8e2 test(querier): pin keyless-row semantics for filter operators (#12456)
## Summary

Adds an integration-test matrix that pins the deliberate keyless-row
contract for filter operators, independent of any feature work:

- **Negative operators are a set complement over all rows.** A row that
does not carry the key at all must match `!=`, `NOT IN`, `NOT LIKE`, and
`NOT CONTAINS`. Users opt into presence explicitly with `AND key
EXISTS`.
- **Positive operators carry an implicit existence guard**
(`FilterOperator.AddDefaultExistsFilter`), so keyless rows never
false-positive against sentinel defaults.
- **`EXISTS` / `NOT EXISTS` partition rows exactly** by key presence,
and `!= x AND EXISTS` is the documented composition for "present and not
x".
- **Numeric attributes inherit the map-default sentinel**: a missing key
reads as `0`, so `num != 5` includes keyless rows while `num != 0`
excludes them. This conflation is deliberate and pinned by name
(`numeric_neq_zero_sentinel_conflation`) as the reference point for any
value-expression change.

Coverage: 46 cases — one shared matrix over traces and logs (resource
and attribute contexts), metric labels (series without the label), the
numeric sentinel, and the EXISTS composition. The contract, matrix, seed
data, and assertions live together in one file so the contract reads top
to bottom.

## Why

These semantics were enforced only implicitly by the operator list in
`AddDefaultExistsFilter`, with no test naming the intent. That gap
allows an implementation change to alter negative-filter results
silently and lets new tests calibrate expectations against the
implementation instead of the contract. The attribute names used here
are deliberately outside every semantic-convention family, so this file
pins the base contract regardless of the semconv overlay state and
serves as the oracle that family-field behavior
(`queriertraces/13_semconv_evolution.py` in the semconv stack) must
mirror.

## Testing

- `uv run pytest --basetemp=./tmp/ --reuse
integration/tests/queriercommon/06_keyless_semantics.py` — 46/46 passed
against a stack built from main-based sources, and 2×46 against a
long-lived shared stack, confirming the set-based assertions stay stable
under environment reuse.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-07 15:30:07 +00:00
Srikanth Chekuri
a711cda7ba feat: generate semantic convention families (#12441)
## Summary

Adds the semantic-convention evolution foundation:

- vendors the OpenTelemetry schema and SigNoz overlay
- generates Go and TypeScript family tables deterministically
- exposes the Go resolver API for family members, current names, and
historical names
- adds generation checks and unit tests

Related to #6143.

## Stack

1. #12441 — Foundations (base: main)
2. #12442 — Phase 1 query (base: #12441)
3. #12443 — Phase 1 services (base: #12442)
4. #12444 — Phase 1 closure gate (base: #12443)
5. #12445 — Phase 2 signals (base: #12444)
6. #12446 — Phase 3 migration UX (base: #12445)
7. #12447 — Phase 4 rollout (base: #12446)

**Current layer:** #12441

## Testing

- `go test ./scripts/semconv`
- `go test ./pkg/types/telemetrytypes/semconv`
- `make semconv-check`

## Risk and rollback

This layer is additive apart from CI generation checks. Roll back by
reverting this PR; no stored telemetry is changed.
2026-08-07 15:27:50 +00:00
144 changed files with 6738 additions and 3593 deletions

19
.claude/rules/pytest.md Normal file
View File

@@ -0,0 +1,19 @@
---
paths:
- "tests/**/*.py"
---
# pytest conventions
For the Python integration suite under `tests/`. Setup, running, and suite layout live in [`docs/contributing/tests/integration.md`](../../docs/contributing/tests/integration.md).
- **No `_`-prefixed helper functions in test modules — this is the rule that matters most.** A reader must be able to see what a test does in its body alone, without chasing private helpers that scatter the meaning across the file. Inline the logic: an expression, a comprehension, a few repeated lines are all fine — repetition across tests is cheaper than indirection. When several tests genuinely share non-trivial setup or assertions, that is what fixtures are for — in `tests/fixtures/`, see the next rule. A module-level `_helper()` is never the answer.
- **Fixtures live in `tests/fixtures/` — never under `integration/tests/`.** Not in test modules, not in suite `conftest.py` files. `tests/fixtures/` is the shared library (auth, signoz, clickhouse, logs/metrics/traces seeding, …): reuse what's there before writing anything new; when a new fixture is genuinely needed, add it to the matching `tests/fixtures/` module and register new modules in `tests/conftest.py` `pytest_plugins`. **The one exception: SigNoz-level fixtures in a suite's `conftest.py`.** A suite that needs its own SigNoz spun up with different envs (`create_signoz`/`create_migrator` with `env_overrides` + `cache_key` — e.g. basepath, metricreduction, querier_json_body) keeps that in its `conftest.py`; that is always okay.
- **Fixture only when there is a lifecycle; otherwise a plain function.** A fixture earns its indirection by owning setup/teardown (`yield` + cleanup — `insert_metrics` truncating on teardown) or by provisioning a resource (containers, SigNoz instances). A stateless action or lookup (`create_saved_view`, `find_saved_view_by_name`, wiping a resource list) is a plain importable function in the matching `tests/fixtures/` module, taking `signoz`/`token` as ordinary arguments — never wrap a plain callable in a fixture-factory just to inject `signoz`.
- **Fixtures own their cleanup.** When a test needs seeded state, put the seed + cleanup pair in a fixture (`yield`, then tear down) so tests in the same suite don't interfere — the pattern `insert_metrics` sets: yield a callable, truncate on teardown.
- **Fixture-factory over indirect parametrization.** A fixture that yields a callable (e.g. `insert_metrics(metrics)`) is clearer than `@pytest.mark.parametrize(..., indirect=True)` + `request.param` — the value is an explicit argument, not resolved by magic.
- **Skip at collection, not inside the test body.** Use `pytest.param(..., marks=pytest.mark.skip(reason="…"))` so a skipped case shows as SKIPPED-with-reason **and** short-circuits before its fixtures run (no environment spin-up for a test that won't execute).
- **Test config comes from explicit `--flags`, not the environment.** Wire configuration as pytest options declared in `tests/conftest.py` (`pytest_addoption` — e.g. `--sqlstore-provider`, `--clickhouse-version`); do **not** add `os.environ` fallbacks inside tests or fixtures.
- **snake_case parametrize ids.** `ids=["fill_gaps", "fill_zero"]`, not camelCase.
- **Name suite files with the two-digit prefix (`NN_*.py`).** `pyproject.toml` restricts collection to `[0-9][0-9]_*.py` (plus the bootstrap `setup.py` / `run.py`) — a file that doesn't match is silently never collected.
- **Always run pytest from `tests/`.** `--import-mode=importlib` is what allows same-basename files across suites (`querier/01_logs.py` vs `rawexportdata/01_logs.py`), but it disables pytest's implicit `sys.path` injection — `import fixtures` only resolves via `pythonpath = ["."]` from that rootdir.

View File

@@ -53,6 +53,21 @@ jobs:
with:
PRIMUS_REF: main
GO_VERSION: 1.24
semconv-generated:
if: |
github.event_name == 'merge_group' ||
(github.event_name == 'pull_request' && ! github.event.pull_request.head.repo.fork && github.event.pull_request.user.login != 'dependabot[bot]' && ! contains(github.event.pull_request.labels.*.name, 'safe-to-test')) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test'))
runs-on: ubuntu-latest
steps:
- name: self-checkout
uses: actions/checkout@v4
- name: go-install
uses: actions/setup-go@v5
with:
go-version: "1.24"
- name: check-semconv-generated-files
run: go run ./scripts/semconv -check
build:
if: |
github.event_name == 'merge_group' ||

View File

@@ -58,7 +58,6 @@ jobs:
- querierai
- rawexportdata
- promqlconformance
- promapiconformance
- querierauthz
- role
- rootuser

View File

@@ -233,6 +233,10 @@ py-clean: ## Clear all pycache and pytest cache from tests directory recursively
##############################################################
# generate commands
##############################################################
.PHONY: semconv-generate
semconv-generate: ## Regenerate semantic-convention families for Go and TypeScript
@go run ./scripts/semconv
.PHONY: gen-mocks
gen-mocks:
@echo ">> Generating mocks"

View File

@@ -464,27 +464,50 @@ components:
type: string
type: object
AuthtypesAuthDomainConfig:
discriminator:
mapping:
google: '#/components/schemas/AuthtypesAuthDomainConfigGoogle'
oidc: '#/components/schemas/AuthtypesAuthDomainConfigOIDC'
saml: '#/components/schemas/AuthtypesAuthDomainConfigSAML'
propertyName: kind
oneOf:
- $ref: '#/components/schemas/AuthtypesSamlConfig'
- $ref: '#/components/schemas/AuthtypesGoogleConfig'
- $ref: '#/components/schemas/AuthtypesOIDCConfig'
- $ref: '#/components/schemas/AuthtypesAuthDomainConfigSAML'
- $ref: '#/components/schemas/AuthtypesAuthDomainConfigGoogle'
- $ref: '#/components/schemas/AuthtypesAuthDomainConfigOIDC'
type: object
AuthtypesAuthDomainConfigGoogle:
properties:
googleAuthConfig:
$ref: '#/components/schemas/AuthtypesGoogleConfig'
oidcConfig:
$ref: '#/components/schemas/AuthtypesOIDCConfig'
roleMapping:
$ref: '#/components/schemas/AuthtypesRoleMapping'
samlConfig:
$ref: '#/components/schemas/AuthtypesSamlConfig'
ssoEnabled:
type: boolean
ssoType:
kind:
$ref: '#/components/schemas/AuthtypesAuthNProvider'
spec:
$ref: '#/components/schemas/AuthtypesGoogleConfig'
required:
- kind
- spec
type: object
AuthtypesAuthDomainConfigOIDC:
properties:
kind:
$ref: '#/components/schemas/AuthtypesAuthNProvider'
spec:
$ref: '#/components/schemas/AuthtypesOIDCConfig'
required:
- kind
- spec
type: object
AuthtypesAuthDomainConfigSAML:
properties:
kind:
$ref: '#/components/schemas/AuthtypesAuthNProvider'
spec:
$ref: '#/components/schemas/AuthtypesSamlConfig'
required:
- kind
- spec
type: object
AuthtypesAuthNProvider:
enum:
- google_auth
- google
- saml
- email_password
- oidc
@@ -531,12 +554,16 @@ components:
createdAt:
format: date-time
type: string
enabled:
type: boolean
id:
type: string
name:
type: string
orgId:
type: string
roleMapping:
$ref: '#/components/schemas/AuthtypesRoleMapping'
updatedAt:
format: date-time
type: string
@@ -616,6 +643,9 @@ components:
type: string
serviceAccountJson:
type: string
required:
- clientId
- clientSecret
type: object
AuthtypesOIDCConfig:
properties:
@@ -633,6 +663,10 @@ components:
type: string
issuerAlias:
type: string
required:
- issuer
- clientId
- clientSecret
type: object
AuthtypesOrgSessionContext:
properties:
@@ -654,8 +688,15 @@ components:
properties:
config:
$ref: '#/components/schemas/AuthtypesAuthDomainConfig'
enabled:
type: boolean
name:
type: string
roleMapping:
$ref: '#/components/schemas/AuthtypesRoleMapping'
required:
- name
- config
type: object
AuthtypesPostableEmailPasswordSession:
properties:
@@ -762,14 +803,18 @@ components:
properties:
attributeMapping:
$ref: '#/components/schemas/AuthtypesAttributeMapping'
certificate:
type: string
entityId:
type: string
insecureSkipAuthNRequestsSigned:
type: boolean
samlCert:
type: string
samlEntity:
type: string
samlIdp:
location:
type: string
required:
- entityId
- location
- certificate
type: object
AuthtypesSessionContext:
properties:
@@ -809,6 +854,12 @@ components:
properties:
config:
$ref: '#/components/schemas/AuthtypesAuthDomainConfig'
enabled:
type: boolean
roleMapping:
$ref: '#/components/schemas/AuthtypesRoleMapping'
required:
- config
type: object
AuthtypesUpdatableRole:
properties:
@@ -10511,275 +10562,6 @@ paths:
summary: Update public dashboard
tags:
- dashboard
/api/v1/domains:
get:
deprecated: false
description: This endpoint lists all auth domains
operationId: ListAuthDomains
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/AuthtypesGettableAuthDomain'
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: List all auth domains
tags:
- authdomains
post:
deprecated: false
description: This endpoint creates an auth domain
operationId: CreateAuthDomain
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/AuthtypesPostableAuthDomain'
responses:
"201":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/TypesIdentifiable'
status:
type: string
required:
- status
- data
type: object
description: Created
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Conflict
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Create auth domain
tags:
- authdomains
/api/v1/domains/{id}:
delete:
deprecated: false
description: This endpoint deletes an auth domain
operationId: DeleteAuthDomain
parameters:
- in: path
name: id
required: true
schema:
type: string
responses:
"204":
description: No Content
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Delete auth domain
tags:
- authdomains
get:
deprecated: false
description: This endpoint returns an auth domain by ID
operationId: GetAuthDomain
parameters:
- in: path
name: id
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/AuthtypesGettableAuthDomain'
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Get auth domain by ID
tags:
- authdomains
put:
deprecated: false
description: This endpoint updates an auth domain
operationId: UpdateAuthDomain
parameters:
- in: path
name: id
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/AuthtypesUpdatableAuthDomain'
responses:
"204":
description: No Content
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Conflict
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Update auth domain
tags:
- authdomains
/api/v1/downtime_schedules:
get:
deprecated: false
@@ -14804,6 +14586,275 @@ paths:
summary: Update user preference
tags:
- preferences
/api/v2/auth_domains:
get:
deprecated: false
description: This endpoint lists all auth domains
operationId: ListAuthDomains
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/AuthtypesGettableAuthDomain'
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: List all auth domains
tags:
- authdomains
post:
deprecated: false
description: This endpoint creates an auth domain
operationId: CreateAuthDomain
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/AuthtypesPostableAuthDomain'
responses:
"201":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/TypesIdentifiable'
status:
type: string
required:
- status
- data
type: object
description: Created
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Conflict
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Create auth domain
tags:
- authdomains
/api/v2/auth_domains/{id}:
delete:
deprecated: false
description: This endpoint deletes an auth domain
operationId: DeleteAuthDomain
parameters:
- in: path
name: id
required: true
schema:
type: string
responses:
"204":
description: No Content
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Delete auth domain
tags:
- authdomains
get:
deprecated: false
description: This endpoint returns an auth domain by ID
operationId: GetAuthDomain
parameters:
- in: path
name: id
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/AuthtypesGettableAuthDomain'
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Get auth domain by ID
tags:
- authdomains
put:
deprecated: false
description: This endpoint updates an auth domain
operationId: UpdateAuthDomain
parameters:
- in: path
name: id
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/AuthtypesUpdatableAuthDomain'
responses:
"204":
description: No Content
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Conflict
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Update auth domain
tags:
- authdomains
/api/v2/dashboard_views:
get:
deprecated: false

View File

@@ -349,7 +349,7 @@ func (Step) JSONSchema() (jsonschema.Schema, error) {
### `oneOf` with a discriminator
For a sum type whose variants are keyed by a property (e.g. `kind`), expose the variants via `JSONSchemaOneOf()` and add a discriminator. Without it, code generators intersect the variants (`A & B & C`) instead of producing a clean discriminated union (`A | B | C`).
For a sum type whose variants are keyed by a property (e.g. `kind`), expose the variants via `JSONSchemaOneOf()` and add a discriminator. Without it, code generators intersect the variants (`A & B & C`) instead of producing a clean discriminated union (`A | B | C`). How to model the sum type itself is covered in [types.md](types.md#sum-types-the-kindspec-envelope) — this section is only about its schema.
The parent keeps its `JSONSchemaOneOf()` (the `oneOf` itself) and *additionally* tags it via `PrepareJSONSchema` with the `x-signoz-discriminator` extension; `signoz.attachDiscriminators` then promotes that marker to a real OpenAPI 3 `discriminator` (and strips the duplicate parent properties) after reflection.

View File

@@ -61,31 +61,37 @@ type Channel struct {
```go
type AuthDomain struct {
storableAuthDomain *StorableAuthDomain
authDomainConfig *AuthDomainConfig
storableAuthDomain *StorableAuthDomain
storableAuthDomainConfig *StorableAuthDomainConfig
}
type StorableAuthDomain struct {
bun.BaseModel `bun:"table:auth_domain"`
types.Identifiable
Name string `bun:"name"`
Data string `bun:"data"` // AuthDomainConfig serialized as JSON
Data string `bun:"data"` // StorableAuthDomainConfig serialized as JSON
OrgID valuer.UUID `bun:"org_id"`
types.TimeAuditable
}
type PostableAuthDomain struct {
Config AuthDomainConfig `json:"config"`
Name string `json:"name"`
Name string `json:"name" required:"true"`
Enabled bool `json:"enabled"`
Config AuthDomainConfig `json:"config" required:"true"`
RoleMapping *RoleMapping `json:"roleMapping"`
}
type UpdateableAuthDomain struct {
Config AuthDomainConfig `json:"config"` // Name intentionally absent
type UpdatableAuthDomain struct {
Enabled bool `json:"enabled"` // Name intentionally absent
Config AuthDomainConfig `json:"config" required:"true"`
RoleMapping *RoleMapping `json:"roleMapping"`
}
type GettableAuthDomain struct {
*StorableAuthDomain
*AuthDomainConfig
StorableAuthDomain
Enabled bool `json:"enabled"`
Config AuthDomainConfig `json:"config"`
RoleMapping *RoleMapping `json:"roleMapping"`
AuthNProviderInfo *AuthNProviderInfo `json:"authNProviderInfo"`
}
```
@@ -93,11 +99,74 @@ type GettableAuthDomain struct {
Each flavor exists for a concrete reason:
- `StorableAuthDomain` stores the typed config as an opaque `Data string` column, so the schema does not need to migrate every time a config field is added.
- `PostableAuthDomain` carries the config as a structured object (not a string) for the request.
- `UpdateableAuthDomain` excludes `Name` because a domain's name cannot change after creation.
- `PostableAuthDomain` carries the config as a structured object (not a string) for the request. `AuthDomainConfig` is a kind/spec envelope — see the next section.
- `UpdatableAuthDomain` excludes `Name` because a domain's name cannot change after creation.
- `GettableAuthDomain` adds `AuthNProviderInfo`, which is derived at read time and never persisted.
The core `AuthDomain` holds the two live halves — `storableAuthDomain` and `authDomainConfig` — and owns business methods such as `Update(config)`. Conversions use the `New<Output>From<Input>` form: `NewAuthDomainFromConfig`, `NewAuthDomainFromStorableAuthDomain`, `NewGettableAuthDomainFromAuthDomain`.
The core `AuthDomain` holds the two live halves — `storableAuthDomain` and `storableAuthDomainConfig` — and owns business methods such as `Update(updatable)`. Conversions use the `New<Output>From<Input>` form: `NewAuthDomainFromPostableAuthDomain`, `NewAuthDomainFromStorableAuthDomain`, `NewGettableAuthDomainFromAuthDomain`.
## Sum types: the kind/spec envelope
When a domain type is a *sum type* — exactly one of several variants, selected by a discriminator — model it as an envelope with a `kind` and a `spec`:
```go
type AuthDomainConfig struct {
Kind AuthNProvider `json:"kind" required:"true"`
Spec any `json:"spec" required:"true"`
}
```
```json
{ "kind": "saml", "spec": { "entityId": "...", "location": "...", "certificate": "..." } }
```
`Kind` is a `valuer.String` enum implementing `Enum()`; `Spec` holds exactly one concrete variant type. `RuleThresholdData` and `EvaluationEnvelope` in `pkg/types/ruletypes/` and `AuthDomainConfig` in `pkg/types/authtypes/` are the canonical examples. (`QueryEnvelope` in querybuildertypes uses `type` as the discriminator key for historical reasons; new envelopes use `kind`.)
### The envelope goes at the point of variance, not the resource root
Put the envelope on the field that actually varies. The resource root is almost never a sum type — an auth domain always has a `name`, `enabled`, and `roleMapping` regardless of provider; only its provider configuration varies, so the envelope is the `config` field:
```json
{ "name": "signoz.io", "enabled": true, "config": { "kind": "saml", "spec": { "..." : "..." } }, "roleMapping": null }
```
Hoisting `kind`/`spec` to the root would turn the whole resource into a `oneOf`: every flavor (`PostableX`, `UpdatableX`, `GettableX`) then needs one variant schema per kind, each repeating the common fields; every new common field has to be added to all of them; and generated clients get unions of large objects instead of one small union that narrows on `config.kind`. A root-level `kind` also collides with the resource-model meaning of the word — in the Kubernetes/Perses model, root `kind` answers "what resource is this" (`Dashboard`), never "which flavor of config does it hold".
The other domains already follow this placement:
- **Rules** — plain root; envelopes on the varying fields: `thresholds: {kind, spec}` and `evaluation: {kind, spec}`.
- **Dashboards** — Perses resource model: metadata at the root plus one typed `spec`; the unions sit deep inside, at each panel/query/variable plugin (`{kind, spec}` in `perses_plugin_wrappers.go`).
- **Saved views** — root `{schemaVersion, spec}`, where `spec` is a *versioning* envelope holding one fixed type, not a union; the unions are inside it (`spec.queries: [{type, spec}]`). Same word, different job — a versioned body is not a discriminated union.
### Why this tagging style
Of the union encodings in common use, the envelope is the *adjacently tagged* one — tag and payload side by side — as used by the Kubernetes resource model, Perses plugins, CloudFormation (`Type` + `Properties`), and Grafana provisioning (`type` + `settings`). Variant payloads stay collision-free, and each kind maps to a named wrapper schema that carries the discriminator, which is exactly what OpenAPI generators need. The alternatives lose on those points: *internally tagged* (`{"type": "saml", ...fields flattened}` — Stripe, GitHub webhooks) mixes common and variant fields, admits cross-variant key collisions, and forces every variant schema to redeclare the discriminator; *sibling optional fields* (`{"type": "saml", "samlConfig": {}, "oidcConfig": {}}` — classic Kubernetes `VolumeSource`, and the pre-envelope auth domain) is the anti-pattern the first rule below exists to prevent.
The rules that make the envelope work:
- **Never model variants as sibling fields.** A struct with `SAML *SamlConfig`, `Google *GoogleConfig`, `OIDC *OIDCConfig` next to a discriminator cannot be expressed as an OpenAPI discriminated union, forces nilability checks on every consumer, and silently admits contradictory payloads (kind=saml with a google config). The chosen variant *is* the payload.
- **The envelope owns `UnmarshalJSON`.** Decode `kind` first, then switch on it to decode and validate the matching concrete type into `Spec`. Unknown kinds and missing specs are rejected at the boundary:
```go
func (typ *AuthDomainConfig) UnmarshalJSON(data []byte) error {
var raw map[string]json.RawMessage
// ... unmarshal raw, decode raw["kind"] ...
switch kind {
case AuthNProviderSAML:
spec := SamlConfig{}
if err := json.Unmarshal(raw["spec"], &spec); err != nil {
return err
}
typ.Spec = spec
// ... one case per kind, default rejects ...
}
typ.Kind = kind
return nil
}
```
- **Consumers type-assert on `Spec`** (`config.Spec.(SamlConfig)`) after switching on `Kind`. If assertion sites multiply, add typed accessors on the envelope (see `EvaluationEnvelope.GetEvaluation()`).
- **OpenAPI needs one unexported variant struct per kind** (`authDomainConfigSAML{Kind; Spec SamlConfig}`), exposed via `JSONSchemaOneOf()` and mapped via `PrepareJSONSchema` with the `x-signoz-discriminator` extension. The schema mechanics are covered in [handler.md](handler.md#oneof-with-a-discriminator).
- **A persisted legacy shape stays in a `StorableX`.** If rows were written before the envelope existed, keep the old JSON shape in a storable type (`StorableAuthDomainConfig` keeps `ssoType` + sibling configs) and convert to/from the envelope at the type boundary — the data layer never changes shape retroactively.
## Conventions that tie the flavors together
@@ -139,6 +208,8 @@ Both are optional. Do not introduce them if `PostableX` already covers the case.
- Every domain package defines the core type `X`. Only `X` is mandatory.
- Add `PostableX` / `GettableX` / `UpdatableX` / `StorableX` one at a time, only when the shape actually diverges from `X`.
- Model sum types as a `{kind, spec}` envelope with a validating `UnmarshalJSON` — never as sibling variant fields next to a discriminator.
- The envelope goes on the field that varies, never at the resource root — common fields stay on the resource, outside the union.
- Domain logic lives on `X`, not on the flavor types.
- Conversions can be a `New<Output>From<Input>` constructor or a receiver-style `ToY()` method — pick whichever reads best at the call site.
- Use a type alias when two shapes are truly identical.

View File

@@ -53,7 +53,7 @@ func New(store authtypes.AuthNStore, licensing licensing.Licensing, providerSett
}
func (a *AuthN) LoginURL(ctx context.Context, siteURL *url.URL, authDomain *authtypes.AuthDomain) (string, error) {
if authDomain.AuthDomainConfig().AuthNProvider != authtypes.AuthNProviderOIDC {
if authDomain.StorableAuthDomainConfig().AuthNProvider != authtypes.AuthNProviderOIDC {
return "", errors.Newf(errors.TypeInternal, authtypes.ErrCodeAuthDomainMismatch, "domain type is not oidc")
}
@@ -106,14 +106,14 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
return nil, err
}
if claims == nil && authDomain.AuthDomainConfig().OIDC.GetUserInfo {
if claims == nil && authDomain.StorableAuthDomainConfig().OIDC.GetUserInfo {
claims, err = a.claimsFromUserInfo(ctx, oidcProvider, token)
if err != nil {
return nil, err
}
}
emailClaim, ok := claims[authDomain.AuthDomainConfig().OIDC.ClaimMapping.Email].(string)
emailClaim, ok := claims[authDomain.StorableAuthDomainConfig().OIDC.ClaimMapping.Email].(string)
if !ok {
return nil, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "oidc: missing email in claims")
}
@@ -123,7 +123,7 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "oidc: failed to parse email").WithAdditional(err.Error())
}
if !authDomain.AuthDomainConfig().OIDC.InsecureSkipEmailVerified {
if !authDomain.StorableAuthDomainConfig().OIDC.InsecureSkipEmailVerified {
emailVerifiedClaim, ok := claims["email_verified"].(bool)
if !ok {
return nil, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "oidc: missing email_verified in claims")
@@ -135,14 +135,14 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
}
name := ""
if nameClaim := authDomain.AuthDomainConfig().OIDC.ClaimMapping.Name; nameClaim != "" {
if nameClaim := authDomain.StorableAuthDomainConfig().OIDC.ClaimMapping.Name; nameClaim != "" {
if n, ok := claims[nameClaim].(string); ok {
name = n
}
}
var groups []string
if groupsClaim := authDomain.AuthDomainConfig().OIDC.ClaimMapping.Groups; groupsClaim != "" {
if groupsClaim := authDomain.StorableAuthDomainConfig().OIDC.ClaimMapping.Groups; groupsClaim != "" {
if claimValue, exists := claims[groupsClaim]; exists {
switch g := claimValue.(type) {
case []any:
@@ -161,7 +161,7 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
}
role := ""
if roleClaim := authDomain.AuthDomainConfig().OIDC.ClaimMapping.Role; roleClaim != "" {
if roleClaim := authDomain.StorableAuthDomainConfig().OIDC.ClaimMapping.Role; roleClaim != "" {
if r, ok := claims[roleClaim].(string); ok {
role = r
}
@@ -177,11 +177,11 @@ func (a *AuthN) ProviderInfo(ctx context.Context, authDomain *authtypes.AuthDoma
}
func (a *AuthN) oidcProviderAndoauth2Config(ctx context.Context, siteURL *url.URL, authDomain *authtypes.AuthDomain) (*oidc.Provider, *oauth2.Config, error) {
if authDomain.AuthDomainConfig().OIDC.IssuerAlias != "" {
ctx = oidc.InsecureIssuerURLContext(ctx, authDomain.AuthDomainConfig().OIDC.IssuerAlias)
if authDomain.StorableAuthDomainConfig().OIDC.IssuerAlias != "" {
ctx = oidc.InsecureIssuerURLContext(ctx, authDomain.StorableAuthDomainConfig().OIDC.IssuerAlias)
}
oidcProvider, err := oidc.NewProvider(ctx, authDomain.AuthDomainConfig().OIDC.Issuer)
oidcProvider, err := oidc.NewProvider(ctx, authDomain.StorableAuthDomainConfig().OIDC.Issuer)
if err != nil {
return nil, nil, err
}
@@ -189,13 +189,13 @@ func (a *AuthN) oidcProviderAndoauth2Config(ctx context.Context, siteURL *url.UR
scopes := make([]string, len(defaultScopes))
copy(scopes, defaultScopes)
if authDomain.AuthDomainConfig().RoleMapping != nil && len(authDomain.AuthDomainConfig().RoleMapping.GroupMappings) > 0 {
if authDomain.StorableAuthDomainConfig().RoleMapping != nil && len(authDomain.StorableAuthDomainConfig().RoleMapping.GroupMappings) > 0 {
scopes = append(scopes, "groups")
}
return oidcProvider, &oauth2.Config{
ClientID: authDomain.AuthDomainConfig().OIDC.ClientID,
ClientSecret: authDomain.AuthDomainConfig().OIDC.ClientSecret,
ClientID: authDomain.StorableAuthDomainConfig().OIDC.ClientID,
ClientSecret: authDomain.StorableAuthDomainConfig().OIDC.ClientSecret,
Endpoint: oidcProvider.Endpoint(),
Scopes: scopes,
RedirectURL: (&url.URL{
@@ -212,7 +212,7 @@ func (a *AuthN) claimsFromIDToken(ctx context.Context, authDomain *authtypes.Aut
return nil, errors.New(errors.TypeNotFound, errors.CodeNotFound, "oidc: no id_token in token response")
}
verifier := provider.Verifier(&oidc.Config{ClientID: authDomain.AuthDomainConfig().OIDC.ClientID})
verifier := provider.Verifier(&oidc.Config{ClientID: authDomain.StorableAuthDomainConfig().OIDC.ClientID})
idToken, err := verifier.Verify(ctx, rawIDToken)
if err != nil {
return nil, errors.Newf(errors.TypeForbidden, errors.CodeForbidden, "oidc: failed to verify token").WithAdditional(err.Error())

View File

@@ -40,7 +40,7 @@ func New(ctx context.Context, store authtypes.AuthNStore, licensing licensing.Li
}
func (a *AuthN) LoginURL(ctx context.Context, siteURL *url.URL, authDomain *authtypes.AuthDomain) (string, error) {
if authDomain.AuthDomainConfig().AuthNProvider != authtypes.AuthNProviderSAML {
if authDomain.StorableAuthDomainConfig().AuthNProvider != authtypes.AuthNProviderSAML {
return "", errors.Newf(errors.TypeInternal, authtypes.ErrCodeAuthDomainMismatch, "saml: domain type is not saml")
}
@@ -101,19 +101,19 @@ func (a *AuthN) HandleCallback(ctx context.Context, formValues url.Values) (*aut
}
name := ""
if nameAttribute := authDomain.AuthDomainConfig().SAML.AttributeMapping.Name; nameAttribute != "" {
if nameAttribute := authDomain.StorableAuthDomainConfig().SAML.AttributeMapping.Name; nameAttribute != "" {
if val := assertionInfo.Values.Get(nameAttribute); val != "" {
name = val
}
}
var groups []string
if groupAttribute := authDomain.AuthDomainConfig().SAML.AttributeMapping.Groups; groupAttribute != "" {
if groupAttribute := authDomain.StorableAuthDomainConfig().SAML.AttributeMapping.Groups; groupAttribute != "" {
groups = assertionInfo.Values.GetAll(groupAttribute)
}
role := ""
if roleAttribute := authDomain.AuthDomainConfig().SAML.AttributeMapping.Role; roleAttribute != "" {
if roleAttribute := authDomain.StorableAuthDomainConfig().SAML.AttributeMapping.Role; roleAttribute != "" {
if val := assertionInfo.Values.Get(roleAttribute); val != "" {
role = val
}
@@ -142,11 +142,11 @@ func (a *AuthN) serviceProvider(siteURL *url.URL, authDomain *authtypes.AuthDoma
// The ServiceProviderIssuer is the client id in case of keycloak. Since we set it to the host here, we need to set the client id == host in keycloak.
// For AWSSSO, this is the value of Application SAML audience.
return &saml2.SAMLServiceProvider{
IdentityProviderSSOURL: authDomain.AuthDomainConfig().SAML.SamlIdp,
IdentityProviderIssuer: authDomain.AuthDomainConfig().SAML.SamlEntity,
IdentityProviderSSOURL: authDomain.StorableAuthDomainConfig().SAML.Location,
IdentityProviderIssuer: authDomain.StorableAuthDomainConfig().SAML.EntityID,
ServiceProviderIssuer: siteURL.Host,
AssertionConsumerServiceURL: acsURL.String(),
SignAuthnRequests: !authDomain.AuthDomainConfig().SAML.InsecureSkipAuthNRequestsSigned,
SignAuthnRequests: !authDomain.StorableAuthDomainConfig().SAML.InsecureSkipAuthNRequestsSigned,
AllowMissingAttributes: true,
IDPCertificateStore: certStore,
SPKeyStore: dsig.RandomKeyStoreForTest(),
@@ -159,15 +159,15 @@ func (a *AuthN) getCertificateStore(authDomain *authtypes.AuthDomain) (dsig.X509
}
var certBytes []byte
if strings.Contains(authDomain.AuthDomainConfig().SAML.SamlCert, "-----BEGIN CERTIFICATE-----") {
block, _ := pem.Decode([]byte(authDomain.AuthDomainConfig().SAML.SamlCert))
if strings.Contains(authDomain.StorableAuthDomainConfig().SAML.Certificate, "-----BEGIN CERTIFICATE-----") {
block, _ := pem.Decode([]byte(authDomain.StorableAuthDomainConfig().SAML.Certificate))
if block == nil {
return certStore, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "no valid pem cert found")
}
certBytes = block.Bytes
} else {
certData, err := base64.StdEncoding.DecodeString(authDomain.AuthDomainConfig().SAML.SamlCert)
certData, err := base64.StdEncoding.DecodeString(authDomain.StorableAuthDomainConfig().SAML.Certificate)
if err != nil {
return certStore, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to read certificate: %s", err.Error())
}

View File

@@ -38,14 +38,14 @@ import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
*/
export const listAuthDomains = (signal?: AbortSignal) => {
return GeneratedAPIInstance<ListAuthDomains200>({
url: `/api/v1/domains`,
url: `/api/v2/auth_domains`,
method: 'GET',
signal,
});
};
export const getListAuthDomainsQueryKey = () => {
return [`/api/v1/domains`] as const;
return [`/api/v2/auth_domains`] as const;
};
export const getListAuthDomainsQueryOptions = <
@@ -125,7 +125,7 @@ export const createAuthDomain = (
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<CreateAuthDomain201>({
url: `/api/v1/domains`,
url: `/api/v2/auth_domains`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: authtypesPostableAuthDomainDTO,
@@ -208,7 +208,7 @@ export const deleteAuthDomain = (
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v1/domains/${id}`,
url: `/api/v2/auth_domains/${id}`,
method: 'DELETE',
signal,
});
@@ -287,7 +287,7 @@ export const getAuthDomain = (
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<GetAuthDomain200>({
url: `/api/v1/domains/${id}`,
url: `/api/v2/auth_domains/${id}`,
method: 'GET',
signal,
});
@@ -296,7 +296,7 @@ export const getAuthDomain = (
export const getGetAuthDomainQueryKey = ({
id,
}: GetAuthDomainPathParameters) => {
return [`/api/v1/domains/${id}`] as const;
return [`/api/v2/auth_domains/${id}`] as const;
};
export const getGetAuthDomainQueryOptions = <
@@ -389,7 +389,7 @@ export const updateAuthDomain = (
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v1/domains/${id}`,
url: `/api/v2/auth_domains/${id}`,
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
data: authtypesUpdatableAuthDomainDTO,

View File

@@ -1861,8 +1861,19 @@ export interface AuthtypesAttributeMappingDTO {
role?: string;
}
export enum AuthtypesAuthDomainConfigSAMLDTOKind {
saml = 'saml',
}
export interface AuthtypesSamlConfigDTO {
attributeMapping?: AuthtypesAttributeMappingDTO;
/**
* @type string
*/
certificate: string;
/**
* @type string
*/
entityId: string;
/**
* @type boolean
*/
@@ -1870,17 +1881,21 @@ export interface AuthtypesSamlConfigDTO {
/**
* @type string
*/
samlCert?: string;
/**
* @type string
*/
samlEntity?: string;
/**
* @type string
*/
samlIdp?: string;
location: string;
}
export interface AuthtypesAuthDomainConfigSAMLDTO {
/**
* @type string
* @enum saml
*/
kind: AuthtypesAuthDomainConfigSAMLDTOKind;
spec: AuthtypesSamlConfigDTO;
}
export enum AuthtypesAuthDomainConfigGoogleDTOKind {
google = 'google',
}
export type AuthtypesGoogleConfigDTODomainToAdminEmail = {
[key: string]: string;
};
@@ -1893,11 +1908,11 @@ export interface AuthtypesGoogleConfigDTO {
/**
* @type string
*/
clientId?: string;
clientId: string;
/**
* @type string
*/
clientSecret?: string;
clientSecret: string;
/**
* @type object
*/
@@ -1924,16 +1939,28 @@ export interface AuthtypesGoogleConfigDTO {
serviceAccountJson?: string;
}
export interface AuthtypesAuthDomainConfigGoogleDTO {
/**
* @type string
* @enum google
*/
kind: AuthtypesAuthDomainConfigGoogleDTOKind;
spec: AuthtypesGoogleConfigDTO;
}
export enum AuthtypesAuthDomainConfigOIDCDTOKind {
oidc = 'oidc',
}
export interface AuthtypesOIDCConfigDTO {
claimMapping?: AuthtypesAttributeMappingDTO;
/**
* @type string
*/
clientId?: string;
clientId: string;
/**
* @type string
*/
clientSecret?: string;
clientSecret: string;
/**
* @type boolean
*/
@@ -1945,79 +1972,33 @@ export interface AuthtypesOIDCConfigDTO {
/**
* @type string
*/
issuer?: string;
issuer: string;
/**
* @type string
*/
issuerAlias?: string;
}
export type AuthtypesRoleMappingDTOGroupMappingsAnyOf = {
[key: string]: string;
};
/**
* @nullable
*/
export type AuthtypesRoleMappingDTOGroupMappings =
AuthtypesRoleMappingDTOGroupMappingsAnyOf | null;
export interface AuthtypesRoleMappingDTO {
export interface AuthtypesAuthDomainConfigOIDCDTO {
/**
* @type string
* @enum oidc
*/
defaultRole?: string;
/**
* @type object,null
*/
groupMappings?: AuthtypesRoleMappingDTOGroupMappings;
/**
* @type boolean
*/
useRoleAttribute?: boolean;
kind: AuthtypesAuthDomainConfigOIDCDTOKind;
spec: AuthtypesOIDCConfigDTO;
}
export type AuthtypesAuthDomainConfigDTO =
| AuthtypesAuthDomainConfigSAMLDTO
| AuthtypesAuthDomainConfigGoogleDTO
| AuthtypesAuthDomainConfigOIDCDTO;
export enum AuthtypesAuthNProviderDTO {
google_auth = 'google_auth',
google = 'google',
saml = 'saml',
email_password = 'email_password',
oidc = 'oidc',
}
export type AuthtypesAuthDomainConfigDTO =
| (AuthtypesSamlConfigDTO & {
googleAuthConfig?: AuthtypesGoogleConfigDTO;
oidcConfig?: AuthtypesOIDCConfigDTO;
roleMapping?: AuthtypesRoleMappingDTO;
samlConfig?: AuthtypesSamlConfigDTO;
/**
* @type boolean
*/
ssoEnabled?: boolean;
ssoType?: AuthtypesAuthNProviderDTO;
})
| (AuthtypesGoogleConfigDTO & {
googleAuthConfig?: AuthtypesGoogleConfigDTO;
oidcConfig?: AuthtypesOIDCConfigDTO;
roleMapping?: AuthtypesRoleMappingDTO;
samlConfig?: AuthtypesSamlConfigDTO;
/**
* @type boolean
*/
ssoEnabled?: boolean;
ssoType?: AuthtypesAuthNProviderDTO;
})
| (AuthtypesOIDCConfigDTO & {
googleAuthConfig?: AuthtypesGoogleConfigDTO;
oidcConfig?: AuthtypesOIDCConfigDTO;
roleMapping?: AuthtypesRoleMappingDTO;
samlConfig?: AuthtypesSamlConfigDTO;
/**
* @type boolean
*/
ssoEnabled?: boolean;
ssoType?: AuthtypesAuthNProviderDTO;
});
export interface AuthtypesAuthNProviderInfoDTO {
/**
* @type string,null
@@ -2055,6 +2036,31 @@ export interface AuthtypesDeprecatedPostableUserRoleDTO {
id: string;
}
export type AuthtypesRoleMappingDTOGroupMappingsAnyOf = {
[key: string]: string;
};
/**
* @nullable
*/
export type AuthtypesRoleMappingDTOGroupMappings =
AuthtypesRoleMappingDTOGroupMappingsAnyOf | null;
export interface AuthtypesRoleMappingDTO {
/**
* @type string
*/
defaultRole?: string;
/**
* @type object,null
*/
groupMappings?: AuthtypesRoleMappingDTOGroupMappings;
/**
* @type boolean
*/
useRoleAttribute?: boolean;
}
export interface AuthtypesGettableAuthDomainDTO {
authNProviderInfo?: AuthtypesAuthNProviderInfoDTO;
config?: AuthtypesAuthDomainConfigDTO;
@@ -2063,6 +2069,10 @@ export interface AuthtypesGettableAuthDomainDTO {
* @format date-time
*/
createdAt?: string;
/**
* @type boolean
*/
enabled?: boolean;
/**
* @type string
*/
@@ -2075,6 +2085,7 @@ export interface AuthtypesGettableAuthDomainDTO {
* @type string
*/
orgId?: string;
roleMapping?: AuthtypesRoleMappingDTO;
/**
* @type string
* @format date-time
@@ -2271,11 +2282,16 @@ export interface AuthtypesOrgSessionContextDTO {
}
export interface AuthtypesPostableAuthDomainDTO {
config?: AuthtypesAuthDomainConfigDTO;
config: AuthtypesAuthDomainConfigDTO;
/**
* @type boolean
*/
enabled?: boolean;
/**
* @type string
*/
name?: string;
name: string;
roleMapping?: AuthtypesRoleMappingDTO;
}
export interface AuthtypesPostableEmailPasswordSessionDTO {
@@ -2408,7 +2424,12 @@ export interface AuthtypesTransactionDTO {
}
export interface AuthtypesUpdatableAuthDomainDTO {
config?: AuthtypesAuthDomainConfigDTO;
config: AuthtypesAuthDomainConfigDTO;
/**
* @type boolean
*/
enabled?: boolean;
roleMapping?: AuthtypesRoleMappingDTO;
}
export interface AuthtypesUpdatableRoleDTO {
@@ -10425,42 +10446,6 @@ export type CreatePublicDashboard201 = {
export type UpdatePublicDashboardPathParameters = {
id: string;
};
export type ListAuthDomains200 = {
/**
* @type array
*/
data: AuthtypesGettableAuthDomainDTO[];
/**
* @type string
*/
status: string;
};
export type CreateAuthDomain201 = {
data: TypesIdentifiableDTO;
/**
* @type string
*/
status: string;
};
export type DeleteAuthDomainPathParameters = {
id: string;
};
export type GetAuthDomainPathParameters = {
id: string;
};
export type GetAuthDomain200 = {
data: AuthtypesGettableAuthDomainDTO;
/**
* @type string
*/
status: string;
};
export type UpdateAuthDomainPathParameters = {
id: string;
};
export type ListDowntimeSchedulesParams = {
/**
* @type boolean,null
@@ -11159,6 +11144,42 @@ export type GetUserPreference200 = {
export type UpdateUserPreferencePathParameters = {
name: string;
};
export type ListAuthDomains200 = {
/**
* @type array
*/
data: AuthtypesGettableAuthDomainDTO[];
/**
* @type string
*/
status: string;
};
export type CreateAuthDomain201 = {
data: TypesIdentifiableDTO;
/**
* @type string
*/
status: string;
};
export type DeleteAuthDomainPathParameters = {
id: string;
};
export type GetAuthDomainPathParameters = {
id: string;
};
export type GetAuthDomain200 = {
data: AuthtypesGettableAuthDomainDTO;
/**
* @type string
*/
status: string;
};
export type UpdateAuthDomainPathParameters = {
id: string;
};
export type ListDashboardViews200 = {
data: DashboardtypesListableDashboardViewDTO;
/**

View File

@@ -0,0 +1,32 @@
// Code generated by scripts/semconv. DO NOT EDIT.
export type SemconvFamily = {
readonly current: string;
readonly old: readonly string[];
readonly kind: 'attribute' | 'metric';
readonly contexts: readonly string[];
readonly signals: readonly string[];
readonly applyToMetrics: readonly string[];
readonly valueMap: Readonly<Record<string, string>>;
};
export const SEMCONV_FAMILIES: readonly SemconvFamily[] = [
{
current: 'db.system.name',
old: ['db.system'],
kind: 'attribute',
contexts: [],
signals: [],
applyToMetrics: [],
valueMap: {},
},
{
current: 'deployment.environment.name',
old: ['deployment.environment'],
kind: 'attribute',
contexts: [],
signals: [],
applyToMetrics: [],
valueMap: {},
},
] as const;

View File

@@ -16,7 +16,7 @@ interface AuthNProvider {
function getAuthNProviders(samlEnabled: boolean): AuthNProvider[] {
return [
{
key: AuthtypesAuthNProviderDTO.google_auth,
key: AuthtypesAuthNProviderDTO.google,
title: 'Google Apps Authentication',
description: 'Let members sign-in with a Google workspace account',
icon: <SolidGoogle size={37} />,

View File

@@ -8,6 +8,10 @@ import {
useUpdateAuthDomain,
} from 'api/generated/services/authdomains';
import {
AuthtypesAuthDomainConfigDTO,
AuthtypesAuthDomainConfigGoogleDTOKind,
AuthtypesAuthDomainConfigOIDCDTOKind,
AuthtypesAuthDomainConfigSAMLDTOKind,
AuthtypesAuthNProviderDTO,
AuthtypesGettableAuthDomainDTO,
AuthtypesGoogleConfigDTO,
@@ -41,7 +45,7 @@ function configureAuthnProvider(
switch (authnProvider) {
case 'saml':
return <ConfigureSAMLAuthnProvider isCreate={isCreate} />;
case 'google_auth':
case 'google':
return <ConfigureGoogleAuthAuthnProvider isCreate={isCreate} />;
case 'oidc':
return <ConfigureOIDCAuthnProvider isCreate={isCreate} />;
@@ -61,7 +65,7 @@ function CreateOrEdit(props: CreateOrEditProps): JSX.Element {
const [form] = Form.useForm<FormValues>();
const [authnProvider, setAuthnProvider] = useState<
AuthtypesAuthNProviderDTO | ''
>(record?.config?.ssoType || '');
>((record?.config?.kind as unknown as AuthtypesAuthNProviderDTO) ?? '');
const { showErrorModal } = useErrorModal();
const { featureFlags } = useAppContext();
@@ -147,6 +151,33 @@ function CreateOrEdit(props: CreateOrEditProps): JSX.Element {
};
}, [form]);
// Prepares the kind/spec config envelope for API payload
const getConfig = useCallback((): AuthtypesAuthDomainConfigDTO | undefined => {
switch (authnProvider) {
case AuthtypesAuthNProviderDTO.saml:
return {
kind: AuthtypesAuthDomainConfigSAMLDTOKind.saml,
spec: form.getFieldValue('samlConfig'),
};
case AuthtypesAuthNProviderDTO.google: {
const spec = getGoogleAuthConfig();
return spec
? {
kind: AuthtypesAuthDomainConfigGoogleDTOKind.google,
spec,
}
: undefined;
}
case AuthtypesAuthNProviderDTO.oidc:
return {
kind: AuthtypesAuthDomainConfigOIDCDTOKind.oidc,
spec: form.getFieldValue('oidcConfig'),
};
default:
return undefined;
}
}, [authnProvider, form, getGoogleAuthConfig]);
const onSubmitHandler = useCallback(async (): Promise<void> => {
try {
await form.validateFields();
@@ -159,24 +190,21 @@ function CreateOrEdit(props: CreateOrEditProps): JSX.Element {
}
const name = form.getFieldValue('name');
const googleAuthConfig = getGoogleAuthConfig();
const samlConfig = form.getFieldValue('samlConfig');
const oidcConfig = form.getFieldValue('oidcConfig');
const config = getConfig();
const roleMapping = getRoleMapping();
if (!config) {
return;
}
if (isCreate) {
createAuthDomain(
{
data: {
name,
config: {
ssoEnabled: true,
ssoType: authnProvider,
googleAuthConfig,
samlConfig,
oidcConfig,
roleMapping,
},
enabled: true,
config,
roleMapping,
},
},
{
@@ -196,14 +224,9 @@ function CreateOrEdit(props: CreateOrEditProps): JSX.Element {
{
pathParams: { id: record.id },
data: {
config: {
ssoEnabled: form.getFieldValue('ssoEnabled'),
ssoType: authnProvider,
googleAuthConfig,
samlConfig,
oidcConfig,
roleMapping,
},
enabled: form.getFieldValue('enabled'),
config,
roleMapping,
},
},
{
@@ -219,7 +242,7 @@ function CreateOrEdit(props: CreateOrEditProps): JSX.Element {
authnProvider,
createAuthDomain,
form,
getGoogleAuthConfig,
getConfig,
getRoleMapping,
handleError,
isCreate,
@@ -245,8 +268,7 @@ function CreateOrEdit(props: CreateOrEditProps): JSX.Element {
name="auth-domain"
initialValues={defaultTo(prepareInitialValues(record), {
name: '',
ssoEnabled: false,
ssoType: '',
enabled: false,
})}
form={form}
layout="vertical"

View File

@@ -1,4 +1,8 @@
import { AuthtypesAuthNProviderDTO } from 'api/generated/services/sigNoz.schemas';
import {
AuthtypesAuthDomainConfigGoogleDTOKind,
AuthtypesAuthDomainConfigOIDCDTOKind,
AuthtypesAuthDomainConfigSAMLDTOKind,
} from 'api/generated/services/sigNoz.schemas';
import {
convertDomainMappingsToList,
@@ -82,8 +86,7 @@ describe('prepareInitialValues', () => {
it('returns empty defaults when no record is provided', () => {
expect(prepareInitialValues(undefined)).toStrictEqual({
name: '',
ssoEnabled: false,
ssoType: '',
enabled: false,
});
});
@@ -91,15 +94,20 @@ describe('prepareInitialValues', () => {
const result = prepareInitialValues({
id: 'domain-1',
name: 'example.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.saml,
roleMapping: {
defaultRole: 'VIEWER',
useRoleAttribute: false,
groupMappings: { admins: 'ADMIN', viewers: 'VIEWER' },
kind: AuthtypesAuthDomainConfigSAMLDTOKind.saml,
spec: {
location: 'https://idp.example.com/sso',
entityId: 'urn:example:idp',
certificate: 'CERT',
},
},
roleMapping: {
defaultRole: 'VIEWER',
useRoleAttribute: false,
groupMappings: { admins: 'ADMIN', viewers: 'VIEWER' },
},
});
expect(result.roleMapping?.groupMappingsList).toStrictEqual([
@@ -112,10 +120,10 @@ describe('prepareInitialValues', () => {
const result = prepareInitialValues({
id: 'domain-1',
name: 'example.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.google_auth,
googleAuthConfig: {
kind: AuthtypesAuthDomainConfigGoogleDTOKind.google,
spec: {
clientId: 'id',
clientSecret: 'secret',
domainToAdminEmail: { 'example.com': 'admin@example.com' },
@@ -132,11 +140,16 @@ describe('prepareInitialValues', () => {
const result = prepareInitialValues({
id: 'domain-1',
name: 'example.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.oidc,
roleMapping: { defaultRole: 'VIEWER', useRoleAttribute: true },
kind: AuthtypesAuthDomainConfigOIDCDTOKind.oidc,
spec: {
issuer: 'https://oidc.example.com',
clientId: 'id',
clientSecret: 'secret',
},
},
roleMapping: { defaultRole: 'VIEWER', useRoleAttribute: true },
});
expect(result.roleMapping?.groupMappingsList).toStrictEqual([]);

View File

@@ -1,4 +1,7 @@
import {
AuthtypesAuthDomainConfigGoogleDTOKind,
AuthtypesAuthDomainConfigOIDCDTOKind,
AuthtypesAuthDomainConfigSAMLDTOKind,
AuthtypesGettableAuthDomainDTO,
AuthtypesGoogleConfigDTO,
AuthtypesOIDCConfigDTO,
@@ -9,8 +12,7 @@ import {
// Form values interface for internal use (includes array-based fields for UI)
export interface FormValues {
name?: string;
ssoEnabled?: boolean;
ssoType?: string;
enabled?: boolean;
googleAuthConfig?: AuthtypesGoogleConfigDTO & {
domainToAdminEmailList?: Array<{ domain?: string; adminEmail?: string }>;
};
@@ -107,31 +109,36 @@ export function prepareInitialValues(
if (!record) {
return {
name: '',
ssoEnabled: false,
ssoType: '',
enabled: false,
};
}
const config = record.config ?? {};
const { config } = record;
return {
name: record.name,
ssoEnabled: config.ssoEnabled,
ssoType: config.ssoType,
samlConfig: config.samlConfig ?? undefined,
oidcConfig: config.oidcConfig ?? undefined,
googleAuthConfig: config.googleAuthConfig
enabled: record.enabled,
samlConfig:
config?.kind === AuthtypesAuthDomainConfigSAMLDTOKind.saml
? config.spec
: undefined,
oidcConfig:
config?.kind === AuthtypesAuthDomainConfigOIDCDTOKind.oidc
? config.spec
: undefined,
googleAuthConfig:
config?.kind === AuthtypesAuthDomainConfigGoogleDTOKind.google
? {
...config.spec,
domainToAdminEmailList: convertDomainMappingsToList(
config.spec.domainToAdminEmail,
),
}
: undefined,
roleMapping: record.roleMapping
? {
...config.googleAuthConfig,
domainToAdminEmailList: convertDomainMappingsToList(
config.googleAuthConfig.domainToAdminEmail,
),
}
: undefined,
roleMapping: config.roleMapping
? {
...config.roleMapping,
...record.roleMapping,
groupMappingsList: convertGroupMappingsToList(
config.roleMapping.groupMappings,
record.roleMapping.groupMappings,
),
}
: undefined,

View File

@@ -76,7 +76,7 @@ function ConfigureSAMLAuthnProvider({
</Tooltip>
</label>
<Form.Item
name={['samlConfig', 'samlIdp']}
name={['samlConfig', 'location']}
className="authn-provider__form-item"
rules={[
{
@@ -98,7 +98,7 @@ function ConfigureSAMLAuthnProvider({
</Tooltip>
</label>
<Form.Item
name={['samlConfig', 'samlEntity']}
name={['samlConfig', 'entityId']}
className="authn-provider__form-item"
rules={[
{
@@ -120,7 +120,7 @@ function ConfigureSAMLAuthnProvider({
</Tooltip>
</label>
<Form.Item
name={['samlConfig', 'samlCert']}
name={['samlConfig', 'certificate']}
className="authn-provider__form-item"
rules={[
{

View File

@@ -31,7 +31,7 @@ function SSOEnforcementToggle({
useUpdateAuthDomain<AxiosError<RenderErrorResponseDTO>>();
const onChangeHandler = (checked: boolean): void => {
if (!record.id) {
if (!record.id || !record.config) {
return;
}
@@ -41,14 +41,9 @@ function SSOEnforcementToggle({
{
pathParams: { id: record.id },
data: {
config: {
ssoEnabled: checked,
ssoType: record.config?.ssoType,
googleAuthConfig: record.config?.googleAuthConfig,
oidcConfig: record.config?.oidcConfig,
samlConfig: record.config?.samlConfig,
roleMapping: record.config?.roleMapping,
},
enabled: checked,
config: record.config,
roleMapping: record.roleMapping,
},
},
{

View File

@@ -58,7 +58,7 @@ describe('AuthDomain', () => {
});
});
it('reflects ssoEnabled state from nested config in each row toggle', async () => {
it('reflects the enabled state in each row toggle', async () => {
server.use(
rest.get(AUTH_DOMAINS_LIST_ENDPOINT, (_, res, ctx) =>
res(ctx.status(200), ctx.json(mockDomainsListResponse)),
@@ -68,9 +68,9 @@ describe('AuthDomain', () => {
render(<AuthDomain />);
// mockDomainsListResponse rows:
// [0] signoz.io → config.ssoEnabled: true
// [1] example.com → config.ssoEnabled: false
// [2] corp.io → config.ssoEnabled: true
// [0] signoz.io → enabled: true
// [1] example.com → enabled: false
// [2] corp.io → enabled: true
const switches = await screen.findAllByRole('switch');
expect(switches).toHaveLength(3);
expect(switches[0]).toBeChecked();

View File

@@ -112,9 +112,7 @@ describe('CreateEdit — save payload correctness', () => {
await waitFor(() => expect(capturedPayload).not.toBeNull());
expect(capturedPayload).toMatchObject({
config: expect.objectContaining({
roleMapping: expect.objectContaining({ groupMappings: {} }),
}),
roleMapping: expect.objectContaining({ groupMappings: {} }),
});
});
@@ -161,7 +159,7 @@ describe('CreateEdit — save payload correctness', () => {
expect(capturedPayload).toMatchObject({
config: expect.objectContaining({
googleAuthConfig: expect.objectContaining({
spec: expect.objectContaining({
domainToAdminEmail: {},
}),
}),

View File

@@ -188,8 +188,8 @@ describe('CreateEdit — role mapping uses API roles', () => {
// SSO role mapping matches roles by name, so the payload carries the
// role *name*, not the opaque id.
expect(payload.get().config.roleMapping.defaultRole).toBe(editorRole.name);
expect(payload.get().config.roleMapping.defaultRole).not.toBe(editorRole.id);
expect(payload.get().roleMapping.defaultRole).toBe(editorRole.name);
expect(payload.get().roleMapping.defaultRole).not.toBe(editorRole.id);
});
it('defaults a fresh role mapping to the signoz-viewer role name', async () => {
@@ -221,8 +221,8 @@ describe('CreateEdit — role mapping uses API roles', () => {
await waitFor(() => expect(payload.get()).not.toBeNull());
expect(payload.get().config.roleMapping.defaultRole).toBe(viewerRole.name);
expect(payload.get().config.roleMapping.defaultRole).not.toBe(viewerRole.id);
expect(payload.get().roleMapping.defaultRole).toBe(viewerRole.name);
expect(payload.get().roleMapping.defaultRole).not.toBe(viewerRole.id);
});
it('still defaults to signoz-viewer when the roles fetch returns empty', async () => {
@@ -249,7 +249,7 @@ describe('CreateEdit — role mapping uses API roles', () => {
await waitFor(() => expect(payload.get()).not.toBeNull());
// The Form.Item initialValue (signoz-viewer) survives an empty roles list.
expect(payload.get().config.roleMapping.defaultRole).toBe(viewerRole.name);
expect(payload.get().roleMapping.defaultRole).toBe(viewerRole.name);
});
it('loads a stored role mapping by role name and round-trips it on save', async () => {
@@ -280,8 +280,8 @@ describe('CreateEdit — role mapping uses API roles', () => {
await waitFor(() => expect(payload.get()).not.toBeNull());
expect(payload.get().config.roleMapping.defaultRole).toBe(editorRole.name);
expect(payload.get().config.roleMapping.groupMappings).toStrictEqual({
expect(payload.get().roleMapping.defaultRole).toBe(editorRole.name);
expect(payload.get().roleMapping.groupMappings).toStrictEqual({
'admin-group': 'signoz-admin',
'dev-team': 'signoz-editor',
viewers: 'signoz-viewer',

View File

@@ -1,6 +1,9 @@
import { fireEvent, render, screen, waitFor } from 'tests/test-utils';
import { rest, server } from 'mocks-server/server';
import { AuthtypesGettableAuthDomainDTO } from 'api/generated/services/sigNoz.schemas';
import {
AuthtypesAuthDomainConfigGoogleDTO,
AuthtypesGettableAuthDomainDTO,
} from 'api/generated/services/sigNoz.schemas';
import CreateEdit from '../CreateEdit/CreateEdit';
import {
@@ -48,11 +51,10 @@ jest.mock('@signozhq/ui/button', () => ({
type SavedPayload = {
config: {
googleAuthConfig?: Record<string, unknown>;
samlConfig?: Record<string, unknown>;
oidcConfig?: Record<string, unknown>;
roleMapping?: Record<string, unknown>;
kind?: string;
spec?: Record<string, unknown>;
};
roleMapping?: Record<string, unknown>;
};
async function submitForm(
@@ -81,7 +83,7 @@ describe('CreateEdit — payload sanitization', () => {
it('sends core fields and omits workspace fields when fetchGroups is not set', async () => {
const payload = await submitForm(mockGoogleAuthDomain);
const g = payload.config.googleAuthConfig;
const g = payload.config.spec;
expect(g?.clientId).toBe('test-client-id');
expect(g?.clientSecret).toBe('test-client-secret');
expect(g?.allowedGroups).toBeUndefined();
@@ -91,18 +93,20 @@ describe('CreateEdit — payload sanitization', () => {
});
it('strips workspace fields when fetchGroups is false', async () => {
const googleConfig =
mockGoogleAuthWithWorkspaceGroups.config as AuthtypesAuthDomainConfigGoogleDTO;
const payload = await submitForm({
...mockGoogleAuthWithWorkspaceGroups,
config: {
...mockGoogleAuthWithWorkspaceGroups.config,
googleAuthConfig: {
...mockGoogleAuthWithWorkspaceGroups.config?.googleAuthConfig,
...googleConfig,
spec: {
...googleConfig.spec,
fetchGroups: false,
},
},
});
const g = payload.config.googleAuthConfig;
const g = payload.config.spec;
expect(g?.fetchGroups).toBe(false);
expect(g?.allowedGroups).toBeUndefined();
expect(g?.serviceAccountJson).toBeUndefined();
@@ -113,7 +117,7 @@ describe('CreateEdit — payload sanitization', () => {
it('includes all workspace fields when fetchGroups is true', async () => {
const payload = await submitForm(mockGoogleAuthWithWorkspaceGroups);
const g = payload.config.googleAuthConfig;
const g = payload.config.spec;
expect(g?.fetchGroups).toBe(true);
expect(g?.serviceAccountJson).toBe('{"type": "service_account"}');
expect(g?.fetchTransitiveGroupMembership).toBe(true);
@@ -131,10 +135,10 @@ describe('CreateEdit — payload sanitization', () => {
it('sends core and attributeMapping fields', async () => {
const payload = await submitForm(mockSamlWithAttributeMapping);
const s = payload.config.samlConfig;
expect(s?.samlIdp).toBe('https://idp.saml-attrs.com/sso');
expect(s?.samlEntity).toBe('urn:saml-attrs:idp');
expect(s?.samlCert).toBe('MOCK_CERTIFICATE_ATTRS');
const s = payload.config.spec;
expect(s?.location).toBe('https://idp.saml-attrs.com/sso');
expect(s?.entityId).toBe('urn:saml-attrs:idp');
expect(s?.certificate).toBe('MOCK_CERTIFICATE_ATTRS');
expect(s?.insecureSkipAuthNRequestsSigned).toBe(true);
const attr = s?.attributeMapping as Record<string, unknown>;
@@ -148,7 +152,7 @@ describe('CreateEdit — payload sanitization', () => {
it('sends all fields including claimMapping', async () => {
const payload = await submitForm(mockOidcWithClaimMapping);
const o = payload.config.oidcConfig;
const o = payload.config.spec;
expect(o?.issuer).toBe('https://oidc.claims.com');
expect(o?.issuerAlias).toBe('https://alias.claims.com');
expect(o?.clientId).toBe('claims-client-id');
@@ -168,24 +172,21 @@ describe('CreateEdit — payload sanitization', () => {
it('strips groupMappings when useRoleAttribute is true', async () => {
const payload = await submitForm({
...mockDomainWithRoleMapping,
config: {
...mockDomainWithRoleMapping.config,
roleMapping: {
...mockDomainWithRoleMapping.config?.roleMapping,
useRoleAttribute: true,
},
roleMapping: {
...mockDomainWithRoleMapping.roleMapping,
useRoleAttribute: true,
},
});
expect(payload.config.roleMapping?.useRoleAttribute).toBe(true);
expect(payload.config.roleMapping?.groupMappings).toBeUndefined();
expect(payload.roleMapping?.useRoleAttribute).toBe(true);
expect(payload.roleMapping?.groupMappings).toBeUndefined();
});
it('sends groupMappings when useRoleAttribute is false', async () => {
const payload = await submitForm(mockDomainWithRoleMapping);
expect(payload.config.roleMapping?.useRoleAttribute).toBe(false);
expect(payload.config.roleMapping?.groupMappings).toStrictEqual({
expect(payload.roleMapping?.useRoleAttribute).toBe(false);
expect(payload.roleMapping?.groupMappings).toStrictEqual({
'admin-group': 'signoz-admin',
'dev-team': 'signoz-editor',
viewers: 'signoz-viewer',

View File

@@ -57,7 +57,7 @@ describe('SSOEnforcementToggle', () => {
isDefaultChecked={false}
record={{
...mockGoogleAuthDomain,
config: { ...mockGoogleAuthDomain.config, ssoEnabled: false },
enabled: false,
}}
/>,
);
@@ -95,9 +95,7 @@ describe('SSOEnforcementToggle', () => {
expect(mockUpdateAPI).toHaveBeenCalledTimes(1);
expect(mockUpdateAPI).toHaveBeenCalledWith(
expect.objectContaining({
config: expect.objectContaining({
ssoEnabled: false,
}),
enabled: false,
}),
);
});

View File

@@ -1,22 +1,24 @@
import {
AuthtypesAuthNProviderDTO,
AuthtypesAuthDomainConfigGoogleDTOKind,
AuthtypesAuthDomainConfigOIDCDTOKind,
AuthtypesAuthDomainConfigSAMLDTOKind,
AuthtypesGettableAuthDomainDTO,
} from 'api/generated/services/sigNoz.schemas';
// API Endpoints
export const AUTH_DOMAINS_LIST_ENDPOINT = '*/api/v1/domains';
export const AUTH_DOMAINS_CREATE_ENDPOINT = '*/api/v1/domains';
export const AUTH_DOMAINS_UPDATE_ENDPOINT = '*/api/v1/domains/:id';
export const AUTH_DOMAINS_DELETE_ENDPOINT = '*/api/v1/domains/:id';
export const AUTH_DOMAINS_LIST_ENDPOINT = '*/api/v2/auth_domains';
export const AUTH_DOMAINS_CREATE_ENDPOINT = '*/api/v2/auth_domains';
export const AUTH_DOMAINS_UPDATE_ENDPOINT = '*/api/v2/auth_domains/:id';
export const AUTH_DOMAINS_DELETE_ENDPOINT = '*/api/v2/auth_domains/:id';
// Mock Auth Domain with Google Auth
export const mockGoogleAuthDomain: AuthtypesGettableAuthDomainDTO = {
id: 'domain-1',
name: 'signoz.io',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.google_auth,
googleAuthConfig: {
kind: AuthtypesAuthDomainConfigGoogleDTOKind.google,
spec: {
clientId: 'test-client-id',
clientSecret: 'test-client-secret',
},
@@ -30,13 +32,13 @@ export const mockGoogleAuthDomain: AuthtypesGettableAuthDomainDTO = {
export const mockSamlAuthDomain: AuthtypesGettableAuthDomainDTO = {
id: 'domain-2',
name: 'example.com',
enabled: false,
config: {
ssoEnabled: false,
ssoType: AuthtypesAuthNProviderDTO.saml,
samlConfig: {
samlIdp: 'https://idp.example.com/sso',
samlEntity: 'urn:example:idp',
samlCert: 'MOCK_CERTIFICATE',
kind: AuthtypesAuthDomainConfigSAMLDTOKind.saml,
spec: {
location: 'https://idp.example.com/sso',
entityId: 'urn:example:idp',
certificate: 'MOCK_CERTIFICATE',
},
},
authNProviderInfo: {
@@ -48,10 +50,10 @@ export const mockSamlAuthDomain: AuthtypesGettableAuthDomainDTO = {
export const mockOidcAuthDomain: AuthtypesGettableAuthDomainDTO = {
id: 'domain-3',
name: 'corp.io',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.oidc,
oidcConfig: {
kind: AuthtypesAuthDomainConfigOIDCDTOKind.oidc,
spec: {
issuer: 'https://oidc.corp.io',
clientId: 'oidc-client-id',
clientSecret: 'oidc-client-secret',
@@ -66,22 +68,22 @@ export const mockOidcAuthDomain: AuthtypesGettableAuthDomainDTO = {
export const mockDomainWithRoleMapping: AuthtypesGettableAuthDomainDTO = {
id: 'domain-4',
name: 'enterprise.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.saml,
samlConfig: {
samlIdp: 'https://idp.enterprise.com/sso',
samlEntity: 'urn:enterprise:idp',
samlCert: 'MOCK_CERTIFICATE',
kind: AuthtypesAuthDomainConfigSAMLDTOKind.saml,
spec: {
location: 'https://idp.enterprise.com/sso',
entityId: 'urn:enterprise:idp',
certificate: 'MOCK_CERTIFICATE',
},
roleMapping: {
defaultRole: 'signoz-editor',
useRoleAttribute: false,
groupMappings: {
'admin-group': 'signoz-admin',
'dev-team': 'signoz-editor',
viewers: 'signoz-viewer',
},
},
roleMapping: {
defaultRole: 'signoz-editor',
useRoleAttribute: false,
groupMappings: {
'admin-group': 'signoz-admin',
'dev-team': 'signoz-editor',
viewers: 'signoz-viewer',
},
},
authNProviderInfo: {
@@ -94,18 +96,18 @@ export const mockDomainWithDirectRoleAttribute: AuthtypesGettableAuthDomainDTO =
{
id: 'domain-5',
name: 'direct-role.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.oidc,
oidcConfig: {
kind: AuthtypesAuthDomainConfigOIDCDTOKind.oidc,
spec: {
issuer: 'https://oidc.direct-role.com',
clientId: 'direct-role-client-id',
clientSecret: 'direct-role-client-secret',
},
roleMapping: {
defaultRole: 'signoz-viewer',
useRoleAttribute: true,
},
},
roleMapping: {
defaultRole: 'signoz-viewer',
useRoleAttribute: true,
},
authNProviderInfo: {
relayStatePath: 'api/v1/sso/relay/domain-5',
@@ -116,10 +118,10 @@ export const mockDomainWithDirectRoleAttribute: AuthtypesGettableAuthDomainDTO =
export const mockOidcWithClaimMapping: AuthtypesGettableAuthDomainDTO = {
id: 'domain-6',
name: 'oidc-claims.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.oidc,
oidcConfig: {
kind: AuthtypesAuthDomainConfigOIDCDTOKind.oidc,
spec: {
issuer: 'https://oidc.claims.com',
issuerAlias: 'https://alias.claims.com',
clientId: 'claims-client-id',
@@ -143,13 +145,13 @@ export const mockOidcWithClaimMapping: AuthtypesGettableAuthDomainDTO = {
export const mockSamlWithAttributeMapping: AuthtypesGettableAuthDomainDTO = {
id: 'domain-7',
name: 'saml-attrs.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.saml,
samlConfig: {
samlIdp: 'https://idp.saml-attrs.com/sso',
samlEntity: 'urn:saml-attrs:idp',
samlCert: 'MOCK_CERTIFICATE_ATTRS',
kind: AuthtypesAuthDomainConfigSAMLDTOKind.saml,
spec: {
location: 'https://idp.saml-attrs.com/sso',
entityId: 'urn:saml-attrs:idp',
certificate: 'MOCK_CERTIFICATE_ATTRS',
insecureSkipAuthNRequestsSigned: true,
attributeMapping: {
name: 'user_display_name',
@@ -168,10 +170,10 @@ export const mockGoogleAuthWithWorkspaceGroups: AuthtypesGettableAuthDomainDTO =
{
id: 'domain-8',
name: 'google-groups.com',
enabled: true,
config: {
ssoEnabled: true,
ssoType: AuthtypesAuthNProviderDTO.google_auth,
googleAuthConfig: {
kind: AuthtypesAuthDomainConfigGoogleDTOKind.google,
spec: {
clientId: 'google-groups-client-id',
clientSecret: 'google-groups-client-secret',
insecureSkipEmailVerified: false,
@@ -218,7 +220,7 @@ export const mockUpdateSuccessResponse = {
status: 'success',
data: {
...mockGoogleAuthDomain,
config: { ...mockGoogleAuthDomain.config, ssoEnabled: false },
enabled: false,
},
};

View File

@@ -26,7 +26,7 @@ import './AuthDomain.styles.scss';
import '../../IngestionSettings/IngestionSettings.styles.scss';
export const SSOType = new Map<string, string>([
['google_auth', 'Google Auth'],
['google', 'Google Auth'],
['saml', 'SAML'],
['email_password', 'Email Password'],
['oidc', 'OIDC'],
@@ -121,8 +121,8 @@ function AuthDomain(): JSX.Element {
},
{
title: 'Enforce SSO',
dataIndex: ['config', 'ssoEnabled'],
key: 'ssoEnabled',
dataIndex: 'enabled',
key: 'enabled',
width: 80,
render: (
value: boolean,
@@ -158,7 +158,7 @@ function AuthDomain(): JSX.Element {
onClick={(): void => setRecord(record)}
variant="link"
>
Configure {SSOType.get(record.config?.ssoType || '')}
Configure {SSOType.get(record.config?.kind || '')}
</Button>
<Button
className="auth-domain-list-action-link delete"

View File

@@ -10,7 +10,7 @@ import (
)
func (provider *provider) addAuthDomainRoutes(router *mux.Router) error {
if err := router.Handle("/api/v1/domains", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.List), handler.OpenAPIDef{
if err := router.Handle("/api/v2/auth_domains", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.List), handler.OpenAPIDef{
ID: "ListAuthDomains",
Tags: []string{"authdomains"},
Summary: "List all auth domains",
@@ -27,7 +27,7 @@ func (provider *provider) addAuthDomainRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/domains", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Create), handler.OpenAPIDef{
if err := router.Handle("/api/v2/auth_domains", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Create), handler.OpenAPIDef{
ID: "CreateAuthDomain",
Tags: []string{"authdomains"},
Summary: "Create auth domain",
@@ -44,7 +44,7 @@ func (provider *provider) addAuthDomainRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/domains/{id}", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Get), handler.OpenAPIDef{
if err := router.Handle("/api/v2/auth_domains/{id}", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Get), handler.OpenAPIDef{
ID: "GetAuthDomain",
Tags: []string{"authdomains"},
Summary: "Get auth domain by ID",
@@ -61,7 +61,7 @@ func (provider *provider) addAuthDomainRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/domains/{id}", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Update), handler.OpenAPIDef{
if err := router.Handle("/api/v2/auth_domains/{id}", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Update), handler.OpenAPIDef{
ID: "UpdateAuthDomain",
Tags: []string{"authdomains"},
Summary: "Update auth domain",
@@ -78,7 +78,7 @@ func (provider *provider) addAuthDomainRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/domains/{id}", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Delete), handler.OpenAPIDef{
if err := router.Handle("/api/v2/auth_domains/{id}", handler.New(provider.authzMiddleware.AdminAccess(provider.authDomainHandler.Delete), handler.OpenAPIDef{
ID: "DeleteAuthDomain",
Tags: []string{"authdomains"},
Summary: "Delete auth domain",

View File

@@ -59,7 +59,7 @@ func (a *AuthN) LoginURL(ctx context.Context, siteURL *url.URL, authDomain *auth
return "", err
}
if authDomain.AuthDomainConfig().AuthNProvider != authtypes.AuthNProviderGoogleAuth {
if authDomain.StorableAuthDomainConfig().AuthNProvider != authtypes.AuthNProviderGoogle {
return "", errors.Newf(errors.TypeInternal, authtypes.ErrCodeAuthDomainMismatch, "domain type is not google")
}
@@ -111,7 +111,7 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
return nil, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "google: no id_token in token response")
}
verifier := oidcProvider.Verifier(&oidc.Config{ClientID: authDomain.AuthDomainConfig().Google.ClientID})
verifier := oidcProvider.Verifier(&oidc.Config{ClientID: authDomain.StorableAuthDomainConfig().Google.ClientID})
idToken, err := verifier.Verify(ctx, rawIDToken)
if err != nil {
a.settings.Logger().ErrorContext(ctx, "google: failed to verify token", errors.Attr(err))
@@ -135,7 +135,7 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
return nil, errors.Newf(errors.TypeForbidden, errors.CodeForbidden, "google: unexpected hd claim")
}
if !authDomain.AuthDomainConfig().Google.InsecureSkipEmailVerified {
if !authDomain.StorableAuthDomainConfig().Google.InsecureSkipEmailVerified {
if !claims.EmailVerified {
a.settings.Logger().ErrorContext(ctx, "google: email is not verified", slog.String("email", claims.Email))
return nil, errors.Newf(errors.TypeForbidden, errors.CodeForbidden, "google: email is not verified")
@@ -148,14 +148,14 @@ func (a *AuthN) HandleCallback(ctx context.Context, query url.Values) (*authtype
}
var groups []string
if authDomain.AuthDomainConfig().Google.FetchGroups {
groups, err = a.fetchGoogleWorkspaceGroups(ctx, claims.Email, authDomain.AuthDomainConfig().Google)
if authDomain.StorableAuthDomainConfig().Google.FetchGroups {
groups, err = a.fetchGoogleWorkspaceGroups(ctx, claims.Email, authDomain.StorableAuthDomainConfig().Google)
if err != nil {
a.settings.Logger().ErrorContext(ctx, "google: could not fetch groups", errors.Attr(err))
return nil, errors.Newf(errors.TypeInternal, errors.CodeInternal, "google: could not fetch groups").WithAdditional(err.Error())
}
allowedGroups := authDomain.AuthDomainConfig().Google.AllowedGroups
allowedGroups := authDomain.StorableAuthDomainConfig().Google.AllowedGroups
if len(allowedGroups) > 0 {
groups = filterGroups(groups, allowedGroups)
if len(groups) == 0 {
@@ -175,8 +175,8 @@ func (a *AuthN) ProviderInfo(ctx context.Context, authDomain *authtypes.AuthDoma
func (a *AuthN) oauth2Config(siteURL *url.URL, authDomain *authtypes.AuthDomain, provider *oidc.Provider) *oauth2.Config {
return &oauth2.Config{
ClientID: authDomain.AuthDomainConfig().Google.ClientID,
ClientSecret: authDomain.AuthDomainConfig().Google.ClientSecret,
ClientID: authDomain.StorableAuthDomainConfig().Google.ClientID,
ClientSecret: authDomain.StorableAuthDomainConfig().Google.ClientSecret,
Endpoint: provider.Endpoint(),
Scopes: scopes,
RedirectURL: (&url.URL{

View File

@@ -26,7 +26,7 @@ func (getter *getter) OnBeforeRoleDelete(ctx context.Context, orgID valuer.UUID,
referencedBy := make([]string, 0)
for _, domain := range domains {
for _, mappedRole := range domain.AuthDomainConfig().RoleMapping.RoleNames() {
for _, mappedRole := range domain.StorableAuthDomainConfig().RoleMapping.RoleNames() {
if mappedRole == roleName {
referencedBy = append(referencedBy, domain.StorableAuthDomain().Name)
break

View File

@@ -38,7 +38,7 @@ func (handler *handler) Create(rw http.ResponseWriter, req *http.Request) {
return
}
authDomain, err := authtypes.NewAuthDomainFromConfig(body.Name, &body.Config, valuer.MustNewUUID(claims.OrgID))
authDomain, err := authtypes.NewAuthDomainFromPostableAuthDomain(body, valuer.MustNewUUID(claims.OrgID))
if err != nil {
render.Error(rw, err)
return
@@ -99,7 +99,13 @@ func (handler *handler) Get(rw http.ResponseWriter, req *http.Request) {
return
}
render.Success(rw, http.StatusOK, authtypes.NewGettableAuthDomainFromAuthDomain(authDomain, handler.module.GetAuthNProviderInfo(ctx, authDomain)))
gettableAuthDomain, err := authtypes.NewGettableAuthDomainFromAuthDomain(authDomain, handler.module.GetAuthNProviderInfo(ctx, authDomain))
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, gettableAuthDomain)
}
func (handler *handler) List(rw http.ResponseWriter, r *http.Request) {
@@ -120,7 +126,13 @@ func (handler *handler) List(rw http.ResponseWriter, r *http.Request) {
authDomains := make([]*authtypes.GettableAuthDomain, len(domains))
for i, domain := range domains {
authDomains[i] = authtypes.NewGettableAuthDomainFromAuthDomain(domain, handler.module.GetAuthNProviderInfo(ctx, domain))
gettableAuthDomain, err := authtypes.NewGettableAuthDomainFromAuthDomain(domain, handler.module.GetAuthNProviderInfo(ctx, domain))
if err != nil {
render.Error(rw, err)
return
}
authDomains[i] = gettableAuthDomain
}
render.Success(rw, http.StatusOK, authDomains)
@@ -154,7 +166,7 @@ func (handler *handler) Update(rw http.ResponseWriter, r *http.Request) {
return
}
err = authDomain.Update(&body.Config)
err = authDomain.Update(body)
if err != nil {
render.Error(rw, err)
return

View File

@@ -33,7 +33,7 @@ func (module *module) Get(ctx context.Context, id valuer.UUID) (*authtypes.AuthD
}
func (module *module) GetAuthNProviderInfo(ctx context.Context, domain *authtypes.AuthDomain) *authtypes.AuthNProviderInfo {
if callbackAuthN, ok := module.authNs[domain.AuthDomainConfig().AuthNProvider].(authn.CallbackAuthN); ok {
if callbackAuthN, ok := module.authNs[domain.StorableAuthDomainConfig().AuthNProvider].(authn.CallbackAuthN); ok {
return callbackAuthN.ProviderInfo(ctx, domain)
}
return &authtypes.AuthNProviderInfo{}
@@ -72,7 +72,7 @@ func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[strin
stats := make(map[string]any)
for _, domain := range domains {
key := "authdomain." + domain.AuthDomainConfig().AuthNProvider.StringValue() + ".count"
key := "authdomain." + domain.StorableAuthDomainConfig().AuthNProvider.StringValue() + ".count"
if value, ok := stats[key]; ok {
stats[key] = value.(int64) + 1
} else {
@@ -86,7 +86,7 @@ func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[strin
}
func (module *module) validateRoleMapping(ctx context.Context, domain *authtypes.AuthDomain) error {
roleNames := domain.AuthDomainConfig().RoleMapping.RoleNames()
roleNames := domain.StorableAuthDomainConfig().RoleMapping.RoleNames()
if len(roleNames) == 0 {
return nil
}

View File

@@ -75,7 +75,7 @@ func (handler *handler) CreateSessionByGoogleCallback(rw http.ResponseWriter, re
values := req.URL.Query()
redirectURL, err := handler.module.CreateCallbackAuthNSession(ctx, authtypes.AuthNProviderGoogleAuth, values)
redirectURL, err := handler.module.CreateCallbackAuthNSession(ctx, authtypes.AuthNProviderGoogle, values)
if err != nil {
http.Redirect(rw, req, handler.getRedirectURLFromErr(err), http.StatusSeeOther)
return

View File

@@ -152,7 +152,7 @@ func (module *module) CreateCallbackAuthNSession(ctx context.Context, authNProvi
return "", err
}
roleMapping := authDomain.AuthDomainConfig().RoleMapping
roleMapping := authDomain.StorableAuthDomainConfig().RoleMapping
roleAttributeExists := false
if roleMapping != nil && roleMapping.UseRoleAttribute && callbackIdentity.Role != "" {
@@ -215,11 +215,11 @@ func (module *module) getOrgSessionContext(ctx context.Context, org *types.Organ
return authtypes.NewOrgSessionContext(org.ID, org.Name).AddPasswordAuthNSupport(authtypes.AuthNProviderEmailPassword), nil
}
if !authDomain.AuthDomainConfig().SSOEnabled {
if !authDomain.StorableAuthDomainConfig().SSOEnabled {
return authtypes.NewOrgSessionContext(org.ID, org.Name).AddPasswordAuthNSupport(authtypes.AuthNProviderEmailPassword), nil
}
provider, err := getProvider[authn.CallbackAuthN](authDomain.AuthDomainConfig().AuthNProvider, module.authNs)
provider, err := getProvider[authn.CallbackAuthN](authDomain.StorableAuthDomainConfig().AuthNProvider, module.authNs)
if err != nil {
return nil, err
}
@@ -233,7 +233,7 @@ func (module *module) getOrgSessionContext(ctx context.Context, org *types.Organ
return nil, err
}
return authtypes.NewOrgSessionContext(org.ID, org.Name).AddCallbackAuthNSupport(authDomain.AuthDomainConfig().AuthNProvider, loginURL), nil
return authtypes.NewOrgSessionContext(org.ID, org.Name).AddCallbackAuthNSupport(authDomain.StorableAuthDomainConfig().AuthNProvider, loginURL), nil
}
func getProvider[T authn.AuthN](authNProvider authtypes.AuthNProvider, authNs map[authtypes.AuthNProvider]authn.AuthN) (T, error) {

View File

@@ -1,9 +0,0 @@
package prometheus
import "net/http"
type Handler interface {
Query(http.ResponseWriter, *http.Request)
QueryRange(http.ResponseWriter, *http.Request)
}

View File

@@ -1,259 +0,0 @@
// Package promapi serves the Prometheus HTTP query API over a
// prometheus.Prometheus provider: /query and /query_range in the shape of
// Prometheus' /api/v1 endpoints (https://prometheus.io/docs/prometheus/latest/querying/api/),
// intended to be mounted under a distinguishing prefix (/prometheus/api/v1)
// so PromQL-only endpoints are separate from the SigNoz query APIs. The
// request and response contracts follow Prometheus: form-encoded GET/POST
// params, {"status":"success","data":{resultType,result}} on success and
// {"status":"error","errorType","error"} with Prometheus' status codes on
// failure — so Prometheus-compatible clients can point at the prefix.
package promapi
import (
"context"
"encoding/json"
"log/slog"
"math"
"net/http"
"strconv"
"time"
promModel "github.com/prometheus/common/model"
"github.com/prometheus/prometheus/promql"
"github.com/prometheus/prometheus/promql/parser"
"github.com/prometheus/prometheus/util/stats"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/prometheus"
)
type handler struct {
logger *slog.Logger
prom prometheus.Prometheus
}
func NewHandler(logger *slog.Logger, prom prometheus.Prometheus) prometheus.Handler {
return &handler{logger: logger, prom: prom}
}
type errorType string
const (
errBadData errorType = "bad_data"
errExec errorType = "execution"
errCanceled errorType = "canceled"
errTimeout errorType = "timeout"
errInternal errorType = "internal"
)
type queryData struct {
ResultType parser.ValueType `json:"resultType"`
Result parser.Value `json:"result"`
Stats stats.QueryStats `json:"stats,omitempty"`
}
type response struct {
Status string `json:"status"`
Data *queryData `json:"data,omitempty"`
ErrorType errorType `json:"errorType,omitempty"`
Error string `json:"error,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Infos []string `json:"infos,omitempty"`
}
// QueryRange evaluates an expression over a grid: query, start, end, step,
// and optional timeout/stats params, all in Prometheus' formats.
func (h *handler) QueryRange(w http.ResponseWriter, r *http.Request) {
start, err := parseTime(r.FormValue("start"))
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
end, err := parseTime(r.FormValue("end"))
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
if end.Before(start) {
h.respondError(r.Context(), w, errBadData, errors.NewInvalidInputf(errors.CodeInvalidInput, "end timestamp must not be before start time"))
return
}
step, err := parseDuration(r.FormValue("step"))
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
if step <= 0 {
h.respondError(r.Context(), w, errBadData, errors.NewInvalidInputf(errors.CodeInvalidInput, "zero or negative query resolution step widths are not accepted. Try a positive integer"))
return
}
// The engine materializes every point of every series; an unbounded
// grid is an unbounded allocation. 11,000 points covers 60s resolution
// for a week or 1h resolution for a year.
if end.Sub(start)/step > 11000 {
h.respondError(r.Context(), w, errBadData, errors.NewInvalidInputf(errors.CodeInvalidInput, "exceeded maximum resolution of 11,000 points per timeseries. Try decreasing the query resolution (?step=XX)"))
return
}
ctx, cancel, err := h.contextWithTimeout(r)
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
defer cancel()
if h.tryRangeExecutor(ctx, w, r, start, end, step) {
return
}
qry, err := h.prom.Engine().NewRangeQuery(ctx, h.prom.Storage(), nil, r.FormValue("query"), start, end, step)
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
h.exec(ctx, w, r, qry)
}
// tryRangeExecutor serves the query the way a RangeExecutor provider is
// designed to serve: evaluated inside the datastore when the shape allows.
// It reports whether the response was written.
func (h *handler) tryRangeExecutor(ctx context.Context, w http.ResponseWriter, r *http.Request, start, end time.Time, step time.Duration) bool {
re, ok := h.prom.(prometheus.RangeExecutor)
if !ok {
return false
}
matrix, served, err := re.TryExecuteRange(ctx, r.FormValue("query"), start, end, step)
if err != nil {
h.respondError(ctx, w, errExec, err)
return true
}
if !served {
return false
}
h.respond(ctx, w, &queryData{ResultType: matrix.Type(), Result: matrix}, nil, nil)
return true
}
// Query evaluates an expression at a single instant: query and optional
// time/timeout/stats params. A missing time evaluates at the server's now,
// as in Prometheus.
func (h *handler) Query(w http.ResponseWriter, r *http.Request) {
ts := time.Now()
if t := r.FormValue("time"); t != "" {
var err error
ts, err = parseTime(t)
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
}
ctx, cancel, err := h.contextWithTimeout(r)
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
defer cancel()
qry, err := h.prom.Engine().NewInstantQuery(ctx, h.prom.Storage(), nil, r.FormValue("query"), ts)
if err != nil {
h.respondError(r.Context(), w, errBadData, err)
return
}
h.exec(ctx, w, r, qry)
}
func (h *handler) exec(ctx context.Context, w http.ResponseWriter, r *http.Request, qry promql.Query) {
defer qry.Close()
res := qry.Exec(ctx)
if res.Err != nil {
h.logger.ErrorContext(ctx, "error evaluating promql query", errors.Attr(res.Err))
switch res.Err.(type) {
case promql.ErrQueryCanceled:
h.respondError(ctx, w, errCanceled, res.Err)
case promql.ErrQueryTimeout:
h.respondError(ctx, w, errTimeout, res.Err)
case promql.ErrStorage:
h.respondError(ctx, w, errInternal, res.Err)
default:
h.respondError(ctx, w, errExec, res.Err)
}
return
}
data := &queryData{ResultType: res.Value.Type(), Result: res.Value}
if r.FormValue("stats") != "" {
data.Stats = stats.NewQueryStats(qry.Stats())
}
warnings, infos := res.Warnings.AsStrings(r.FormValue("query"), 10, 10)
h.respond(ctx, w, data, warnings, infos)
}
func (h *handler) contextWithTimeout(r *http.Request) (context.Context, context.CancelFunc, error) {
ctx := r.Context()
if to := r.FormValue("timeout"); to != "" {
timeout, err := parseDuration(to)
if err != nil {
return nil, nil, err
}
ctx, cancel := context.WithTimeout(ctx, timeout)
return ctx, cancel, nil
}
ctx, cancel := context.WithCancel(ctx)
return ctx, cancel, nil
}
func (h *handler) respond(ctx context.Context, w http.ResponseWriter, data *queryData, warnings, infos []string) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
if err := json.NewEncoder(w).Encode(&response{Status: "success", Data: data, Warnings: warnings, Infos: infos}); err != nil {
h.logger.ErrorContext(ctx, "error writing prometheus api response", errors.Attr(err))
}
}
// respondError follows Prometheus' status-code mapping: bad_data 400,
// execution 422, canceled/timeout 503, internal 500.
func (h *handler) respondError(ctx context.Context, w http.ResponseWriter, typ errorType, err error) {
code := http.StatusInternalServerError
switch typ {
case errBadData:
code = http.StatusBadRequest
case errExec:
code = http.StatusUnprocessableEntity
case errCanceled, errTimeout:
code = http.StatusServiceUnavailable
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
if encErr := json.NewEncoder(w).Encode(&response{Status: "error", ErrorType: typ, Error: err.Error()}); encErr != nil {
h.logger.ErrorContext(ctx, "error writing prometheus api error response", errors.Attr(encErr))
}
}
// parseTime accepts Prometheus' time formats: float unix seconds or RFC3339.
func parseTime(s string) (time.Time, error) {
if t, err := strconv.ParseFloat(s, 64); err == nil {
sec, ns := math.Modf(t)
return time.Unix(int64(sec), int64(ns*float64(time.Second))), nil
}
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
return t, nil
}
return time.Time{}, errors.NewInvalidInputf(errors.CodeInvalidInput, "cannot parse %q to a valid timestamp", s)
}
// parseDuration accepts Prometheus' duration formats: float seconds or a
// duration string like 5m.
func parseDuration(s string) (time.Duration, error) {
if d, err := strconv.ParseFloat(s, 64); err == nil {
ts := d * float64(time.Second)
if ts > float64(math.MaxInt64) || ts < float64(math.MinInt64) {
return 0, errors.NewInvalidInputf(errors.CodeInvalidInput, "cannot parse %q to a valid duration. It overflows int64", s)
}
return time.Duration(ts), nil
}
if d, err := promModel.ParseDuration(s); err == nil {
return time.Duration(d), nil
}
return 0, errors.NewInvalidInputf(errors.CodeInvalidInput, "cannot parse %q to a valid duration", s)
}

View File

@@ -23,7 +23,11 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
traces uint64
tracesLastSeenAt time.Time
)
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), max(timestamp) FROM %s", tracesTable)).Scan(&traces, &tracesLastSeenAt); err == nil {
tracesLastSeenExpr := "max(timestamp)"
if q.hasColumn(ctx, tracestelemetryschema.DBName, tracestelemetryschema.SpanIndexV3TableName, "inserted_at") {
tracesLastSeenExpr = "max(inserted_at)"
}
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), %s FROM %s", tracesLastSeenExpr, tracesTable)).Scan(&traces, &tracesLastSeenAt); err == nil {
stats["telemetry.traces.count"] = traces
if tracesLastSeenAt.Unix() != 0 {
stats["telemetry.traces.last_observed.time"] = tracesLastSeenAt.UTC()
@@ -37,7 +41,11 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
logs uint64
logsLastSeenAt time.Time
)
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), fromUnixTimestamp64Nano(max(timestamp)) FROM %s", logsTable)).Scan(&logs, &logsLastSeenAt); err == nil {
logsLastSeenExpr := "fromUnixTimestamp64Nano(max(timestamp))"
if q.hasColumn(ctx, logstelemetryschema.DBName, logstelemetryschema.LogsV2TableName, "inserted_at") {
logsLastSeenExpr = "max(inserted_at)"
}
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), %s FROM %s", logsLastSeenExpr, logsTable)).Scan(&logs, &logsLastSeenAt); err == nil {
stats["telemetry.logs.count"] = logs
if logsLastSeenAt.Unix() != 0 {
stats["telemetry.logs.last_observed.time"] = logsLastSeenAt.UTC()
@@ -51,7 +59,11 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
metrics uint64
metricsLastSeenAt time.Time
)
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), toDateTime(max(unix_milli) / 1000) FROM %s", metricsTable)).Scan(&metrics, &metricsLastSeenAt); err == nil {
metricsLastSeenExpr := "toDateTime(max(unix_milli) / 1000)"
if q.hasColumn(ctx, metricstelemetryschema.DBName, metricstelemetryschema.SamplesV4TableName, "inserted_at_unix_milli") {
metricsLastSeenExpr = "fromUnixTimestamp64Milli(max(inserted_at_unix_milli))"
}
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), %s FROM %s", metricsLastSeenExpr, metricsTable)).Scan(&metrics, &metricsLastSeenAt); err == nil {
stats["telemetry.metrics.count"] = metrics
if metricsLastSeenAt.Unix() != 0 {
stats["telemetry.metrics.last_observed.time"] = metricsLastSeenAt.UTC()
@@ -63,3 +75,12 @@ func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, e
return stats, nil
}
func (q *querier) hasColumn(ctx context.Context, database, table, column string) bool {
var exists bool
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT hasColumnInTable(?, ?, ?)", database, table, column).Scan(&exists); err != nil {
q.logger.DebugContext(ctx, "failed to check column existence", errors.Attr(err))
return false
}
return exists
}

View File

@@ -485,9 +485,6 @@ func (aH *APIHandler) Respond(w http.ResponseWriter, data interface{}) {
func (aH *APIHandler) RegisterRoutes(router *mux.Router, am *middleware.AuthZ) {
router.HandleFunc("/api/v1/query_range", am.ViewAccess(aH.queryRangeMetrics)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/query", am.ViewAccess(aH.queryMetrics)).Methods(http.MethodGet)
router.HandleFunc("/prometheus/api/v1/query_range", am.ViewAccess(aH.Signoz.Handlers.PrometheusHandler.QueryRange)).Methods(http.MethodGet, http.MethodPost)
router.HandleFunc("/prometheus/api/v1/query", am.ViewAccess(aH.Signoz.Handlers.PrometheusHandler.Query)).Methods(http.MethodGet, http.MethodPost)
router.HandleFunc("/api/v1/rules", am.ViewAccess(aH.listRules)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/rules/{id}", am.ViewAccess(aH.getRule)).Methods(http.MethodGet)
router.HandleFunc("/api/v1/rules", am.EditAccess(aH.createRule)).Methods(http.MethodPost)

View File

@@ -0,0 +1,22 @@
// Code generated by scripts/semconv. DO NOT EDIT.
package semconv
var families = []Family{
{
Current: "db.system.name",
Old: []string{"db.system"},
Kind: KindAttribute,
Contexts: nil,
Signals: nil,
ApplyToMetrics: nil,
},
{
Current: "deployment.environment.name",
Old: []string{"deployment.environment"},
Kind: KindAttribute,
Contexts: nil,
Signals: nil,
ApplyToMetrics: nil,
},
}

127
pkg/semconv/semconv.go Normal file
View File

@@ -0,0 +1,127 @@
package semconv
import (
"slices"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
//go:generate go run ../../scripts/semconv
// Kind identifies whether a family describes an attribute or a metric name.
type Kind struct {
valuer.String
}
// Family is one logical telemetry field. Old is ordered from the most recent
// predecessor to the oldest one and therefore also defines fallback order.
type Family struct {
Current string
Old []string
Kind Kind
Contexts []telemetrytypes.FieldContext
Signals []telemetrytypes.Signal
ApplyToMetrics []string
ValueMap map[string]string
}
var (
KindAttribute = Kind{String: valuer.NewString("attribute")}
KindMetric = Kind{String: valuer.NewString("metric")}
)
var memberToFamilies, familyMembers = buildIndexes()
// Enum returns the acceptable values for Kind.
func (Kind) Enum() []any {
return []any{KindAttribute, KindMetric}
}
// Lookup returns the enabled family containing selector.Name for kind. The
// returned family must not be modified.
func Lookup(kind Kind, selector telemetrytypes.FieldKeySelector) (Family, bool) {
idx, ok := lookupIndex(kind, selector)
if !ok {
return Family{}, false
}
return families[idx], true
}
// Members returns the current name first, followed by historical names in
// fallback order. A name outside an enabled family is returned unchanged. The
// returned slice must not be modified.
func Members(kind Kind, selector telemetrytypes.FieldKeySelector) []string {
idx, ok := lookupIndex(kind, selector)
if !ok {
return []string{selector.Name}
}
return familyMembers[idx]
}
// Current returns the current name for selector.Name, or the input name when
// it does not belong to an enabled family.
func Current(kind Kind, selector telemetrytypes.FieldKeySelector) string {
idx, ok := lookupIndex(kind, selector)
if !ok {
return selector.Name
}
return families[idx].Current
}
// All returns every enabled family. The returned slice and families must not be
// modified.
func All() []Family {
return families
}
func buildIndexes() (map[string][]int, [][]string) {
index := make(map[string][]int)
members := make([][]string, len(families))
for i, family := range families {
members[i] = make([]string, 0, len(family.Old)+1)
members[i] = append(members[i], family.Current)
members[i] = append(members[i], family.Old...)
index[family.Current] = append(index[family.Current], i)
for _, old := range family.Old {
index[old] = append(index[old], i)
}
}
return index, members
}
func lookupIndex(kind Kind, selector telemetrytypes.FieldKeySelector) (int, bool) {
for _, idx := range memberToFamilies[selector.Name] {
if matchesSelector(families[idx], kind, selector) {
return idx, true
}
}
return 0, false
}
func matchesSelector(family Family, kind Kind, selector telemetrytypes.FieldKeySelector) bool {
if family.Kind != kind {
return false
}
if selector.Signal != telemetrytypes.SignalUnspecified && len(family.Signals) > 0 {
if !slices.Contains(family.Signals, selector.Signal) {
return false
}
}
if selector.FieldContext != telemetrytypes.FieldContextUnspecified && len(family.Contexts) > 0 {
if !slices.Contains(family.Contexts, selector.FieldContext) {
return false
}
}
if selector.Signal == telemetrytypes.SignalMetrics && len(family.ApplyToMetrics) > 0 {
if selector.MetricContext == nil {
return false
}
return slices.Contains(family.ApplyToMetrics, selector.MetricContext.MetricName)
}
return true
}

View File

@@ -0,0 +1,80 @@
package semconv
import (
"testing"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
)
func TestMembersReturnsCurrentBeforeHistoricalName(t *testing.T) {
selector := telemetrytypes.FieldKeySelector{
Name: "deployment.environment",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
}
assert.Equal(t,
[]string{"deployment.environment.name", "deployment.environment"},
Members(KindAttribute, selector),
"members should use current-first fallback order",
)
}
func TestCurrentReturnsCanonicalName(t *testing.T) {
selector := telemetrytypes.FieldKeySelector{
Name: "deployment.environment",
Signal: telemetrytypes.SignalTraces,
FieldContext: telemetrytypes.FieldContextResource,
}
assert.Equal(t,
"deployment.environment.name",
Current(KindAttribute, selector),
"historical name should resolve to the current family name",
)
}
func TestAllScopedFamilyMatchesSupportedScopes(t *testing.T) {
tests := []struct {
name string
signal telemetrytypes.Signal
fieldContext telemetrytypes.FieldContext
}{
{name: "trace resource", signal: telemetrytypes.SignalTraces, fieldContext: telemetrytypes.FieldContextResource},
{name: "trace attribute", signal: telemetrytypes.SignalTraces, fieldContext: telemetrytypes.FieldContextAttribute},
{name: "log resource", signal: telemetrytypes.SignalLogs, fieldContext: telemetrytypes.FieldContextResource},
{name: "log attribute", signal: telemetrytypes.SignalLogs, fieldContext: telemetrytypes.FieldContextAttribute},
{name: "metric resource", signal: telemetrytypes.SignalMetrics, fieldContext: telemetrytypes.FieldContextResource},
{name: "metric attribute", signal: telemetrytypes.SignalMetrics, fieldContext: telemetrytypes.FieldContextAttribute},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
selector := telemetrytypes.FieldKeySelector{
Name: "deployment.environment",
Signal: test.signal,
FieldContext: test.fieldContext,
}
assert.Equal(t,
"deployment.environment.name",
Current(KindAttribute, selector),
"an all-scoped family should match every supported signal and attribute context",
)
})
}
}
func TestMembersReturnsInputWhenKindDoesNotMatch(t *testing.T) {
selector := telemetrytypes.FieldKeySelector{
Name: "deployment.environment",
Signal: telemetrytypes.SignalTraces,
}
assert.Equal(t,
[]string{"deployment.environment"},
Members(KindMetric, selector),
"an attribute family must not match a metric-name lookup",
)
}

View File

@@ -22,6 +22,6 @@ func NewAuthNs(ctx context.Context, providerSettings factory.ProviderSettings, s
return map[authtypes.AuthNProvider]authn.AuthN{
authtypes.AuthNProviderEmailPassword: emailPasswordAuthN,
authtypes.AuthNProviderGoogleAuth: googleCallbackAuthN,
authtypes.AuthNProviderGoogle: googleCallbackAuthN,
}, nil
}

View File

@@ -48,8 +48,6 @@ import (
"github.com/SigNoz/signoz/pkg/modules/tracedetail/impltracedetail"
"github.com/SigNoz/signoz/pkg/modules/tracefunnel"
"github.com/SigNoz/signoz/pkg/modules/tracefunnel/impltracefunnel"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/prometheus/promapi"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/ruler/signozruler"
@@ -83,7 +81,6 @@ type Handlers struct {
RuleStateHistory rulestatehistory.Handler
SpanMapperHandler spanmapper.Handler
AlertmanagerHandler alertmanager.Handler
PrometheusHandler prometheus.Handler
TraceDetail tracedetail.Handler
RulerHandler ruler.Handler
LLMPricingRuleHandler llmpricingrule.Handler
@@ -104,7 +101,6 @@ func NewHandlers(
zeusService zeus.Zeus,
registryHandler factory.Handler,
alertmanagerService alertmanager.Alertmanager,
prometheusService prometheus.Prometheus,
rulerService ruler.Ruler,
statsAggregator statsreporter.Aggregator,
) Handlers {
@@ -133,7 +129,6 @@ func NewHandlers(
CloudIntegrationHandler: implcloudintegration.NewHandler(modules.CloudIntegration),
SpanMapperHandler: implspanmapper.NewHandler(modules.SpanMapper),
AlertmanagerHandler: signozalertmanager.NewHandler(alertmanagerService),
PrometheusHandler: promapi.NewHandler(providerSettings.Logger, prometheusService),
TraceDetail: impltracedetail.NewHandler(modules.TraceDetail),
RulerHandler: signozruler.NewHandler(rulerService),
LLMPricingRuleHandler: impllmpricingrule.NewHandler(modules.LLMPricingRule),

View File

@@ -63,7 +63,7 @@ func TestNewHandlers(t *testing.T) {
querierHandler := querier.NewHandler(providerSettings, nil, nil)
registryHandler := factory.NewHandler(nil)
handlers := NewHandlers(modules, providerSettings, nil, querierHandler, nil, nil, nil, nil, nil, nil, nil, registryHandler, alertmanager, nil, nil, nil)
handlers := NewHandlers(modules, providerSettings, nil, querierHandler, nil, nil, nil, nil, nil, nil, nil, registryHandler, alertmanager, nil, nil)
reflectVal := reflect.ValueOf(handlers)
for i := 0; i < reflectVal.NumField(); i++ {
f := reflectVal.Field(i)

View File

@@ -617,7 +617,7 @@ func New(
// Initialize all handlers for the modules
registryHandler := factory.NewHandler(registry)
handlers := NewHandlers(modules, providerSettings, analytics, querierHandler, licensing, global, flagger, gateway, telemetryMetadataStore, authz, zeus, registryHandler, alertmanager, prometheus, rulerInstance, statsAggregator)
handlers := NewHandlers(modules, providerSettings, analytics, querierHandler, licensing, global, flagger, gateway, telemetryMetadataStore, authz, zeus, registryHandler, alertmanager, rulerInstance, statsAggregator)
// Initialize the API server (after registry so it can access service health)
apiserverInstance, err := factory.NewProviderFromNamedMap(

View File

@@ -15,6 +15,7 @@ import (
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
)
type restructureSavedViewSpec struct {
@@ -164,6 +165,15 @@ func (migration *restructureSavedViewSpec) Up(ctx context.Context, db *bun.DB) e
return err
}
var orgIDs []string
if err := tx.NewSelect().Model((*types.Organization)(nil)).Column("id").Scan(ctx, &orgIDs); err != nil {
return err
}
validOrgIDs := make(map[string]struct{}, len(orgIDs))
for _, id := range orgIDs {
validOrgIDs[id] = struct{}{}
}
// drop table `saved_views`
for _, sql := range migration.sqlschema.Operator().DropTable(savedViewsTable) {
if _, err := tx.ExecContext(ctx, string(sql)); err != nil {
@@ -210,6 +220,13 @@ func (migration *restructureSavedViewSpec) Up(ctx context.Context, db *bun.DB) e
continue // orphaned row from a pre-existing org_id backfill gap; nothing sane to attach it to
}
// to avoid foreign key constraint issues
if _, ok := validOrgIDs[old.OrgID]; !ok {
skipped++
migration.settings.Logger.WarnContext(ctx, "saved view references an org that no longer exists, skipping", slog.String("org_id", old.OrgID), slog.String("saved_view_id", old.ID))
continue
}
var compositeQuery legacySavedViewCompositeQuery
if err := json.Unmarshal([]byte(old.Data), &compositeQuery); err != nil {
failed++

View File

@@ -16,7 +16,7 @@ var (
)
var (
AuthNProviderGoogleAuth = AuthNProvider{valuer.NewString("google_auth")}
AuthNProviderGoogle = AuthNProvider{valuer.NewString("google")}
AuthNProviderSAML = AuthNProvider{valuer.NewString("saml")}
AuthNProviderEmailPassword = AuthNProvider{valuer.NewString("email_password")}
AuthNProviderOIDC = AuthNProvider{valuer.NewString("oidc")}
@@ -158,7 +158,7 @@ func (typ *Identity) ToClaims() Claims {
func (AuthNProvider) Enum() []any {
return []any{
AuthNProviderGoogleAuth,
AuthNProviderGoogle,
AuthNProviderSAML,
AuthNProviderEmailPassword,
AuthNProviderOIDC,

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/swaggest/jsonschema-go"
"github.com/uptrace/bun"
)
@@ -30,7 +31,9 @@ var (
type GettableAuthDomain struct {
StorableAuthDomain
Enabled bool `json:"enabled"`
Config AuthDomainConfig `json:"config"`
RoleMapping *RoleMapping `json:"roleMapping"`
AuthNProviderInfo *AuthNProviderInfo `json:"authNProviderInfo"`
}
@@ -39,12 +42,16 @@ type AuthNProviderInfo struct {
}
type PostableAuthDomain struct {
Config AuthDomainConfig `json:"config"`
Name string `json:"name"`
Name string `json:"name" required:"true"`
Enabled bool `json:"enabled"`
Config AuthDomainConfig `json:"config" required:"true"`
RoleMapping *RoleMapping `json:"roleMapping"`
}
type UpdatableAuthDomain struct {
Config AuthDomainConfig `json:"config"`
Enabled bool `json:"enabled"`
Config AuthDomainConfig `json:"config" required:"true"`
RoleMapping *RoleMapping `json:"roleMapping"`
}
type StorableAuthDomain struct {
@@ -57,36 +64,96 @@ type StorableAuthDomain struct {
types.TimeAuditable
}
// TODO: the oneOf emitted by JSONSchemaOneOf is not the shape OpenAPI wants
// for a discriminated union. OpenAPI's discriminator requires every oneOf
// branch to be a $ref to a named component and a sibling property whose value
// selects the variant. ssoType is already discriminator-shaped, but the
// variant payload lives in a sibling field (samlConfig / googleAuthConfig /
// oidcConfig) instead of being the payload itself, so no discriminator can
// be attached. Refactor AuthDomainConfig into an envelope (see
// ruletypes.RuleThresholdData for the pattern) where the chosen config is
// the payload and ssoType is the discriminator.
type AuthDomainConfig struct {
SSOEnabled bool `json:"ssoEnabled"`
AuthNProvider AuthNProvider `json:"ssoType"`
SAML *SamlConfig `json:"samlConfig"`
Google *GoogleConfig `json:"googleAuthConfig"`
OIDC *OIDCConfig `json:"oidcConfig"`
RoleMapping *RoleMapping `json:"roleMapping"`
Kind AuthNProvider `json:"kind" required:"true"`
Spec any `json:"spec" required:"true"`
}
// authDomainConfigSAML is the OpenAPI schema for an AuthDomainConfig with kind=saml.
type authDomainConfigSAML struct {
Kind AuthNProvider `json:"kind" description:"The kind of authn provider." required:"true"`
Spec SamlConfig `json:"spec" description:"The saml configuration." required:"true"`
}
// authDomainConfigGoogle is the OpenAPI schema for an AuthDomainConfig with kind=google.
type authDomainConfigGoogle struct {
Kind AuthNProvider `json:"kind" description:"The kind of authn provider." required:"true"`
Spec GoogleConfig `json:"spec" description:"The google auth configuration." required:"true"`
}
// authDomainConfigOIDC is the OpenAPI schema for an AuthDomainConfig with kind=oidc.
type authDomainConfigOIDC struct {
Kind AuthNProvider `json:"kind" description:"The kind of authn provider." required:"true"`
Spec OIDCConfig `json:"spec" description:"The oidc configuration." required:"true"`
}
var (
_ jsonschema.OneOfExposer = AuthDomainConfig{}
_ jsonschema.Preparer = AuthDomainConfig{}
)
// JSONSchemaOneOf returns the oneOf variants for the AuthDomainConfig discriminated union.
// Each variant represents a different authn provider kind with its corresponding spec schema.
func (AuthDomainConfig) JSONSchemaOneOf() []any {
return []any{
authDomainConfigSAML{},
authDomainConfigGoogle{},
authDomainConfigOIDC{},
}
}
// PrepareJSONSchema marks the schema with x-signoz-discriminator;
// signoz.attachDiscriminators promotes it to a real OpenAPI 3
// discriminator after reflection.
func (AuthDomainConfig) PrepareJSONSchema(schema *jsonschema.Schema) error {
if schema.ExtraProperties == nil {
schema.ExtraProperties = map[string]any{}
}
schema.ExtraProperties["x-signoz-discriminator"] = map[string]any{
"propertyName": "kind",
"mapping": map[string]string{
AuthNProviderSAML.StringValue(): "#/components/schemas/AuthtypesAuthDomainConfigSAML",
AuthNProviderGoogle.StringValue(): "#/components/schemas/AuthtypesAuthDomainConfigGoogle",
AuthNProviderOIDC.StringValue(): "#/components/schemas/AuthtypesAuthDomainConfigOIDC",
},
}
return nil
}
// StorableAuthDomainConfig is the JSON document persisted in StorableAuthDomain.Data.
// Its shape (and the shapes it nests) must stay backward compatible with existing rows.
type StorableAuthDomainConfig struct {
SSOEnabled bool `json:"ssoEnabled"`
AuthNProvider AuthNProvider `json:"ssoType"`
SAML *StorableSamlConfig `json:"samlConfig"`
Google *GoogleConfig `json:"googleAuthConfig"`
OIDC *OIDCConfig `json:"oidcConfig"`
RoleMapping *RoleMapping `json:"roleMapping"`
}
// storableAuthNProviderGoogle is the value persisted in ssoType for google domains,
// kept for compatibility with rows written before the provider was renamed.
var storableAuthNProviderGoogle = AuthNProvider{valuer.NewString("google_auth")}
type AuthDomain struct {
storableAuthDomain *StorableAuthDomain
authDomainConfig *AuthDomainConfig
storableAuthDomain *StorableAuthDomain
storableAuthDomainConfig *StorableAuthDomainConfig
}
func NewAuthDomainFromConfig(name string, config *AuthDomainConfig, orgID valuer.UUID) (*AuthDomain, error) {
data, err := json.Marshal(config)
func NewAuthDomainFromPostableAuthDomain(postableAuthDomain *PostableAuthDomain, orgID valuer.UUID) (*AuthDomain, error) {
storableAuthDomainConfig, err := newStorableAuthDomainConfig(postableAuthDomain.Enabled, postableAuthDomain.Config, postableAuthDomain.RoleMapping)
if err != nil {
return nil, err
}
return NewAuthDomain(name, string(data), orgID)
data, err := json.Marshal(storableAuthDomainConfig)
if err != nil {
return nil, err
}
return NewAuthDomain(postableAuthDomain.Name, string(data), orgID)
}
func NewAuthDomain(name string, data string, orgID valuer.UUID) (*AuthDomain, error) {
@@ -107,22 +174,85 @@ func NewAuthDomain(name string, data string, orgID valuer.UUID) (*AuthDomain, er
}
func NewAuthDomainFromStorableAuthDomain(storableAuthDomain *StorableAuthDomain) (*AuthDomain, error) {
authDomainConfig := new(AuthDomainConfig)
if err := json.Unmarshal([]byte(storableAuthDomain.Data), authDomainConfig); err != nil {
storableAuthDomainConfig := new(StorableAuthDomainConfig)
if err := json.Unmarshal([]byte(storableAuthDomain.Data), storableAuthDomainConfig); err != nil {
return nil, err
}
return &AuthDomain{
storableAuthDomain: storableAuthDomain,
authDomainConfig: authDomainConfig,
storableAuthDomain: storableAuthDomain,
storableAuthDomainConfig: storableAuthDomainConfig,
}, nil
}
func NewGettableAuthDomainFromAuthDomain(authDomain *AuthDomain, authNProviderInfo *AuthNProviderInfo) *GettableAuthDomain {
func NewGettableAuthDomainFromAuthDomain(authDomain *AuthDomain, authNProviderInfo *AuthNProviderInfo) (*GettableAuthDomain, error) {
config, err := newAuthDomainConfigFromStorableAuthDomainConfig(authDomain.StorableAuthDomainConfig())
if err != nil {
return nil, err
}
return &GettableAuthDomain{
StorableAuthDomain: *authDomain.StorableAuthDomain(),
Config: *authDomain.AuthDomainConfig(),
Enabled: authDomain.StorableAuthDomainConfig().SSOEnabled,
Config: config,
RoleMapping: authDomain.StorableAuthDomainConfig().RoleMapping,
AuthNProviderInfo: authNProviderInfo,
}, nil
}
func newStorableAuthDomainConfig(enabled bool, config AuthDomainConfig, roleMapping *RoleMapping) (*StorableAuthDomainConfig, error) {
storableAuthDomainConfig := &StorableAuthDomainConfig{
SSOEnabled: enabled,
AuthNProvider: config.Kind,
RoleMapping: roleMapping,
}
switch config.Kind {
case AuthNProviderSAML:
spec, ok := config.Spec.(SamlConfig)
if !ok {
return nil, errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "saml config is required")
}
samlConfig := StorableSamlConfig(spec)
storableAuthDomainConfig.SAML = &samlConfig
case AuthNProviderGoogle:
spec, ok := config.Spec.(GoogleConfig)
if !ok {
return nil, errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "google auth config is required")
}
storableAuthDomainConfig.Google = &spec
case AuthNProviderOIDC:
spec, ok := config.Spec.(OIDCConfig)
if !ok {
return nil, errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "oidc config is required")
}
storableAuthDomainConfig.OIDC = &spec
default:
return nil, errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "invalid authn provider %q", config.Kind.StringValue())
}
return storableAuthDomainConfig, nil
}
func newAuthDomainConfigFromStorableAuthDomainConfig(storableAuthDomainConfig *StorableAuthDomainConfig) (AuthDomainConfig, error) {
switch storableAuthDomainConfig.AuthNProvider {
case AuthNProviderSAML:
return AuthDomainConfig{Kind: AuthNProviderSAML, Spec: SamlConfig(*storableAuthDomainConfig.SAML)}, nil
case AuthNProviderGoogle:
return AuthDomainConfig{Kind: AuthNProviderGoogle, Spec: *storableAuthDomainConfig.Google}, nil
case AuthNProviderOIDC:
return AuthDomainConfig{Kind: AuthNProviderOIDC, Spec: *storableAuthDomainConfig.OIDC}, nil
default:
return AuthDomainConfig{}, errors.Newf(errors.TypeInternal, ErrCodeAuthDomainInvalidConfig, "invalid authn provider %q", storableAuthDomainConfig.AuthNProvider.StringValue())
}
}
@@ -130,17 +260,22 @@ func (typ *AuthDomain) StorableAuthDomain() *StorableAuthDomain {
return typ.storableAuthDomain
}
func (typ *AuthDomain) AuthDomainConfig() *AuthDomainConfig {
return typ.authDomainConfig
func (typ *AuthDomain) StorableAuthDomainConfig() *StorableAuthDomainConfig {
return typ.storableAuthDomainConfig
}
func (typ *AuthDomain) Update(config *AuthDomainConfig) error {
data, err := json.Marshal(config)
func (typ *AuthDomain) Update(updatableAuthDomain *UpdatableAuthDomain) error {
storableAuthDomainConfig, err := newStorableAuthDomainConfig(updatableAuthDomain.Enabled, updatableAuthDomain.Config, updatableAuthDomain.RoleMapping)
if err != nil {
return err
}
typ.authDomainConfig = config
data, err := json.Marshal(storableAuthDomainConfig)
if err != nil {
return err
}
typ.storableAuthDomainConfig = storableAuthDomainConfig
typ.storableAuthDomain.Data = string(data)
typ.storableAuthDomain.UpdatedAt = time.Now()
return nil
@@ -163,15 +298,84 @@ func (typ *PostableAuthDomain) UnmarshalJSON(data []byte) error {
}
func (typ *AuthDomainConfig) UnmarshalJSON(data []byte) error {
type Alias AuthDomainConfig
var raw map[string]json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return errors.Wrapf(err, errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "failed to unmarshal auth domain config")
}
kindData, ok := raw["kind"]
if !ok {
return errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "kind is required")
}
var kind AuthNProvider
if err := json.Unmarshal(kindData, &kind); err != nil {
return errors.Wrapf(err, errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "failed to unmarshal kind")
}
specData, ok := raw["spec"]
if !ok {
return errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "spec is required")
}
switch kind {
case AuthNProviderSAML:
spec := SamlConfig{}
if err := json.Unmarshal(specData, &spec); err != nil {
return err
}
typ.Spec = spec
case AuthNProviderGoogle:
spec := GoogleConfig{}
if err := json.Unmarshal(specData, &spec); err != nil {
return err
}
typ.Spec = spec
case AuthNProviderOIDC:
spec := OIDCConfig{}
if err := json.Unmarshal(specData, &spec); err != nil {
return err
}
typ.Spec = spec
default:
return errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "invalid authn provider %q", kind.StringValue())
}
typ.Kind = kind
return nil
}
func (typ StorableAuthDomainConfig) MarshalJSON() ([]byte, error) {
type Alias StorableAuthDomainConfig
temp := Alias(typ)
if temp.AuthNProvider == AuthNProviderGoogle {
temp.AuthNProvider = storableAuthNProviderGoogle
}
return json.Marshal(temp)
}
func (typ *StorableAuthDomainConfig) UnmarshalJSON(data []byte) error {
type Alias StorableAuthDomainConfig
var temp Alias
if err := json.Unmarshal(data, &temp); err != nil {
return err
}
if temp.AuthNProvider == storableAuthNProviderGoogle {
temp.AuthNProvider = AuthNProviderGoogle
}
switch temp.AuthNProvider {
case AuthNProviderGoogleAuth:
case AuthNProviderGoogle:
if temp.Google == nil {
return errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "google auth config is required")
}
@@ -190,17 +394,8 @@ func (typ *AuthDomainConfig) UnmarshalJSON(data []byte) error {
return errors.Newf(errors.TypeInvalidInput, ErrCodeAuthDomainInvalidConfig, "invalid authn provider %q", temp.AuthNProvider.StringValue())
}
*typ = AuthDomainConfig(temp)
*typ = StorableAuthDomainConfig(temp)
return nil
}
func (AuthDomainConfig) JSONSchemaOneOf() []any {
return []any{
SamlConfig{},
GoogleConfig{},
OIDCConfig{},
}
}
type AuthDomainStore interface {

View File

@@ -12,10 +12,10 @@ const wildCardDomain = "*"
type GoogleConfig struct {
// ClientID is the application's ID. For example, 292085223830.apps.googleusercontent.com.
ClientID string `json:"clientId"`
ClientID string `json:"clientId" required:"true"`
// It is the application's secret.
ClientSecret string `json:"clientSecret"`
ClientSecret string `json:"clientSecret" required:"true"`
// What is the meaning of this? Should we remove this?
RedirectURI string `json:"redirectURI"`

View File

@@ -8,7 +8,7 @@ import (
type OIDCConfig struct {
// It is the URL identifier for the service. For example: "https://accounts.google.com" or "https://login.salesforce.com".
Issuer string `json:"issuer"`
Issuer string `json:"issuer" required:"true"`
// Some offspec providers like Azure, Oracle IDCS have oidc discovery url different from issuer url which causes issuerValidation to fail
// This provides a way to override the Issuer url from the .well-known/openid-configuration issuer
@@ -16,10 +16,10 @@ type OIDCConfig struct {
IssuerAlias string `json:"issuerAlias"`
// It is the application's ID.
ClientID string `json:"clientId"`
ClientID string `json:"clientId" required:"true"`
// It is the application's secret.
ClientSecret string `json:"clientSecret"`
ClientSecret string `json:"clientSecret" required:"true"`
// Mapping of claims to the corresponding fields in the token.
ClaimMapping AttributeMapping `json:"claimMapping"`

View File

@@ -7,14 +7,14 @@ import (
)
type SamlConfig struct {
// The entityID of the SAML identity provider. It can typically be found in the EntityID attribute of the EntityDescriptor element in the SAML metadata of the identity provider. Example: <md:EntityDescriptor xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" entityID="{samlEntity}">
SamlEntity string `json:"samlEntity"`
// The entityID of the SAML identity provider. It can typically be found in the EntityID attribute of the EntityDescriptor element in the SAML metadata of the identity provider. Example: <md:EntityDescriptor xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" entityID="{entityId}">
EntityID string `json:"entityId" required:"true"`
// The SSO endpoint of the SAML identity provider. It can typically be found in the SingleSignOnService element in the SAML metadata of the identity provider. Example: <md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="{samlIdp}"/>
SamlIdp string `json:"samlIdp"`
// The SSO endpoint of the SAML identity provider. It can typically be found in the Location attribute of the SingleSignOnService element in the SAML metadata of the identity provider. Example: <md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="{location}"/>
Location string `json:"location" required:"true"`
// The certificate of the SAML identity provider. It can typically be found in the X509Certificate element in the SAML metadata of the identity provider. Example: <ds:X509Certificate><ds:X509Certificate>{samlCert}</ds:X509Certificate></ds:X509Certificate>
SamlCert string `json:"samlCert"`
// The certificate of the SAML identity provider. It can typically be found in the X509Certificate element in the SAML metadata of the identity provider. Example: <ds:X509Certificate><ds:X509Certificate>{certificate}</ds:X509Certificate></ds:X509Certificate>
Certificate string `json:"certificate" required:"true"`
// Whether to skip signing the SAML requests. It can typically be found in the WantAuthnRequestsSigned attribute of the IDPSSODescriptor element in the SAML metadata of the identity provider. Example: <md:IDPSSODescriptor WantAuthnRequestsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
// For providers like jumpcloud, this should be set to true.
@@ -25,6 +25,17 @@ type SamlConfig struct {
AttributeMapping AttributeMapping `json:"attributeMapping"`
}
// StorableSamlConfig is SamlConfig in its persisted shape. It differs from SamlConfig
// only in JSON keys, which are kept for compatibility with rows written before the
// keys were renamed.
type StorableSamlConfig struct {
EntityID string `json:"samlEntity"`
Location string `json:"samlIdp"`
Certificate string `json:"samlCert"`
InsecureSkipAuthNRequestsSigned bool `json:"insecureSkipAuthNRequestsSigned"`
AttributeMapping AttributeMapping `json:"attributeMapping"`
}
func (config *SamlConfig) UnmarshalJSON(data []byte) error {
type Alias SamlConfig
@@ -33,24 +44,51 @@ func (config *SamlConfig) UnmarshalJSON(data []byte) error {
return err
}
if temp.SamlEntity == "" {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "samlEntity is required")
samlConfig := SamlConfig(temp)
if err := samlConfig.validate(); err != nil {
return err
}
if temp.SamlIdp == "" {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "samlIdp is required")
*config = samlConfig
return nil
}
func (config *StorableSamlConfig) UnmarshalJSON(data []byte) error {
type Alias StorableSamlConfig
var temp Alias
if err := json.Unmarshal(data, &temp); err != nil {
return err
}
if temp.SamlCert == "" {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "samlCert is required")
samlConfig := SamlConfig(StorableSamlConfig(temp))
if err := samlConfig.validate(); err != nil {
return err
}
if temp.AttributeMapping == (AttributeMapping{}) {
if err := json.Unmarshal([]byte("{}"), &temp.AttributeMapping); err != nil {
*config = StorableSamlConfig(samlConfig)
return nil
}
// validate also assigns the default attribute mapping when none is present.
func (config *SamlConfig) validate() error {
if config.EntityID == "" {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "entityId is required")
}
if config.Location == "" {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "location is required")
}
if config.Certificate == "" {
return errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "certificate is required")
}
if config.AttributeMapping == (AttributeMapping{}) {
if err := json.Unmarshal([]byte("{}"), &config.AttributeMapping); err != nil {
return err
}
}
*config = SamlConfig(temp)
return nil
}

721
scripts/semconv/generate.go Normal file
View File

@@ -0,0 +1,721 @@
package main
import (
"bytes"
"errors"
"flag"
"fmt"
"go/format"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"gopkg.in/yaml.v3"
)
const (
kindAttribute = "attribute"
kindMetric = "metric"
)
type stringListFlag []string
func (f *stringListFlag) String() string { return strings.Join(*f, ",") }
func (f *stringListFlag) Set(value string) error {
*f = append(*f, value)
return nil
}
type schemaFile struct {
FileFormat string `yaml:"file_format"`
SchemaURL string `yaml:"schema_url"`
Versions map[string]schemaVersion `yaml:"versions"`
}
type schemaVersion struct {
All changeSection `yaml:"all"`
Resources changeSection `yaml:"resources"`
Spans changeSection `yaml:"spans"`
Logs changeSection `yaml:"logs"`
Metrics changeSection `yaml:"metrics"`
}
type changeSection struct {
Changes []schemaChange `yaml:"changes"`
}
type schemaChange struct {
RenameAttributes *attributeRename `yaml:"rename_attributes"`
RenameMetrics map[string]string `yaml:"rename_metrics"`
}
type attributeRename struct {
AttributeMap map[string]string `yaml:"attribute_map"`
ApplyToMetrics []string `yaml:"apply_to_metrics"`
}
type overlayFile struct {
DefaultEnabled bool `yaml:"default_enabled"`
// Families is keyed only by current name. One name cannot carry separate
// policies for attribute and metric families; set kind explicitly whenever
// a metric-name family is configured.
Families map[string]overlayFamily `yaml:"families"`
}
type overlayFamily struct {
Enabled *bool `yaml:"enabled"`
Kind string `yaml:"kind"`
Old []string `yaml:"old"`
AddOld []string `yaml:"add_old"`
ExcludeOld []string `yaml:"exclude_old"`
Contexts []string `yaml:"contexts"`
Signals []string `yaml:"signals"`
AddContexts []string `yaml:"add_contexts"`
AddSignals []string `yaml:"add_signals"`
ApplyToMetrics []string `yaml:"apply_to_metrics"`
AddApplyToMetrics []string `yaml:"add_apply_to_metrics"`
ValueMap map[string]string `yaml:"value_map"`
}
type edge struct {
old string
current string
kind string
contexts []string
signals []string
allContexts bool
allSignals bool
applyToMetrics []string
}
type graphKey struct{ kind, name string }
type generatedFamily struct {
Current string
Old []string
Kind string
Contexts []string
Signals []string
ApplyToMetrics []string
ValueMap map[string]string
}
func main() {
root, err := findRepoRoot()
if err != nil {
fatal(err)
}
var schemaPaths stringListFlag
flag.Var(&schemaPaths, "schema", "schema source (repeatable)")
overlayPath := flag.String("overlay", filepath.Join(root, "scripts/semconv/overlay.yaml"), "SigNoz overlay")
goOutput := flag.String("go-out", filepath.Join(root, "pkg/semconv/families_gen.go"), "generated Go output")
tsOutput := flag.String("ts-out", filepath.Join(root, "frontend/src/constants/generated/semconvFamilies.gen.ts"), "generated TypeScript output")
check := flag.Bool("check", false, "fail if generated files are stale")
flag.Parse()
if len(schemaPaths) == 0 {
schemaPaths = append(schemaPaths, filepath.Join(root, "scripts/semconv/schema-1.42.0.yaml"))
}
families, err := generate(schemaPaths, *overlayPath)
if err != nil {
fatal(err)
}
goBytes, err := renderGo(families)
if err != nil {
fatal(err)
}
tsBytes := renderTypeScript(families)
if *check {
if err := checkFile(*goOutput, goBytes); err != nil {
fatal(err)
}
if err := checkFile(*tsOutput, tsBytes); err != nil {
fatal(err)
}
return
}
if err := os.WriteFile(*goOutput, goBytes, 0o644); err != nil {
fatal(err)
}
if err := os.WriteFile(*tsOutput, tsBytes, 0o644); err != nil {
fatal(err)
}
}
func fatal(err error) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
func findRepoRoot() (string, error) {
dir, err := os.Getwd()
if err != nil {
return "", err
}
for {
if _, err := os.Stat(filepath.Join(dir, "go.mod")); err == nil {
return dir, nil
}
parent := filepath.Dir(dir)
if parent == dir {
return "", errors.New("could not find repository root")
}
dir = parent
}
}
func generate(schemaPaths []string, overlayPath string) ([]generatedFamily, error) {
var schemas []schemaFile
for _, path := range schemaPaths {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read schema %s: %w", path, err)
}
var schema schemaFile
if err := decodeKnownFields(data, &schema); err != nil {
return nil, fmt.Errorf("parse schema %s: %w", path, err)
}
schemas = append(schemas, schema)
}
overlayData, err := os.ReadFile(overlayPath)
if err != nil {
return nil, fmt.Errorf("read overlay: %w", err)
}
var overlay overlayFile
if err := decodeKnownFields(overlayData, &overlay); err != nil {
return nil, fmt.Errorf("parse overlay: %w", err)
}
return buildFamilies(schemas, overlay)
}
func decodeKnownFields(data []byte, target any) error {
decoder := yaml.NewDecoder(bytes.NewReader(data))
decoder.KnownFields(true)
return decoder.Decode(target)
}
func collectEdges(schemas []schemaFile) ([]edge, error) {
var edges []edge
for _, schema := range schemas {
versions := make([]string, 0, len(schema.Versions))
versionParts := make(map[string][3]int, len(schema.Versions))
for version := range schema.Versions {
parts, err := parseSchemaVersion(version)
if err != nil {
return nil, err
}
versions = append(versions, version)
versionParts[version] = parts
}
sort.Slice(versions, func(i, j int) bool {
return compareVersionParts(versionParts[versions[i]], versionParts[versions[j]]) < 0
})
for _, versionName := range versions {
version := schema.Versions[versionName]
var versionEdges []edge
sections := []struct {
name string
section changeSection
}{
{name: "all", section: version.All},
{name: "resources", section: version.Resources},
{name: "spans", section: version.Spans},
{name: "logs", section: version.Logs},
{name: "metrics", section: version.Metrics},
}
for _, scoped := range sections {
contexts, signals, allContexts, allSignals, err := scopeForSection(scoped.name)
if err != nil {
return nil, err
}
for _, change := range scoped.section.Changes {
if change.RenameAttributes != nil {
for _, old := range sortedMapKeys(change.RenameAttributes.AttributeMap) {
versionEdges = append(versionEdges, edge{
old: old, current: change.RenameAttributes.AttributeMap[old], kind: kindAttribute,
contexts: contexts, signals: signals,
allContexts: allContexts, allSignals: allSignals,
applyToMetrics: change.RenameAttributes.ApplyToMetrics,
})
}
}
for _, old := range sortedMapKeys(change.RenameMetrics) {
versionEdges = append(versionEdges, edge{
old: old, current: change.RenameMetrics[old], kind: kindMetric,
contexts: []string{"metric"}, signals: []string{"metrics"},
})
}
}
}
if err := rejectSameVersionChains(versionName, versionEdges); err != nil {
return nil, err
}
edges = append(edges, versionEdges...)
}
}
return edges, nil
}
func rejectSameVersionChains(version string, edges []edge) error {
oldNames := make(map[graphKey]struct{}, len(edges))
for _, item := range edges {
oldNames[graphKey{kind: item.kind, name: item.old}] = struct{}{}
}
for _, item := range edges {
if _, ok := oldNames[graphKey{kind: item.kind, name: item.current}]; ok {
return fmt.Errorf(
"schema version %q contains a same-version %s rename chain through %q",
version,
item.kind,
item.current,
)
}
}
return nil
}
func parseSchemaVersion(version string) ([3]int, error) {
parts := strings.Split(version, ".")
if len(parts) != 3 {
return [3]int{}, fmt.Errorf("schema version %q must contain major, minor, and patch numbers", version)
}
var parsed [3]int
for i, part := range parts {
value, err := strconv.Atoi(part)
if err != nil || value < 0 {
return [3]int{}, fmt.Errorf("schema version %q contains invalid numeric component %q", version, part)
}
parsed[i] = value
}
return parsed, nil
}
func compareVersionParts(left, right [3]int) int {
for i := range left {
if left[i] < right[i] {
return -1
}
if left[i] > right[i] {
return 1
}
}
return 0
}
func scopeForSection(section string) (contexts, signals []string, allContexts, allSignals bool, err error) {
switch section {
case "all":
return nil, nil, true, true, nil
case "resources":
return []string{"resource"}, nil, false, true, nil
case "spans":
return []string{"attribute"}, []string{"traces"}, false, false, nil
case "logs":
return []string{"attribute"}, []string{"logs"}, false, false, nil
case "metrics":
return []string{"attribute"}, []string{"metrics"}, false, false, nil
default:
return nil, nil, false, false, fmt.Errorf("unsupported schema section %q", section)
}
}
func buildFamilies(schemas []schemaFile, overlay overlayFile) ([]generatedFamily, error) {
edges, err := collectEdges(schemas)
if err != nil {
return nil, err
}
next := make(map[graphKey]string)
for _, item := range edges {
key := graphKey{kind: item.kind, name: item.old}
if existing, ok := next[key]; ok && existing == item.current {
// Repeated entries are common in chained schema histories. Treat an
// identical edge as a no-op so it cannot sever a later edge in the
// same chain (A -> B, B -> C, then a repeated A -> B).
continue
}
// Schema history occasionally repeats an old name with a newer direct
// destination or rolls a rename back. Edges are collected
// oldest-to-newest, so the latest published current name must be a root.
delete(next, graphKey{kind: item.kind, name: item.current})
next[key] = item.current
}
type familyState struct {
family generatedFamily
distance map[string]int
allContexts bool
allSignals bool
}
states := map[graphKey]*familyState{}
for _, item := range edges {
root, distance, err := rootFor(next, item.kind, item.old)
if err != nil {
return nil, err
}
key := graphKey{kind: item.kind, name: root}
state := states[key]
if state == nil {
state = &familyState{
family: generatedFamily{Current: root, Kind: item.kind},
distance: map[string]int{},
}
states[key] = state
}
if prior, ok := state.distance[item.old]; !ok || distance < prior {
state.distance[item.old] = distance
}
state.allContexts = state.allContexts || item.allContexts
state.allSignals = state.allSignals || item.allSignals
state.family.Contexts = appendUnique(state.family.Contexts, item.contexts...)
state.family.Signals = appendUnique(state.family.Signals, item.signals...)
state.family.ApplyToMetrics = appendUnique(state.family.ApplyToMetrics, item.applyToMetrics...)
}
for _, state := range states {
for old := range state.distance {
if old != state.family.Current {
state.family.Old = append(state.family.Old, old)
}
}
sort.Slice(state.family.Old, func(i, j int) bool {
left, right := state.family.Old[i], state.family.Old[j]
if state.distance[left] != state.distance[right] {
return state.distance[left] < state.distance[right]
}
return left < right
})
if state.allContexts {
state.family.Contexts = nil
} else {
sort.Strings(state.family.Contexts)
}
if state.allSignals {
state.family.Signals = nil
} else {
sort.Strings(state.family.Signals)
}
sort.Strings(state.family.ApplyToMetrics)
}
for _, current := range sortedMapKeys(overlay.Families) {
policy := overlay.Families[current]
kind, err := normalizedOverlayKind(current, policy)
if err != nil {
return nil, err
}
policy.Kind = kind
overlay.Families[current] = policy
key := graphKey{kind: kind, name: current}
state := states[key]
if state == nil {
if len(policy.Old) == 0 {
return nil, fmt.Errorf(
"overlay family %q with kind %q is absent from schemas and has no old members",
current,
kind,
)
}
state = &familyState{
family: generatedFamily{Current: current, Kind: kind, Old: append([]string(nil), policy.Old...)},
distance: map[string]int{},
}
states[key] = state
}
applyOverlay(&state.family, policy)
}
var result []generatedFamily
for key, state := range states {
policy, hasPolicy := overlay.Families[key.name]
enabled := overlay.DefaultEnabled
if hasPolicy && policy.Kind != key.kind {
hasPolicy = false
}
if hasPolicy && policy.Enabled != nil {
enabled = *policy.Enabled
}
if !enabled {
continue
}
if len(state.family.Old) == 0 {
return nil, fmt.Errorf(
"enabled family %q with kind %q has no old members",
state.family.Current,
state.family.Kind,
)
}
sort.Strings(state.family.Contexts)
sort.Strings(state.family.Signals)
sort.Strings(state.family.ApplyToMetrics)
result = append(result, state.family)
}
sort.Slice(result, func(i, j int) bool {
if result[i].Current != result[j].Current {
return result[i].Current < result[j].Current
}
return result[i].Kind < result[j].Kind
})
return result, nil
}
func rootFor(next map[graphKey]string, kind, name string) (string, int, error) {
seen := map[string]bool{}
distance := 0
for {
if seen[name] {
return "", 0, fmt.Errorf("rename cycle for %s %q", kind, name)
}
seen[name] = true
current, ok := next[graphKey{kind: kind, name: name}]
if !ok {
return name, distance, nil
}
name = current
distance++
}
}
func normalizedOverlayKind(current string, policy overlayFamily) (string, error) {
kind := policy.Kind
if kind == "" {
kind = kindAttribute
}
if kind != kindAttribute && kind != kindMetric {
return "", fmt.Errorf("overlay family %q has unsupported kind %q", current, kind)
}
return kind, nil
}
func applyOverlay(family *generatedFamily, policy overlayFamily) {
if policy.Kind != "" {
family.Kind = policy.Kind
}
if policy.Old != nil {
family.Old = append([]string(nil), policy.Old...)
}
family.Old = appendUnique(family.Old, policy.AddOld...)
if len(policy.ExcludeOld) > 0 {
excluded := make(map[string]bool, len(policy.ExcludeOld))
for _, old := range policy.ExcludeOld {
excluded[old] = true
}
family.Old = deleteMatching(family.Old, excluded)
}
if policy.Contexts != nil {
family.Contexts = append([]string(nil), policy.Contexts...)
}
if policy.Signals != nil {
family.Signals = append([]string(nil), policy.Signals...)
}
family.Contexts = appendUnique(family.Contexts, policy.AddContexts...)
family.Signals = appendUnique(family.Signals, policy.AddSignals...)
if policy.ApplyToMetrics != nil {
family.ApplyToMetrics = append([]string(nil), policy.ApplyToMetrics...)
}
family.ApplyToMetrics = appendUnique(family.ApplyToMetrics, policy.AddApplyToMetrics...)
if policy.ValueMap != nil {
family.ValueMap = make(map[string]string, len(policy.ValueMap))
for old, current := range policy.ValueMap {
family.ValueMap[old] = current
}
}
}
func appendUnique(values []string, additions ...string) []string {
seen := make(map[string]bool, len(values)+len(additions))
for _, value := range values {
seen[value] = true
}
for _, value := range additions {
if value == "" || seen[value] {
continue
}
seen[value] = true
values = append(values, value)
}
return values
}
func deleteMatching(values []string, excluded map[string]bool) []string {
result := values[:0]
for _, value := range values {
if !excluded[value] {
result = append(result, value)
}
}
return result
}
func renderGo(families []generatedFamily) ([]byte, error) {
var out bytes.Buffer
out.WriteString("// Code generated by scripts/semconv. DO NOT EDIT.\n\n")
out.WriteString("package semconv\n\n")
needsTelemetryTypes := false
for _, family := range families {
if len(family.Contexts) > 0 || len(family.Signals) > 0 {
needsTelemetryTypes = true
break
}
}
if needsTelemetryTypes {
out.WriteString("import \"github.com/SigNoz/signoz/pkg/types/telemetrytypes\"\n\n")
}
out.WriteString("var families = []Family{\n")
for _, family := range families {
contexts, err := goFieldContextSlice(family.Contexts)
if err != nil {
return nil, fmt.Errorf("render family %q: %w", family.Current, err)
}
signals, err := goSignalSlice(family.Signals)
if err != nil {
return nil, fmt.Errorf("render family %q: %w", family.Current, err)
}
out.WriteString("\t{\n")
fmt.Fprintf(&out, "\t\tCurrent: %s,\n", strconv.Quote(family.Current))
fmt.Fprintf(&out, "\t\tOld: %s,\n", goStringSlice(family.Old))
if family.Kind == kindMetric {
out.WriteString("\t\tKind: KindMetric,\n")
} else {
out.WriteString("\t\tKind: KindAttribute,\n")
}
fmt.Fprintf(&out, "\t\tContexts: %s,\n", contexts)
fmt.Fprintf(&out, "\t\tSignals: %s,\n", signals)
fmt.Fprintf(&out, "\t\tApplyToMetrics: %s,\n", goStringSlice(family.ApplyToMetrics))
if len(family.ValueMap) > 0 {
out.WriteString("\t\tValueMap: map[string]string{\n")
keys := sortedMapKeys(family.ValueMap)
for _, key := range keys {
fmt.Fprintf(&out, "\t\t\t%s: %s,\n", strconv.Quote(key), strconv.Quote(family.ValueMap[key]))
}
out.WriteString("\t\t},\n")
}
out.WriteString("\t},\n")
}
out.WriteString("}\n")
return format.Source(out.Bytes())
}
func goStringSlice(values []string) string {
if len(values) == 0 {
return "nil"
}
quoted := make([]string, len(values))
for i, value := range values {
quoted[i] = strconv.Quote(value)
}
return "[]string{" + strings.Join(quoted, ", ") + "}"
}
func goFieldContextSlice(values []string) (string, error) {
if len(values) == 0 {
return "nil", nil
}
constants := make([]string, len(values))
for i, value := range values {
switch value {
case "metric":
constants[i] = "telemetrytypes.FieldContextMetric"
case "resource":
constants[i] = "telemetrytypes.FieldContextResource"
case "attribute":
constants[i] = "telemetrytypes.FieldContextAttribute"
default:
return "", fmt.Errorf("unsupported field context %q", value)
}
}
return "[]telemetrytypes.FieldContext{" + strings.Join(constants, ", ") + "}", nil
}
func goSignalSlice(values []string) (string, error) {
if len(values) == 0 {
return "nil", nil
}
constants := make([]string, len(values))
for i, value := range values {
switch value {
case "traces":
constants[i] = "telemetrytypes.SignalTraces"
case "logs":
constants[i] = "telemetrytypes.SignalLogs"
case "metrics":
constants[i] = "telemetrytypes.SignalMetrics"
default:
return "", fmt.Errorf("unsupported signal %q", value)
}
}
return "[]telemetrytypes.Signal{" + strings.Join(constants, ", ") + "}", nil
}
func renderTypeScript(families []generatedFamily) []byte {
var out bytes.Buffer
out.WriteString("// Code generated by scripts/semconv. DO NOT EDIT.\n\n")
out.WriteString("export type SemconvFamily = {\n")
out.WriteString("\treadonly current: string;\n\treadonly old: readonly string[];\n")
out.WriteString("\treadonly kind: 'attribute' | 'metric';\n")
out.WriteString("\treadonly contexts: readonly string[];\n\treadonly signals: readonly string[];\n")
out.WriteString("\treadonly applyToMetrics: readonly string[];\n")
out.WriteString("\treadonly valueMap: Readonly<Record<string, string>>;\n};\n\n")
out.WriteString("export const SEMCONV_FAMILIES: readonly SemconvFamily[] = [\n")
for _, family := range families {
out.WriteString("\t{\n")
fmt.Fprintf(&out, "\t\tcurrent: %s,\n", tsString(family.Current))
fmt.Fprintf(&out, "\t\told: %s,\n", tsStringSlice(family.Old))
fmt.Fprintf(&out, "\t\tkind: %s,\n", tsString(family.Kind))
fmt.Fprintf(&out, "\t\tcontexts: %s,\n", tsStringSlice(family.Contexts))
fmt.Fprintf(&out, "\t\tsignals: %s,\n", tsStringSlice(family.Signals))
fmt.Fprintf(&out, "\t\tapplyToMetrics: %s,\n", tsStringSlice(family.ApplyToMetrics))
out.WriteString("\t\tvalueMap: {")
keys := sortedMapKeys(family.ValueMap)
for i, key := range keys {
if i > 0 {
out.WriteString(", ")
}
fmt.Fprintf(&out, "%s: %s", tsString(key), tsString(family.ValueMap[key]))
}
out.WriteString("},\n\t},\n")
}
out.WriteString("] as const;\n")
return out.Bytes()
}
func tsString(value string) string {
quoted := strconv.Quote(value)
return "'" + strings.ReplaceAll(quoted[1:len(quoted)-1], "'", `\'`) + "'"
}
func tsStringSlice(values []string) string {
quoted := make([]string, len(values))
for i, value := range values {
quoted[i] = tsString(value)
}
return "[" + strings.Join(quoted, ", ") + "]"
}
func sortedMapKeys[T any](values map[string]T) []string {
keys := make([]string, 0, len(values))
for key := range values {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
func checkFile(path string, expected []byte) error {
actual, err := os.ReadFile(path)
if err != nil {
return fmt.Errorf("generated file %s is missing: run go run ./scripts/semconv", path)
}
if !bytes.Equal(actual, expected) {
return fmt.Errorf("generated file %s is stale: run go run ./scripts/semconv", path)
}
return nil
}

View File

@@ -0,0 +1,376 @@
package main
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSchemaDecoderRejectsUnsupportedSection(t *testing.T) {
var schema schemaFile
err := decodeKnownFields([]byte(`
versions:
1.0.0:
span_events:
changes:
- rename_events:
event_map:
old: current
`), &schema)
assert.ErrorContains(t, err, "field span_events not found", "unsupported schema sections must fail generation")
}
func TestBuildFamiliesRejectsMalformedSchemaVersion(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
latest:
spans:
changes: []
`), &schema), "test schema must decode")
_, err := buildFamilies([]schemaFile{schema}, overlayFile{})
assert.ErrorContains(t, err, `schema version "latest"`, "malformed versions must not be silently reordered")
}
func TestBuildFamiliesResolvesRenameChain(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
4.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
a: b
3.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
b: c
x: c
2.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
a: b
- rename_attributes:
attribute_map:
a: b
`), &schema), "test schema must decode")
enabled := true
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
"c": {Enabled: &enabled},
}})
require.NoError(t, err)
assert.Equal(t, []generatedFamily{{
Current: "c",
Old: []string{"b", "x", "a"},
Kind: kindAttribute,
Contexts: []string{"attribute"},
Signals: []string{"traces"},
}}, families, "predecessors should be ordered by distance and then name")
}
func TestBuildFamiliesMapsSchemaSectionsToScopes(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
1.0.0:
all:
changes:
- rename_attributes:
attribute_map:
all.old: all.current
resources:
changes:
- rename_attributes:
attribute_map:
resource.old: resource.current
logs:
changes:
- rename_attributes:
attribute_map:
log.old: log.current
metrics:
changes:
- rename_attributes:
attribute_map:
state: cpu.mode
apply_to_metrics: [system.cpu.time]
- rename_metrics:
old.metric: current.metric
`), &schema), "test schema must decode")
enabled := true
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
"all.current": {Enabled: &enabled},
"resource.current": {Enabled: &enabled},
"log.current": {Enabled: &enabled},
"cpu.mode": {Enabled: &enabled},
"current.metric": {Enabled: &enabled, Kind: kindMetric},
}})
require.NoError(t, err)
assert.Equal(t, []generatedFamily{
{
Current: "all.current", Old: []string{"all.old"}, Kind: kindAttribute,
Contexts: nil, Signals: nil,
},
{
Current: "cpu.mode", Old: []string{"state"}, Kind: kindAttribute,
Contexts: []string{"attribute"}, Signals: []string{"metrics"},
ApplyToMetrics: []string{"system.cpu.time"},
},
{
Current: "current.metric", Old: []string{"old.metric"}, Kind: kindMetric,
Contexts: []string{"metric"}, Signals: []string{"metrics"},
},
{
Current: "log.current", Old: []string{"log.old"}, Kind: kindAttribute,
Contexts: []string{"attribute"}, Signals: []string{"logs"},
},
{
Current: "resource.current", Old: []string{"resource.old"}, Kind: kindAttribute,
Contexts: []string{"resource"},
},
}, families, "schema sections should produce their documented signal and context scopes")
}
func TestOverlayAddsFamilyWithoutSchemaHistory(t *testing.T) {
enabled := true
families, err := buildFamilies(nil, overlayFile{Families: map[string]overlayFamily{
"added.current": {
Enabled: &enabled,
Old: []string{"added.old"},
Contexts: []string{"resource"},
Signals: []string{"traces"},
},
}})
require.NoError(t, err)
assert.Equal(t, []generatedFamily{{
Current: "added.current",
Old: []string{"added.old"},
Kind: kindAttribute,
Contexts: []string{"resource"},
Signals: []string{"traces"},
}}, families, "an explicit overlay family should not require schema history")
}
func TestOverlayOverridesGeneratedFamily(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
1.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
old: current
`), &schema), "test schema must decode")
enabled := true
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
"current": {
Enabled: &enabled,
AddOld: []string{"older"},
ExcludeOld: []string{"old"},
AddContexts: []string{"resource"},
AddSignals: []string{"logs"},
ValueMap: map[string]string{"legacy": "current"},
},
}})
require.NoError(t, err)
assert.Equal(t, []generatedFamily{{
Current: "current",
Old: []string{"older"},
Kind: kindAttribute,
Contexts: []string{"attribute", "resource"},
Signals: []string{"logs", "traces"},
ValueMap: map[string]string{"legacy": "current"},
}}, families, "overlay additions and exclusions should be applied to the generated family")
}
func TestOverlayDisablesFamilyWhenDefaultIsEnabled(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
1.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
old: current
`), &schema), "test schema must decode")
disabled := false
families, err := buildFamilies([]schemaFile{schema}, overlayFile{
DefaultEnabled: true,
Families: map[string]overlayFamily{
"current": {Enabled: &disabled},
},
})
require.NoError(t, err)
assert.Empty(t, families, "an explicitly disabled family must override default_enabled")
}
func TestRenderGoIsDeterministic(t *testing.T) {
families := []generatedFamily{{
Current: "current", Old: []string{"old"}, Kind: kindAttribute,
ValueMap: map[string]string{"b": "2", "a": "1"},
}}
first, err := renderGo(families)
require.NoError(t, err)
second, err := renderGo(families)
require.NoError(t, err)
assert.Equal(t, first, second, "Go generation must not depend on map iteration order")
}
func TestRenderGoUsesCanonicalTelemetryTypes(t *testing.T) {
families := []generatedFamily{{
Current: "current", Old: []string{"old"}, Kind: kindAttribute,
Contexts: []string{"resource"}, Signals: []string{"traces"},
}}
output, err := renderGo(families)
require.NoError(t, err)
assert.Contains(t, string(output), "telemetrytypes.FieldContextResource", "generated contexts should use telemetrytypes")
assert.Contains(t, string(output), "telemetrytypes.SignalTraces", "generated signals should use telemetrytypes")
}
func TestRenderTypeScriptIsDeterministic(t *testing.T) {
families := []generatedFamily{{
Current: "current", Old: []string{"old"}, Kind: kindAttribute,
ValueMap: map[string]string{"b": "2", "a": "1"},
}}
assert.Equal(t, renderTypeScript(families), renderTypeScript(families), "TypeScript generation must not depend on map iteration order")
}
func TestBuildFamiliesHandlesRenameRollback(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
2.0.0:
metrics:
changes:
- rename_metrics:
temporary: original
1.0.0:
metrics:
changes:
- rename_metrics:
original: temporary
`), &schema), "test schema must decode")
enabled := true
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
"original": {Enabled: &enabled, Kind: kindMetric},
}})
require.NoError(t, err)
assert.Equal(t, []generatedFamily{{
Current: "original",
Old: []string{"temporary"},
Kind: kindMetric,
Contexts: []string{"metric"},
Signals: []string{"metrics"},
}}, families, "the latest rollback destination should remain the family root")
}
func TestBuildFamiliesRejectsSameVersionRenameChain(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
1.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
x: y
y: z
`), &schema), "test schema must decode")
_, err := buildFamilies([]schemaFile{schema}, overlayFile{})
assert.ErrorContains(t, err, `same-version attribute rename chain through "y"`, "order-sensitive same-version chains must be rejected")
}
func TestBuildFamiliesRejectsOverlayFamilyWithoutHistory(t *testing.T) {
enabled := true
_, err := buildFamilies(nil, overlayFile{Families: map[string]overlayFamily{
"missing": {Enabled: &enabled},
}})
assert.ErrorContains(t, err, `overlay family "missing" with kind "attribute" is absent`, "an overlay cannot invent a family without old members")
}
func TestBuildFamiliesRejectsEnabledFamilyWithoutOldMembers(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
1.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
old: current
`), &schema), "test schema must decode")
enabled := true
_, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
"current": {Enabled: &enabled, ExcludeOld: []string{"old"}},
}})
assert.ErrorContains(t, err, `enabled family "current" with kind "attribute" has no old members`, "exclude_old cannot empty an enabled family")
}
func TestOverlayKindDefaultsToAttribute(t *testing.T) {
var schema schemaFile
require.NoError(t, decodeKnownFields([]byte(`
versions:
1.0.0:
spans:
changes:
- rename_attributes:
attribute_map:
attribute.old: shared.current
metrics:
changes:
- rename_metrics:
metric.old: shared.current
`), &schema), "test schema must decode")
enabled := true
families, err := buildFamilies([]schemaFile{schema}, overlayFile{Families: map[string]overlayFamily{
"shared.current": {Enabled: &enabled},
}})
require.NoError(t, err)
assert.Equal(t, []generatedFamily{{
Current: "shared.current",
Old: []string{"attribute.old"},
Kind: kindAttribute,
Contexts: []string{"attribute"},
Signals: []string{"traces"},
}}, families, "a kind-less overlay policy should affect only the attribute family")
}
func TestCheckFileReportsStaleOutput(t *testing.T) {
path := filepath.Join(t.TempDir(), "generated.go")
require.NoError(t, os.WriteFile(path, []byte("old"), 0o600), "test output must be writable")
assert.ErrorContains(t, checkFile(path, []byte("new")), "is stale", "check mode must reject stale generated output")
}
func TestTypeScriptStringEscapesControlCharacters(t *testing.T) {
assert.Equal(t, `'line\n\t\x01\'\\end'`, tsString("line\n\t\x01'\\end"), "generated TypeScript strings must remain valid literals")
}

View File

@@ -0,0 +1,11 @@
# SigNoz semantic-convention rollout policy.
#
# Families are keyed by their current OpenTelemetry name. Schema-derived
# families are disabled by default so rollout remains explicit and reversible.
default_enabled: false
families:
deployment.environment.name:
enabled: true
db.system.name:
enabled: true

View File

@@ -0,0 +1,760 @@
file_format: 1.1.0
schema_url: https://opentelemetry.io/schemas/1.42.0
versions:
1.42.0:
metrics:
changes:
- rename_metrics:
v8js.memory.heap.limit: v8js.memory.heap.space.size
1.41.1:
1.41.0:
metrics:
changes:
- rename_metrics:
k8s.container.cpu.limit: k8s.container.cpu.limit.desired
k8s.container.cpu.limit_utilization: k8s.container.cpu.limit.utilization
k8s.container.cpu.request: k8s.container.cpu.request.desired
k8s.container.cpu.request_utilization: k8s.container.cpu.request.utilization
k8s.container.memory.limit: k8s.container.memory.limit.desired
k8s.container.memory.request: k8s.container.memory.request.desired
1.40.0:
all:
changes:
- rename_attributes:
attribute_map:
feature_flag.evaluation.error.message: feature_flag.error.message
metrics:
changes:
- rename_metrics:
system.memory.shared: system.memory.linux.shared
1.39.0:
all:
changes:
- rename_attributes:
attribute_map:
linux.memory.slab.state: system.memory.linux.slab.state
peer.service: service.peer.name
rpc.connect_rpc.error_code: rpc.response.status_code
rpc.connect_rpc.request.metadata: rpc.request.metadata
rpc.connect_rpc.response.metadata: rpc.response.metadata
rpc.grpc.request.metadata: rpc.request.metadata
rpc.grpc.response.metadata: rpc.response.metadata
rpc.jsonrpc.request_id: jsonrpc.request.id
rpc.jsonrpc.version: jsonrpc.protocol.version
rpc.system: rpc.system.name
metrics:
changes:
- rename_metrics:
process.open_file_descriptor.count: process.unix.file_descriptor.count
system.linux.memory.available: system.memory.linux.available
system.linux.memory.slab.usage: system.memory.linux.slab.usage
1.38.0:
all:
changes:
- rename_attributes:
attribute_map:
process.context_switch_type: process.context_switch.type
process.paging.fault_type: system.paging.fault.type
system.cpu.logical_number: cpu.logical_number
system.paging.type: system.paging.fault.type
system.process.status: process.state
system.processes.status: process.state
metrics:
changes:
- rename_metrics:
k8s.cronjob.active_jobs: k8s.cronjob.job.active
k8s.daemonset.current_scheduled_nodes: k8s.daemonset.node.current_scheduled
k8s.daemonset.desired_scheduled_nodes: k8s.daemonset.node.desired_scheduled
k8s.daemonset.misscheduled_nodes: k8s.daemonset.node.misscheduled
k8s.daemonset.ready_nodes: k8s.daemonset.node.ready
k8s.deployment.available_pods: k8s.deployment.pod.available
k8s.deployment.desired_pods: k8s.deployment.pod.desired
k8s.hpa.current_pods: k8s.hpa.pod.current
k8s.hpa.desired_pods: k8s.hpa.pod.desired
k8s.hpa.max_pods: k8s.hpa.pod.max
k8s.hpa.min_pods: k8s.hpa.pod.min
k8s.job.active_pods: k8s.job.pod.active
k8s.job.desired_successful_pods: k8s.job.pod.desired_successful
k8s.job.failed_pods: k8s.job.pod.failed
k8s.job.max_parallel_pods: k8s.job.pod.max_parallel
k8s.job.successful_pods: k8s.job.pod.successful
k8s.node.allocatable.cpu: k8s.node.cpu.allocatable
k8s.node.allocatable.ephemeral_storage: k8s.node.ephemeral_storage.allocatable
k8s.node.allocatable.memory: k8s.node.memory.allocatable
k8s.node.allocatable.pods: k8s.node.pod.allocatable
k8s.replicaset.available_pods: k8s.replicaset.pod.available
k8s.replicaset.desired_pods: k8s.replicaset.pod.desired
k8s.replication_controller.available_pods: k8s.replicationcontroller.pod.available
k8s.replication_controller.desired_pods: k8s.replicationcontroller.pod.desired
k8s.replicationcontroller.available_pods: k8s.replicationcontroller.pod.available
k8s.replicationcontroller.desired_pods: k8s.replicationcontroller.pod.desired
k8s.statefulset.current_pods: k8s.statefulset.pod.current
k8s.statefulset.desired_pods: k8s.statefulset.pod.desired
k8s.statefulset.ready_pods: k8s.statefulset.pod.ready
k8s.statefulset.updated_pods: k8s.statefulset.pod.updated
v8js.heap.space.available_size: v8js.memory.heap.space.available_size
v8js.heap.space.physical_size: v8js.memory.heap.space.physical_size
1.37.0:
all:
changes:
- rename_attributes:
attribute_map:
android.state: android.app.state
container.runtime: container.runtime.name
enduser.role: user.roles
gen_ai.openai.request.service_tier: openai.request.service_tier
gen_ai.openai.response.service_tier: openai.response.service_tier
gen_ai.openai.response.system_fingerprint: openai.response.system_fingerprint
gen_ai.system: gen_ai.provider.name
ios.state: ios.app.state
1.36.0:
1.35.0:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1698
- rename_attributes:
attribute_map:
az.namespace: azure.resource_provider.namespace
az.service_request_id: azure.service.request.id
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/issues/1800
- rename_metrics:
system.network.connections: system.network.connection.count
1.34.0:
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/2295
- rename_metrics:
cpu.time: system.cpu.time
cpu.utilization: system.cpu.utilization
cpu.frequency: system.cpu.frequency
1.33.0:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1982
- rename_attributes:
attribute_map:
feature_flag.provider_name: feature_flag.provider.name
# https://github.com/open-telemetry/semantic-conventions/pull/1994
- rename_attributes:
attribute_map:
feature_flag.evaluation.error.message: error.message
1.32.0:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1989
- rename_attributes:
attribute_map:
feature_flag.evaluation.reason: feature_flag.result.reason
feature_flag.variant: feature_flag.result.variant
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/2042
- rename_metrics:
otel.sdk.span.live.count: otel.sdk.span.live
otel.sdk.span.ended.count: otel.sdk.span.ended
otel.sdk.processor.span.processed.count: otel.sdk.processor.span.processed
otel.sdk.exporter.span.inflight.count: otel.sdk.exporter.span.inflight
otel.sdk.exporter.span.exported.count: otel.sdk.exporter.span.exported
1.31.0:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1880
- rename_attributes:
attribute_map:
android.state: android.app.state
io.state: ios.app.state
metrics:
changes:
- rename_metrics:
k8s.replication_controller.desired_pods: k8s.replicationcontroller.desired_pods
k8s.replication_controller.available_pods: k8s.replicationcontroller.available_pods
# https://github.com/open-telemetry/semantic-conventions/pull/1896
- rename_metrics:
system.cpu.time: cpu.time
system.cpu.utilization: cpu.utilization
system.cpu.frequency: cpu.frequency
# https://github.com/open-telemetry/semantic-conventions/pull/1896
- rename_attributes:
attribute_map:
system.cpu.logical_number: cpu.logical_number
1.30.0:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1632
- rename_attributes:
attribute_map:
gen_ai.openai.request.seed: gen_ai.request.seed
system.network.state: network.connection.state
# https://github.com/open-telemetry/semantic-conventions/pull/1624
- rename_attributes:
attribute_map:
code.function: code.function.name
code.filepath: code.file.path
code.lineno: code.line.number
code.column: code.column.number
# https://github.com/open-telemetry/semantic-conventions/pull/1734
- rename_attributes:
attribute_map:
db.system: db.system.name
db.cassandra.coordinator.dc: cassandra.coordinator.dc
db.cassandra.coordinator.id: cassandra.coordinator.id
db.cassandra.consistency_level: cassandra.consistency.level
db.cassandra.idempotence: cassandra.query.idempotent
db.cassandra.page_size: cassandra.page.size
db.cassandra.speculative_execution_count: cassandra.speculative_execution.count
db.cosmosdb.client_id: azure.client.id
db.cosmosdb.connection_mode: azure.cosmosdb.connection.mode
db.cosmosdb.consistency_level: azure.cosmosdb.consistency.level
db.cosmosdb.request_charge: azure.cosmosdb.operation.request_charge
db.cosmosdb.request_content_length: azure.cosmosdb.request.body.size
db.cosmosdb.regions_contacted: azure.cosmosdb.operation.contacted_regions
db.cosmosdb.sub_status_code: azure.cosmosdb.response.sub_status_code
db.elasticsearch.node.name: elasticsearch.node.name
# db.elasticsearch.path_parts is a template attribute, schema transformation
# does not support it, adding as a comment for consistency
# db.elasticsearch.path_parts.<key> -> db.operation.parameter.<key>
metrics:
changes:
- rename_metrics:
db.client.cosmosdb.operation.request_charge: azure.cosmosdb.client.operation.request_charge
db.client.cosmosdb.active_instance.count: azure.cosmosdb.client.active_instance.count
1.29.0:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1520
- rename_attributes:
attribute_map:
process.executable.build_id.profiling: process.executable.build_id.htlhash
# https://github.com/open-telemetry/semantic-conventions/pull/1383
- rename_attributes:
attribute_map:
vcs.repository.change.id: vcs.change.id
vcs.repository.change.title: vcs.change.title
vcs.repository.ref.name: vcs.ref.head.name
vcs.repository.ref.revision: vcs.ref.head.revision
vcs.repository.ref.type: vcs.ref.head.type
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1492
- rename_attributes:
attribute_map:
system.device: network.interface.name
apply_to_metrics:
- container.network.io
- system.network.dropped
- system.network.errors
- system.network.io
- system.network.connections
1.28.0:
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1422
- rename_metrics:
messaging.client.published.messages: messaging.client.sent.messages
1.27.0:
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1216
- rename_attributes:
attribute_map:
tls.client.server_name: server.address
# https://github.com/open-telemetry/semantic-conventions/pull/1075
- rename_attributes:
attribute_map:
deployment.environment: deployment.environment.name
# https://github.com/open-telemetry/semantic-conventions/pull/1245
- rename_attributes:
attribute_map:
messaging.kafka.message.offset: messaging.kafka.offset
# https://github.com/open-telemetry/semantic-conventions/pull/815
- rename_attributes:
attribute_map:
messaging.kafka.consumer.group: messaging.consumer.group.name
messaging.rocketmq.client_group: messaging.consumer.group.name
messaging.eventhubs.consumer.group: messaging.consumer.group.name
messaging.servicebus.destination.subscription_name: messaging.destination.subscription.name
# https://github.com/open-telemetry/semantic-conventions/pull/1200
- rename_attributes:
attribute_map:
gen_ai.usage.completion_tokens: gen_ai.usage.output_tokens
gen_ai.usage.prompt_tokens: gen_ai.usage.input_tokens
spans:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1002
- rename_attributes:
attribute_map:
db.elasticsearch.cluster.name: db.namespace
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/1125
- rename_attributes:
attribute_map:
db.client.connections.state: db.client.connection.state
apply_to_metrics:
- db.client.connection.count
- rename_attributes:
attribute_map:
db.client.connections.pool.name: db.client.connection.pool.name
apply_to_metrics:
- db.client.connection.count
- db.client.connection.idle.max
- db.client.connection.idle.min
- db.client.connection.max
- db.client.connection.pending_requests
- db.client.connection.timeouts
- db.client.connection.create_time
- db.client.connection.wait_time
- db.client.connection.use_time
# https://github.com/open-telemetry/semantic-conventions/pull/1006
- rename_metrics:
messaging.publish.messages: messaging.client.published.messages
# https://github.com/open-telemetry/semantic-conventions/pull/1026
- rename_attributes:
attribute_map:
system.cpu.state: cpu.mode
process.cpu.state: cpu.mode
container.cpu.state: cpu.mode
apply_to_metrics:
- system.cpu.time
- system.cpu.utilization
- process.cpu.time
- process.cpu.utilization
- container.cpu.time
# https://github.com/open-telemetry/semantic-conventions/pull/1265
- rename_metrics:
jvm.buffer.memory.usage: jvm.buffer.memory.used
1.26.0:
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/966
- rename_metrics:
db.client.connections.usage: db.client.connection.count
db.client.connections.idle.max: db.client.connection.idle.max
db.client.connections.idle.min: db.client.connection.idle.min
db.client.connections.max: db.client.connection.max
db.client.connections.pending_requests: db.client.connection.pending_requests
db.client.connections.timeouts: db.client.connection.timeouts
# https://github.com/open-telemetry/semantic-conventions/pull/948
- rename_attributes:
attribute_map:
messaging.client_id: messaging.client.id
# https://github.com/open-telemetry/semantic-conventions/pull/909
- rename_attributes:
attribute_map:
state: db.client.connections.state
apply_to_metrics:
- db.client.connections.usage
- rename_attributes:
attribute_map:
pool.name: db.client.connections.pool.name
apply_to_metrics:
- db.client.connections.usage
- db.client.connections.idle.max
- db.client.connections.idle.min
- db.client.connections.max
- db.client.connections.pending_requests
- db.client.connections.timeouts
- db.client.connections.create_time
- db.client.connections.wait_time
- db.client.connections.use_time
all:
changes:
# https://github:com/open-telemetry/semantic-conventions/pull/731/
- rename_attributes:
attribute_map:
enduser.id: user.id
1.25.0:
spans:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/911
- rename_attributes:
attribute_map:
db.name: db.namespace
# https://github.com/open-telemetry/semantic-conventions/pull/870
- rename_attributes:
attribute_map:
db.sql.table: db.collection.name
db.mongodb.collection: db.collection.name
db.cosmosdb.container: db.collection.name
db.cassandra.table: db.collection.name
# https://github.com/open-telemetry/semantic-conventions/pull/798
- rename_attributes:
attribute_map:
messaging.kafka.destination.partition: messaging.destination.partition.id
# https://github.com/open-telemetry/semantic-conventions/pull/875
- rename_attributes:
attribute_map:
db.operation: db.operation.name
# https://github.com/open-telemetry/semantic-conventions/pull/913
- rename_attributes:
attribute_map:
messaging.operation: messaging.operation.type
# https://github.com/open-telemetry/semantic-conventions/pull/866
- rename_attributes:
attribute_map:
db.statement: db.query.text
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/484
- rename_attributes:
attribute_map:
system.processes.status: system.process.status
apply_to_metrics:
- system.processes.count
- rename_metrics:
system.processes.count: system.process.count
system.processes.created: system.process.created
# https://github.com/open-telemetry/semantic-conventions/pull/625
- rename_attributes:
attribute_map:
container.labels: container.label
k8s.pod.labels: k8s.pod.label
# https://github.com/open-telemetry/semantic-conventions/pull/330
- rename_metrics:
process.threads: process.thread.count
process.open_file_descriptors: process.open_file_descriptor.count
- rename_attributes:
attribute_map:
state: process.cpu.state
apply_to_metrics:
- process.cpu.time
- process.cpu.utilization
- rename_attributes:
attribute_map:
direction: disk.io.direction
apply_to_metrics:
- process.disk.io
- rename_attributes:
attribute_map:
type: process.context_switch_type
apply_to_metrics:
- process.context_switches
- rename_attributes:
attribute_map:
direction: network.io.direction
apply_to_metrics:
- process.network.io
- rename_attributes:
attribute_map:
type: process.paging.fault_type
apply_to_metrics:
- process.paging.faults
all:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/854
- rename_attributes:
attribute_map:
message.type: rpc.message.type
message.id: rpc.message.id
message.compressed_size: rpc.message.compressed_size
message.uncompressed_size: rpc.message.uncompressed_size
1.24.0:
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/536
- rename_metrics:
jvm.memory.usage: jvm.memory.used
jvm.memory.usage_after_last_gc: jvm.memory.used_after_last_gc
# https://github.com/open-telemetry/semantic-conventions/pull/530
- rename_attributes:
attribute_map:
system.network.io.direction: network.io.direction
system.disk.io.direction: disk.io.direction
1.23.1:
1.23.0:
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/20
- rename_attributes:
attribute_map:
thread.daemon: jvm.thread.daemon
apply_to_metrics:
- jvm.thread.count
1.22.0:
spans:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/229
- rename_attributes:
attribute_map:
messaging.message.payload_size_bytes: messaging.message.body.size
# https://github.com/open-telemetry/opentelemetry-specification/pull/374
- rename_attributes:
attribute_map:
http.resend_count: http.request.resend_count
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/224
- rename_metrics:
http.client.duration: http.client.request.duration
http.server.duration: http.server.request.duration
# https://github.com/open-telemetry/semantic-conventions/pull/241
- rename_metrics:
process.runtime.jvm.memory.usage: jvm.memory.usage
process.runtime.jvm.memory.committed: jvm.memory.committed
process.runtime.jvm.memory.limit: jvm.memory.limit
process.runtime.jvm.memory.usage_after_last_gc: jvm.memory.usage_after_last_gc
process.runtime.jvm.gc.duration: jvm.gc.duration
# also https://github.com/open-telemetry/semantic-conventions/pull/252
process.runtime.jvm.threads.count: jvm.thread.count
# also https://github.com/open-telemetry/semantic-conventions/pull/252
process.runtime.jvm.classes.loaded: jvm.class.loaded
# also https://github.com/open-telemetry/semantic-conventions/pull/252
process.runtime.jvm.classes.unloaded: jvm.class.unloaded
# also https://github.com/open-telemetry/semantic-conventions/pull/252
# and https://github.com/open-telemetry/semantic-conventions/pull/60
process.runtime.jvm.classes.current_loaded: jvm.class.count
process.runtime.jvm.cpu.time: jvm.cpu.time
process.runtime.jvm.cpu.recent_utilization: jvm.cpu.recent_utilization
process.runtime.jvm.memory.init: jvm.memory.init
process.runtime.jvm.system.cpu.utilization: jvm.system.cpu.utilization
process.runtime.jvm.system.cpu.load_1m: jvm.system.cpu.load_1m
# https://github.com/open-telemetry/semantic-conventions/pull/253
process.runtime.jvm.buffer.usage: jvm.buffer.memory.usage
# https://github.com/open-telemetry/semantic-conventions/pull/253
process.runtime.jvm.buffer.limit: jvm.buffer.memory.limit
process.runtime.jvm.buffer.count: jvm.buffer.count
# https://github.com/open-telemetry/semantic-conventions/pull/20
- rename_attributes:
attribute_map:
type: jvm.memory.type
pool: jvm.memory.pool.name
apply_to_metrics:
- jvm.memory.usage
- jvm.memory.committed
- jvm.memory.limit
- jvm.memory.usage_after_last_gc
- jvm.memory.init
- rename_attributes:
attribute_map:
name: jvm.gc.name
action: jvm.gc.action
apply_to_metrics:
- jvm.gc.duration
- rename_attributes:
attribute_map:
daemon: thread.daemon
apply_to_metrics:
- jvm.threads.count
- rename_attributes:
attribute_map:
pool: jvm.buffer.pool.name
apply_to_metrics:
- jvm.buffer.memory.usage
- jvm.buffer.memory.limit
- jvm.buffer.count
# https://github.com/open-telemetry/semantic-conventions/pull/89
- rename_attributes:
attribute_map:
state: system.cpu.state
cpu: system.cpu.logical_number
apply_to_metrics:
- system.cpu.time
- system.cpu.utilization
- rename_attributes:
attribute_map:
state: system.memory.state
apply_to_metrics:
- system.memory.usage
- system.memory.utilization
- rename_attributes:
attribute_map:
state: system.paging.state
apply_to_metrics:
- system.paging.usage
- system.paging.utilization
- rename_attributes:
attribute_map:
type: system.paging.type
direction: system.paging.direction
apply_to_metrics:
- system.paging.faults
- system.paging.operations
- rename_attributes:
attribute_map:
device: system.device
direction: system.disk.direction
apply_to_metrics:
- system.disk.io
- system.disk.operations
- system.disk.io_time
- system.disk.operation_time
- system.disk.merged
- rename_attributes:
attribute_map:
device: system.device
state: system.filesystem.state
type: system.filesystem.type
mode: system.filesystem.mode
mountpoint: system.filesystem.mountpoint
apply_to_metrics:
- system.filesystem.usage
- system.filesystem.utilization
- rename_attributes:
attribute_map:
device: system.device
direction: system.network.direction
protocol: network.protocol
state: system.network.state
apply_to_metrics:
- system.network.dropped
- system.network.packets
- system.network.errors
- system.network.io
- system.network.connections
- rename_attributes:
attribute_map:
status: system.processes.status
apply_to_metrics:
- system.processes.count
# https://github.com/open-telemetry/semantic-conventions/pull/247
- rename_metrics:
http.server.request.size: http.server.request.body.size
http.server.response.size: http.server.response.body.size
resources:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/178
- rename_attributes:
attribute_map:
telemetry.auto.version: telemetry.distro.version
1.21.0:
spans:
changes:
# https://github.com/open-telemetry/opentelemetry-specification/pull/3336
- rename_attributes:
attribute_map:
messaging.kafka.client_id: messaging.client_id
messaging.rocketmq.client_id: messaging.client_id
# https://github.com/open-telemetry/opentelemetry-specification/pull/3402
- rename_attributes:
attribute_map:
# net.peer.(name|port) attributes were usually populated on client side
# so they should be usually translated to server.(address|port)
# net.host.* attributes were only populated on server side
net.host.name: server.address
net.host.port: server.port
# was only populated on client side
net.sock.peer.name: server.socket.domain
# net.sock.peer.(addr|port) mapping is not possible
# since they applied to both client and server side
# were only populated on server side
net.sock.host.addr: server.socket.address
net.sock.host.port: server.socket.port
http.client_ip: client.address
# https://github.com/open-telemetry/opentelemetry-specification/pull/3426
- rename_attributes:
attribute_map:
net.protocol.name: network.protocol.name
net.protocol.version: network.protocol.version
net.host.connection.type: network.connection.type
net.host.connection.subtype: network.connection.subtype
net.host.carrier.name: network.carrier.name
net.host.carrier.mcc: network.carrier.mcc
net.host.carrier.mnc: network.carrier.mnc
net.host.carrier.icc: network.carrier.icc
# https://github.com/open-telemetry/opentelemetry-specification/pull/3355
- rename_attributes:
attribute_map:
http.method: http.request.method
http.status_code: http.response.status_code
http.scheme: url.scheme
http.url: url.full
http.request_content_length: http.request.body.size
http.response_content_length: http.response.body.size
metrics:
changes:
# https://github.com/open-telemetry/semantic-conventions/pull/53
- rename_metrics:
process.runtime.jvm.cpu.utilization: process.runtime.jvm.cpu.recent_utilization
1.20.0:
spans:
changes:
# https://github.com/open-telemetry/opentelemetry-specification/pull/3272
- rename_attributes:
attribute_map:
net.app.protocol.name: net.protocol.name
net.app.protocol.version: net.protocol.version
1.19.0:
spans:
changes:
# https://github.com/open-telemetry/opentelemetry-specification/pull/3209
- rename_attributes:
attribute_map:
faas.execution: faas.invocation_id
# https://github.com/open-telemetry/opentelemetry-specification/pull/3188
- rename_attributes:
attribute_map:
faas.id: cloud.resource_id
# https://github.com/open-telemetry/opentelemetry-specification/pull/3190
- rename_attributes:
attribute_map:
http.user_agent: user_agent.original
resources:
changes:
# https://github.com/open-telemetry/opentelemetry-specification/pull/3190
- rename_attributes:
attribute_map:
browser.user_agent: user_agent.original
1.18.0:
1.17.0:
spans:
changes:
# https://github.com/open-telemetry/opentelemetry-specification/pull/2957
- rename_attributes:
attribute_map:
messaging.consumer_id: messaging.consumer.id
messaging.protocol: net.app.protocol.name
messaging.protocol_version: net.app.protocol.version
messaging.destination: messaging.destination.name
messaging.temp_destination: messaging.destination.temporary
messaging.destination_kind: messaging.destination.kind
messaging.message_id: messaging.message.id
messaging.conversation_id: messaging.message.conversation_id
messaging.message_payload_size_bytes: messaging.message.payload_size_bytes
messaging.message_payload_compressed_size_bytes: messaging.message.payload_compressed_size_bytes
messaging.rabbitmq.routing_key: messaging.rabbitmq.destination.routing_key
messaging.kafka.message_key: messaging.kafka.message.key
messaging.kafka.partition: messaging.kafka.destination.partition
messaging.kafka.tombstone: messaging.kafka.message.tombstone
messaging.rocketmq.message_type: messaging.rocketmq.message.type
messaging.rocketmq.message_tag: messaging.rocketmq.message.tag
messaging.rocketmq.message_keys: messaging.rocketmq.message.keys
messaging.kafka.consumer_group: messaging.kafka.consumer.group
1.16.0:
1.15.0:
spans:
changes:
# https://github.com/open-telemetry/opentelemetry-specification/pull/2743
- rename_attributes:
attribute_map:
http.retry_count: http.resend_count
1.14.0:
1.13.0:
spans:
changes:
# https://github.com/open-telemetry/opentelemetry-specification/pull/2614
- rename_attributes:
attribute_map:
net.peer.ip: net.sock.peer.addr
net.host.ip: net.sock.host.addr
1.12.0:
1.11.0:
1.10.0:
1.9.0:
1.8.0:
spans:
changes:
- rename_attributes:
attribute_map:
db.cassandra.keyspace: db.name
db.hbase.namespace: db.name
1.7.0:
1.6.1:
1.5.0:
1.4.0:

View File

@@ -18,11 +18,13 @@ pytest_plugins = [
"fixtures.logs",
"fixtures.traces",
"fixtures.metrics",
"fixtures.queriercommon",
"fixtures.metadata",
"fixtures.meter",
"fixtures.browser",
"fixtures.keycloak",
"fixtures.idp",
"fixtures.googleidp",
"fixtures.notification_channel",
"fixtures.maildev",
"fixtures.alerts",

View File

@@ -329,9 +329,6 @@ def clickhouse(
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.TestContainerClickhouse:
"""
Package-scoped fixture for Clickhouse TestContainer.
"""
return create_clickhouse(
tmpfs=tmpfs,
network=network,

View File

@@ -1,5 +1,3 @@
"""Fixtures for cloud integration tests."""
from collections.abc import Callable
from dataclasses import dataclass, field
from http import HTTPStatus

30
tests/fixtures/dashboards.py vendored Normal file
View File

@@ -0,0 +1,30 @@
from http import HTTPStatus
import requests
from fixtures import types
DASHBOARDS_BASE_URL = "/api/v2/dashboards"
# MaxListLimit caps a single list page, so wiping a shared DB has to drain pages
# until the list comes back empty.
MAX_LIST_LIMIT = 200
def delete_all_dashboards(signoz: types.SigNoz, token: str) -> None:
while True:
response = requests.get(
signoz.self.host_configs["8080"].get(f"{DASHBOARDS_BASE_URL}?limit={MAX_LIST_LIMIT}"),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK, response.text
dashboards = response.json()["data"]["dashboards"]
if not dashboards:
return
for dashboard in dashboards:
del_res = requests.delete(
signoz.self.host_configs["8080"].get(f"{DASHBOARDS_BASE_URL}/{dashboard['id']}"),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert del_res.status_code == HTTPStatus.NO_CONTENT, del_res.text

220
tests/fixtures/googleidp.py vendored Normal file
View File

@@ -0,0 +1,220 @@
import base64
import json
import time
from collections.abc import Callable
from http import HTTPStatus
from urllib.parse import urlparse
import docker
import docker.errors
import pytest
import requests
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import padding, rsa
from testcontainers.core.container import DockerContainer, Network
from wiremock.resources.mappings import HttpMethods, Mapping, MappingRequest, MappingResponse
from fixtures import reuse, tls, types
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
# The google callback authn hardcodes Google's issuer, so the mock must be
# reachable as accounts.google.com over TLS from the signoz container: the
# wiremock container joins the network under that alias and serves HTTPS on 443
# with a certificate issued by the integration CA that signoz trusts.
ISSUER = "https://accounts.google.com"
ISSUER_HOST = "accounts.google.com"
def perform_google_login(
signoz: types.SigNoz,
googleidp: types.TestContainerDocker,
get_session_context: Callable[[str], dict],
email: str,
) -> str:
"""Drive the google login flow for email and return the final redirect URL.
The authorize URL points at https://accounts.google.com (resolvable only
inside the docker network), so it is rewritten to the mock's host-mapped
port, mirroring how the oidc suite rewrites keycloak URLs.
"""
session_context = get_session_context(email)
assert len(session_context["orgs"]) == 1
assert len(session_context["orgs"][0]["authNSupport"]["callback"]) == 1
url = session_context["orgs"][0]["authNSupport"]["callback"][0]["url"]
assert url.startswith(f"{ISSUER}/")
parsed_url = urlparse(url)
authorize_url = googleidp.host_configs["8080"].get(f"{parsed_url.path}?{parsed_url.query}")
response = requests.get(authorize_url, allow_redirects=False, timeout=5)
assert response.status_code == HTTPStatus.FOUND
callback_url = response.headers["Location"]
assert "/api/v1/complete/google" in callback_url
response = requests.get(callback_url, allow_redirects=False, timeout=30)
assert response.status_code == HTTPStatus.SEE_OTHER
return response.headers["Location"]
def google_oidc_mappings(email: str, name: str, hd: str, audience: str, email_verified: bool = True) -> list[Mapping]:
"""Wiremock mappings for one Google OIDC login: discovery, an auto-approving
authorize redirect, a token response with an RS256 id_token for the given
identity, and the JWKS the signoz container verifies it against. The signing
key is ephemeral — the token and JWKS stubs are always installed together."""
signing_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
def base64url(data: bytes) -> str:
return base64.urlsafe_b64encode(data).rstrip(b"=").decode()
now = int(time.time())
claims = {
"iss": ISSUER,
"aud": audience,
"sub": f"google-oauth2|{email}",
"email": email,
"email_verified": email_verified,
"name": name,
"hd": hd,
"iat": now,
"exp": now + 3600,
}
signing_input = base64url(json.dumps({"alg": "RS256", "kid": "googleidp-integration", "typ": "JWT"}).encode()) + "." + base64url(json.dumps(claims).encode())
signature = signing_key.sign(signing_input.encode(), padding.PKCS1v15(), hashes.SHA256())
id_token = signing_input + "." + base64url(signature)
public_numbers = signing_key.public_key().public_numbers()
return [
Mapping(
request=MappingRequest(method=HttpMethods.GET, url_path="/.well-known/openid-configuration"),
response=MappingResponse(
status=200,
json_body={
"issuer": ISSUER,
"authorization_endpoint": f"{ISSUER}/o/oauth2/v2/auth",
"token_endpoint": f"{ISSUER}/token",
"jwks_uri": f"{ISSUER}/jwks",
"response_types_supported": ["code"],
"subject_types_supported": ["public"],
"id_token_signing_alg_values_supported": ["RS256"],
"scopes_supported": ["openid", "email", "profile"],
"token_endpoint_auth_methods_supported": ["client_secret_basic", "client_secret_post"],
},
),
),
Mapping(
request=MappingRequest(method=HttpMethods.GET, url_path="/o/oauth2/v2/auth"),
response=MappingResponse(
status=302,
headers={
# Triple-stache: redirect_uri and state are URLs; handlebars
# would otherwise HTML-escape their special characters.
"Location": "{{{request.query.redirect_uri}}}?code=integration-test-code&state={{{request.query.state}}}",
},
transformers=["response-template"],
),
),
Mapping(
request=MappingRequest(method=HttpMethods.POST, url_path="/token"),
response=MappingResponse(
status=200,
json_body={
"access_token": "integration-test-access-token",
"token_type": "Bearer",
"expires_in": 3600,
"id_token": id_token,
},
),
),
Mapping(
request=MappingRequest(method=HttpMethods.GET, url_path="/jwks"),
response=MappingResponse(
status=200,
json_body={
"keys": [
{
"kty": "RSA",
"use": "sig",
"alg": "RS256",
"kid": "googleidp-integration",
"n": base64url(public_numbers.n.to_bytes((public_numbers.n.bit_length() + 7) // 8, "big")),
"e": base64url(public_numbers.e.to_bytes((public_numbers.e.bit_length() + 7) // 8, "big")),
}
]
},
),
),
]
@pytest.fixture(name="googleidp", scope="package")
def googleidp(
network: Network,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.TestContainerDocker:
"""Wiremock impersonating Google's OIDC provider. Stubs are installed per
test via make_http_mocks with google_oidc_mappings; port 8080 serves the
admin API and the authorize redirect to the test process."""
def create() -> types.TestContainerDocker:
keystore_dir = tls.ensure_server_keystore(pytestconfig, ISSUER_HOST)
container = DockerContainer("wiremock/wiremock:2.35.1-1")
container.with_command(f"--https-port 443 --https-keystore /certs/keystore.p12 --keystore-type PKCS12 --keystore-password {tls.KEYSTORE_PASSWORD} --local-response-templating")
container.with_volume_mapping(str(keystore_dir), "/certs", "ro")
container.with_exposed_ports(8080)
container.with_network(network)
container.with_network_aliases(ISSUER_HOST)
container.start()
host = container.get_container_host_ip()
host_port = container.get_exposed_port(8080)
for attempt in range(20):
try:
response = requests.get(f"http://{host}:{host_port}/__admin/mappings", timeout=2)
if response.status_code == HTTPStatus.OK:
break
except Exception as e: # pylint: disable=broad-exception-caught
logger.info("googleidp attempt %d: %s", attempt + 1, e)
time.sleep(1)
else:
raise TimeoutError("googleidp container did not become ready")
return types.TestContainerDocker(
id=container.get_wrapped_container().id,
host_configs={
"8080": types.TestContainerUrlConfig("http", host, host_port),
},
container_configs={
"443": types.TestContainerUrlConfig("https", ISSUER_HOST, 443),
},
)
def delete(container: types.TestContainerDocker) -> None:
client = docker.from_env()
try:
client.containers.get(container_id=container.id).stop()
client.containers.get(container_id=container.id).remove(v=True)
except docker.errors.NotFound:
logger.info("googleidp container %s already gone", container.id)
def restore(cache: dict) -> types.TestContainerDocker:
return types.TestContainerDocker.from_cache(cache)
return reuse.wrap(
request,
pytestconfig,
"googleidp",
lambda: types.TestContainerDocker(id="", host_configs={}, container_configs={}),
create,
delete,
restore,
)

View File

@@ -24,9 +24,6 @@ def zeus(
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.TestContainerDocker:
"""
Package-scoped fixture for running zeus
"""
def create() -> types.TestContainerDocker:
container = WireMockContainer(image="wiremock/wiremock:2.35.1-1", secure=False)
@@ -76,9 +73,6 @@ def gateway(
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.TestContainerDocker:
"""
Package-scoped fixture for running gateway
"""
def create() -> types.TestContainerDocker:
container = WireMockContainer(image="wiremock/wiremock:2.35.1-1", secure=False)

27
tests/fixtures/idp.py vendored
View File

@@ -7,6 +7,7 @@ import pytest
import requests
from keycloak import KeycloakAdmin
from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
@@ -370,18 +371,26 @@ def idp_login(driver: webdriver.Chrome) -> Callable[[str, str], None]:
password_field.send_keys(password)
# Click the login button
idp_host = urlparse(driver.current_url).netloc
login_button = wait.until(EC.element_to_be_clickable((By.ID, "kc-login")))
login_button.click()
# Wait till kc-login element has vanished from the page, which means that a redirection is taking place.
wait.until(EC.invisibility_of_element((By.ID, "kc-login")))
# Wait till the browser has left the idp host — not just the login page: keycloak's SAML flow inserts an
# auto-submitting interstitial on the idp whose POST is what creates the user in signoz. The button is
# re-queried per poll; a mid-navigation WebDriverException (detached node) just retries the poll.
def _left_idp(drv: webdriver.Chrome) -> bool:
try:
return urlparse(drv.current_url).netloc != idp_host and not drv.find_elements(By.ID, "kc-login")
except WebDriverException:
return False
wait.until(_left_idp)
return _idp_login
@pytest.fixture(name="create_group_idp", scope="function")
def create_group_idp(idp: types.TestContainerIDP) -> Callable[[str], str]:
"""Creates a group in Keycloak IDP."""
client = KeycloakAdmin(
server_url=idp.container.host_configs["6060"].base(),
username=IDP_ROOT_USERNAME,
@@ -410,7 +419,6 @@ def create_user_idp_with_groups(
idp: types.TestContainerIDP,
create_group_idp: Callable[[str], str], # pylint: disable=redefined-outer-name
) -> Callable[[str, str, bool, list[str]], None]:
"""Creates a user in Keycloak IDP with specified groups."""
client = KeycloakAdmin(
server_url=idp.container.host_configs["6060"].base(),
username=IDP_ROOT_USERNAME,
@@ -458,7 +466,6 @@ def add_user_to_group(
idp: types.TestContainerIDP,
create_group_idp: Callable[[str], str], # pylint: disable=redefined-outer-name
) -> Callable[[str, str], None]:
"""Adds an existing user to a group."""
client = KeycloakAdmin(
server_url=idp.container.host_configs["6060"].base(),
username=IDP_ROOT_USERNAME,
@@ -479,7 +486,6 @@ def create_user_idp_with_role(
idp: types.TestContainerIDP,
create_group_idp: Callable[[str], str], # pylint: disable=redefined-outer-name
) -> Callable[[str, str, bool, str, list[str]], None]:
"""Creates a user in Keycloak IDP with a custom role attribute and optional groups."""
client = KeycloakAdmin(
server_url=idp.container.host_configs["6060"].base(),
username=IDP_ROOT_USERNAME,
@@ -527,7 +533,6 @@ def create_user_idp_with_role(
@pytest.fixture(name="setup_user_profile", scope="package")
def setup_user_profile(idp: types.TestContainerIDP) -> Callable[[], None]:
"""Setup Keycloak User Profile with signoz_role attribute."""
def _setup_user_profile() -> None:
client = KeycloakAdmin(
@@ -568,7 +573,6 @@ def setup_user_profile(idp: types.TestContainerIDP) -> Callable[[], None]:
def _ensure_groups_client_scope(client: KeycloakAdmin) -> None:
"""Create 'groups' client scope if it doesn't exist."""
# Check if groups scope exists
scopes = client.get_client_scopes()
groups_scope_exists = any(s.get("name") == "groups" for s in scopes)
@@ -619,9 +623,8 @@ def _ensure_groups_client_scope(client: KeycloakAdmin) -> None:
def get_oidc_domain(signoz: types.SigNoz, admin_token: str) -> dict:
"""Helper to get the OIDC domain."""
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
@@ -632,7 +635,6 @@ def get_oidc_domain(signoz: types.SigNoz, admin_token: str) -> dict:
def get_user_by_email(signoz: types.SigNoz, admin_token: str, email: str) -> dict:
"""Helper to get a user by email."""
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/user"),
timeout=2,
@@ -653,7 +655,6 @@ def perform_oidc_login(
email: str,
password: str,
) -> None:
"""Helper to perform OIDC login flow."""
session_context = get_session_context(email)
url = session_context["orgs"][0]["authNSupport"]["callback"][0]["url"]
parsed_url = urlparse(url)
@@ -664,7 +665,7 @@ def perform_oidc_login(
def get_saml_domain(signoz: types.SigNoz, admin_token: str) -> dict:
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)

View File

@@ -1,5 +1,3 @@
"""Shared constants/helpers for v2 infra-monitoring pod-status tests."""
# All 18 PodCountsByStatus buckets (camelCase, matches inframonitoringtypes.PodCountsByStatus / the API response).
STATUS_BUCKETS = (
"pending",

View File

@@ -1,9 +1,3 @@
"""
Simpler version of metadataexporter for exporting jsontypes for test fixtures.
This exports JSON type metadata to the path_types table by parsing JSON bodies
and extracting all paths with their types, similar to how the real metadataexporter works.
"""
import datetime
import json
from abc import ABC
@@ -21,8 +15,6 @@ from fixtures import types
class JSONPathType(ABC):
"""Represents a JSON path with its type information"""
field_name: str
field_data_type: str
last_seen: np.uint64
@@ -44,7 +36,6 @@ class JSONPathType(ABC):
self.last_seen = np.uint64(int(last_seen.timestamp() * 1e9))
def np_arr(self) -> np.array:
"""Return path type data as numpy array for database insertion"""
return np.array([self.signal, self.field_context, self.field_name, self.field_data_type, self.last_seen])
@@ -145,7 +136,7 @@ def _python_type_to_clickhouse_type(value: Any) -> str:
elif isinstance(value, dict):
return "json"
else:
return "string" # Default fallback
return "string"
def _extract_json_paths(
@@ -154,19 +145,7 @@ def _extract_json_paths(
path_types: dict[str, set[str]] | None = None,
level: int = 0,
) -> dict[str, set[str]]:
"""
Recursively extract all paths and their types from a JSON object.
Matches metadataexporter's analyzePValue logic.
Args:
obj: The JSON object to traverse
current_path: Current path being built (e.g., "user.name")
path_types: Dictionary mapping paths to sets of types found
level: Current nesting level (for depth limiting)
Returns:
Dictionary mapping paths to sets of type strings
"""
"""Matches metadataexporter's analyzePValue logic."""
if path_types is None:
path_types = {}
@@ -179,17 +158,14 @@ def _extract_json_paths(
# Matches Go walkMap which recurses without calling ta.record on the map node.
for key, value in obj.items():
# Build the path for this key
if current_path:
new_path = f"{current_path}.{key}"
else:
new_path = key
# Recurse into the value
_extract_json_paths(value, new_path, path_types, level + 1)
elif isinstance(obj, list):
# Skip empty arrays
if len(obj) == 0:
return path_types
@@ -246,17 +222,6 @@ def _parse_json_bodies_and_extract_paths(
json_bodies: list[str],
timestamp: datetime.datetime | None = None,
) -> list[JSONPathType]:
"""
Parse JSON bodies and extract all paths with their types.
This mimics the behavior of metadataexporter.
Args:
json_bodies: List of JSON body strings to parse
timestamp: Timestamp to use for last_seen (defaults to now)
Returns:
List of JSONPathType objects with all discovered paths and types
"""
if timestamp is None:
timestamp = datetime.datetime.now()
@@ -268,11 +233,9 @@ def _parse_json_bodies_and_extract_paths(
parsed = json.loads(json_body)
_extract_json_paths(parsed, "", all_path_types, level=0)
except (json.JSONDecodeError, TypeError):
# Skip invalid JSON
continue
# Convert to list of JSONPathType objects
# Each path can have multiple types, so we create one JSONPathType per type
# Each path can have multiple types -> one JSONPathType per type
path_type_objects: list[JSONPathType] = []
for path, types_set in all_path_types.items():
for type_str in types_set:
@@ -285,64 +248,34 @@ def _parse_json_bodies_and_extract_paths(
def export_json_types(
clickhouse: types.TestContainerClickhouse,
) -> Generator[Callable[[list[JSONPathType] | list[str] | list[Any]], None], Any]:
"""
Fixture for exporting JSON type metadata to the path_types table.
This is a simpler version of metadataexporter for test fixtures.
"""Write JSON path/type metadata the way the real metadataexporter would.
The function can accept:
1. List of JSONPathType objects (manual specification)
2. List of JSON body strings (auto-extract paths)
3. List of Logs objects (extract from body_json field)
Usage examples:
# Manual specification
export_json_types([
JSONPathType(field_name="user.name", field_data_type="string"),
JSONPathType(field_name="user.age", field_data_type="int64"),
])
# Auto-extract from JSON strings
export_json_types([
'{"user": {"name": "alice", "age": 25}}',
'{"user": {"name": "bob", "age": 30}}',
])
# Auto-extract from Logs objects
export_json_types(logs_list)
Accepts JSONPathType objects (manual specification), raw JSON body strings,
or Logs objects (paths auto-extracted from the JSON body).
"""
def _export_json_types(
data: list[JSONPathType] | list[str] | list[Any], # List[Logs] but avoiding circular import
) -> None:
"""
Export JSON type metadata to signoz_metadata.distributed_field_keys table.
This table stores signal, context, path, and type information for body JSON fields.
"""
path_types: list[JSONPathType] = []
if len(data) == 0:
return
# Determine input type and convert to JSONPathType list
first_item = data[0]
if isinstance(first_item, JSONPathType):
# Already JSONPathType objects
path_types = data # type: ignore
elif isinstance(first_item, str):
# List of JSON strings - parse and extract paths
path_types = _parse_json_bodies_and_extract_paths(data) # type: ignore
else:
# Assume it's a list of Logs objects - extract body_v2
json_bodies: list[str] = []
for log in data: # type: ignore
# Try to get body_v2 attribute
if hasattr(log, "body_v2") and log.body_v2:
json_bodies.append(log.body_v2)
elif hasattr(log, "body") and log.body:
# Fallback to body if body_v2 not available
try:
# Try to parse as JSON
json.loads(log.body)
json_bodies.append(log.body)
except (json.JSONDecodeError, TypeError):
@@ -369,7 +302,6 @@ def export_json_types(
yield _export_json_types
# Cleanup - truncate the local table after tests (following pattern from logs fixture)
clickhouse.conn.query(f"TRUNCATE TABLE signoz_metadata.field_keys ON CLUSTER '{clickhouse.env['SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_CLUSTER']}' SYNC")

View File

@@ -109,9 +109,6 @@ def keeper(
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.TestContainerDocker:
"""
Package-scoped fixture for ClickHouse Keeper TestContainer.
"""
return create_clickhouse_keeper(
tmpfs=tmpfs,
network=network,

View File

@@ -19,9 +19,6 @@ def idp(
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.TestContainerIDP:
"""
Package-scoped fixture for running an idp for SSO/SAML
"""
def create() -> types.TestContainerIDP:
container = KeycloakContainer(

View File

@@ -311,7 +311,6 @@ class Logs(ABC):
self.attribute_keys.append(LogsResourceOrAttributeKeys(name="severity_number", datatype="float64"))
def _get_severity_number(self, severity_text: str) -> np.uint8:
"""Convert severity text to numeric value"""
severity_map = {
"TRACE": 1,
"DEBUG": 5,
@@ -324,7 +323,6 @@ class Logs(ABC):
return np.uint8(severity_map.get(severity_text.upper(), 9)) # Default to INFO
def np_arr(self) -> np.array:
"""Return log data as numpy array for database insertion"""
return np.array(
[
self.ts_bucket_start,
@@ -356,7 +354,6 @@ class Logs(ABC):
cls,
data: dict,
) -> "Logs":
"""Create a Logs instance from a dict."""
# parse timestamp from iso format
timestamp = parse_timestamp(data["timestamp"])
return cls(

View File

@@ -374,6 +374,7 @@ class Metrics(ABC):
file_path: str,
base_time: datetime.datetime | None = None,
metric_name_override: str | None = None,
label_substitutions: dict[str, str] | None = None,
) -> list["Metrics"]:
"""
Load metrics from a JSONL file.
@@ -385,6 +386,9 @@ class Metrics(ABC):
base_time: If provided, all timestamps are shifted so the earliest
timestamp in the file maps to base_time
metric_name_override: If provided, overrides metric_name for all metrics
label_substitutions: If provided, any label whose value equals a key is
rewritten to that key's value (placeholder substitution,
e.g. {"__START_TIME__": start_time.isoformat()})
"""
data_list = []
with open(file_path, encoding="utf-8") as f:
@@ -392,7 +396,13 @@ class Metrics(ABC):
line = line.strip()
if not line:
continue
data_list.append(json.loads(line))
data = json.loads(line)
if label_substitutions:
labels = data.get("labels", {})
for key, value in labels.items():
if value in label_substitutions:
labels[key] = label_substitutions[value]
data_list.append(data)
if not data_list:
return []

View File

@@ -92,9 +92,6 @@ def migrator(
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.Operation:
"""
Package-scoped fixture for running schema migrations.
"""
return create_migrator(
network=network,
clickhouse=clickhouse,

View File

@@ -13,9 +13,6 @@ logger = setup_logger(__name__)
@pytest.fixture(name="network", scope="package")
def network(request: pytest.FixtureRequest, pytestconfig: pytest.Config) -> types.Network:
"""
Package-Scoped fixture for creating a network
"""
def create() -> types.Network:
nw = Network()

View File

@@ -13,9 +13,6 @@ logger = setup_logger(__name__)
@pytest.fixture(name="postgres", scope="package")
def postgres(network: Network, request: pytest.FixtureRequest, pytestconfig: pytest.Config) -> types.TestContainerSQL:
"""
Package-scoped fixture for PostgreSQL TestContainer.
"""
def create() -> types.TestContainerSQL:
version = request.config.getoption("--postgres-version")

View File

@@ -1,44 +0,0 @@
"""Client helpers for the /prometheus/api/v1 endpoints."""
import math
import requests
from fixtures import types
SPECIALS = {"NaN": math.nan, "Inf": math.inf, "+Inf": math.inf, "-Inf": -math.inf}
QUERY_TIMEOUT = 30
def prom_api_get(signoz: types.SigNoz, token: str, path: str, params: dict) -> requests.Response:
return requests.get(
signoz.self.host_configs["8080"].get(path),
params=params,
timeout=QUERY_TIMEOUT,
headers={"authorization": f"Bearer {token}"},
)
def prom_api_value(v: str) -> float:
"""Prometheus API sample values are strings, including "NaN" and "+Inf"."""
if v in SPECIALS:
return SPECIALS[v]
return float(v)
def series_from_prom_result(result_type: str, result) -> dict[tuple, dict[int, float]]:
"""Flattens a matrix/vector/scalar result into
{sorted-labels tuple: {unix_ms: value}}."""
out: dict[tuple, dict[int, float]] = {}
if result_type == "matrix":
for series in result:
points = {round(float(ts) * 1000): prom_api_value(v) for ts, v in series.get("values") or []}
out[tuple(sorted((series.get("metric") or {}).items()))] = points
elif result_type == "vector":
for series in result:
ts, v = series["value"]
out[tuple(sorted((series.get("metric") or {}).items()))] = {round(float(ts) * 1000): prom_api_value(v)}
elif result_type == "scalar":
ts, v = result
out[()] = {round(float(ts) * 1000): prom_api_value(v)}
return out

View File

@@ -1,112 +0,0 @@
"""Shared helpers for suites that replay the frozen promqltest corpus
(tests/integration/testdata/promqltestcorpus/corpus.json)."""
import json
import math
import os
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
import pytest
from fixtures.metrics import Metrics
TESTDATA_DIR = os.path.join(os.path.dirname(__file__), "..", "integration", "testdata", "promqltestcorpus")
CORPUS_FILE = os.path.join(TESTDATA_DIR, "corpus.json")
ISOLATION_GAP_MS = 2 * 3600 * 1000
SPECIALS = {"NaN": math.nan, "Inf": math.inf, "+Inf": math.inf, "-Inf": -math.inf}
def decode_corpus_value(v: float | str) -> float:
if isinstance(v, str):
return SPECIALS[v]
return float(v)
def values_close(a: float, b: float) -> bool:
"""Expected corpus values carry the v5 API's rounding (>=1: three decimal
places; <1: three significant digits). One rounding quantum covers both a
raw-vs-rounded comparison and a boundary that rounds either way."""
if math.isnan(a) or math.isnan(b):
return math.isnan(a) and math.isnan(b)
if math.isinf(a) or math.isinf(b):
return a == b
if a == b:
return True
scale = max(abs(a), abs(b))
if scale >= 1:
quantum = max(1e-3, scale * 1e-9)
else:
quantum = 10 ** (math.floor(math.log10(scale)) - 2)
return abs(a - b) <= quantum + 1e-12
def labelset(labels: dict[str, str]) -> tuple:
return tuple(sorted(labels.items()))
def ledger(filename: str) -> dict[str, str]:
path = os.path.join(TESTDATA_DIR, filename)
if not os.path.exists(path):
return {}
with open(path, encoding="utf-8") as f:
return json.load(f)["divergences"]
@pytest.fixture(name="ingest_promqltest_corpus")
def ingest_promqltest_corpus(insert_metrics: Callable[[list[Metrics]], None]) -> Callable[[], tuple[dict, dict[int, int]]]:
"""Yields a callable that loads the corpus, lays its datasets end to end
on the timeline, ingests every sample, and returns (corpus, dataset base
timestamps).
Dataset bases are hour-aligned: registration rows are hour-bucketed, so
behavior depends on where samples fall relative to hour boundaries, and
exact known-divergences enforcement needs identical placement every run.
Datasets sit on disjoint windows (2h gaps, far beyond the 5m lookback) so
one bulk ingest serves every case without cross-talk."""
def ingest() -> tuple[dict, dict[int, int]]:
with open(CORPUS_FILE, encoding="utf-8") as f:
corpus = json.load(f)
cases_by_dataset: dict[int, list[dict]] = {}
for case in corpus["cases"]:
cases_by_dataset.setdefault(case["dataset"], []).append(case)
spans = {}
for ds in corpus["datasets"]:
sample_max = max((s["samples"][-1][0] for s in ds["series"] if s["samples"]), default=0)
case_max = max((c["end_ms"] for c in cases_by_dataset.get(ds["id"], [])), default=0)
spans[ds["id"]] = max(sample_max, case_max) + corpus["meta"]["lookback_ms"]
hour_ms = 3_600_000
advances = {ds["id"]: -(-(spans[ds["id"]] + ISOLATION_GAP_MS) // hour_ms) * hour_ms for ds in corpus["datasets"]}
total = sum(advances.values())
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
cursor = (int((now - timedelta(hours=1)).timestamp() * 1000) - total) // hour_ms * hour_ms
bases: dict[int, int] = {}
metrics: list[Metrics] = []
for ds in corpus["datasets"]:
bases[ds["id"]] = cursor
for series in ds["series"]:
labels = dict(series["labels"])
metric_name = labels.pop("__name__")
for off_ms, raw in series["samples"]:
stale = raw == "stale"
metrics.append(
Metrics(
metric_name=metric_name,
labels=labels,
timestamp=datetime.fromtimestamp((cursor + off_ms) / 1000, tz=UTC),
value=0.0 if stale else decode_corpus_value(raw),
flags=1 if stale else 0,
)
)
cursor += advances[ds["id"]]
insert_metrics(metrics)
return corpus, bases
return ingest

View File

@@ -704,6 +704,7 @@ def build_raw_query(
order: list[dict] | None = None,
limit: int | None = None,
filter_expression: str | None = None,
select_fields: list[dict] | None = None,
step_interval: int = DEFAULT_STEP_INTERVAL,
disabled: bool = False,
) -> dict:
@@ -723,6 +724,9 @@ def build_raw_query(
if filter_expression:
spec["filter"] = {"expression": filter_expression}
if select_fields:
spec["selectFields"] = select_fields
return {"type": "builder_query", "spec": spec}
@@ -1105,3 +1109,47 @@ def make_scalar_query_request(
"formatOptions": {"formatTableResultForUI": True, "fillGaps": False},
},
)
def run_query_case(signoz: types.SigNoz, token: str, now: datetime, case: dict[str, Any]) -> None:
start_ms = case.get("startMs", int((now - timedelta(seconds=10)).timestamp() * 1000))
end_ms = case.get("endMs", int(now.timestamp() * 1000))
if case["requestType"] == "raw":
query = build_raw_query(
name=case["name"],
signal="logs",
filter_expression=case.get("expression"),
order=case.get("order") or [build_order_by("timestamp", "desc")],
limit=case.get("limit", 100),
step_interval=case.get("stepInterval") or 60,
)
else:
aggregation = case.get("aggregation")
if aggregation and not isinstance(aggregation, list):
aggregations = [build_aggregation(aggregation)]
elif aggregation:
aggregations = aggregation
else:
aggregations = []
query = build_scalar_query(
name=case["name"],
signal="logs",
aggregations=aggregations,
group_by=case.get("groupBy"),
order=case.get("order"),
limit=case.get("limit", 100),
filter_expression=case.get("expression"),
step_interval=case.get("stepInterval") or 60,
)
response = make_query_request(
signoz=signoz,
token=token,
start_ms=start_ms,
end_ms=end_ms,
queries=[query],
request_type=case["requestType"],
)
assert response.status_code == 200, f"HTTP {response.status_code} for case '{case['name']}': {response.text}"
assert case["validate"](response), f"Validation failed for case '{case['name']}': {response.json()}"

View File

@@ -1,8 +1,3 @@
"""
Trace builders for the querierai suite. Every builder pins its spans a few seconds
before the given `now` so `query_window(now)` covers them.
"""
from datetime import datetime, timedelta
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode

124
tests/fixtures/queriercommon.py vendored Normal file
View File

@@ -0,0 +1,124 @@
"""Seed data for the queriercommon keyless-semantics tests.
Three identities exist in every signal. GOLD and SILVER carry the test keys.
NONE carries no key at all. The tests assert which identities a filter
returns, so the membership of NONE is the point of every case.
The attribute names are outside every semantic-convention family, so the
seeded data pins base behavior with any semconv overlay state.
"""
from collections.abc import Callable, Generator
from datetime import UTC, datetime, timedelta
import pytest
from fixtures.logs import Logs
from fixtures.metrics import Metrics
from fixtures.querier import aligned_epoch
from fixtures.traces import TraceIdGenerator, Traces, TracesKind, TracesStatusCode
PREFIX = "keyless-sem"
STRING_KEY = "tenant.tier"
NUMBER_KEY = "retry.count"
METRIC_NAME = "keyless_semantics_gauge"
METRIC_LABEL = "tenant_tier"
# Row identities, keyed by the value of the string key that each row carries.
GOLD = f"{PREFIX}-gold"
SILVER = f"{PREFIX}-silver"
NONE = f"{PREFIX}-none" # carries no string key and no number key
# (identity, string-key value, number-key value, insert offset)
_ROWS = [
(GOLD, "gold", 0, timedelta(seconds=3)),
(SILVER, "silver", 5, timedelta(seconds=2)),
(NONE, None, None, timedelta(seconds=1)),
]
def _resources(identity: str, tier: str | None) -> dict:
base = {"service.name": identity}
if tier is not None:
base[STRING_KEY] = tier
return base
def _attributes(tier: str | None, retries: int | None) -> dict:
attrs: dict = {}
if tier is not None:
attrs[STRING_KEY] = tier
if retries is not None:
attrs[NUMBER_KEY] = retries
return attrs
@pytest.fixture(name="keyless_rows", scope="function")
def keyless_rows(
insert_logs: Callable[[list[Logs]], None],
insert_traces: Callable[[list[Traces]], None],
) -> Generator[datetime]:
"""Inserts one span and one log per identity: GOLD (string "gold",
number 0), SILVER (string "silver", number 5), and NONE (no keys).
Yields the base timestamp. Span name and log body are the identity."""
now = datetime.now(tz=UTC).replace(microsecond=0) - timedelta(minutes=1)
insert_traces(
[
Traces(
timestamp=now - offset,
duration=timedelta(milliseconds=10),
trace_id=TraceIdGenerator.trace_id(),
span_id=TraceIdGenerator.span_id(),
name=identity,
kind=TracesKind.SPAN_KIND_SERVER,
status_code=TracesStatusCode.STATUS_CODE_OK,
resources=_resources(identity, tier),
attributes=_attributes(tier, retries),
)
for identity, tier, retries, offset in _ROWS
]
)
insert_logs(
[
Logs(
timestamp=now - offset,
body=identity,
resources=_resources(identity, tier),
attributes=_attributes(tier, retries),
)
for identity, tier, retries, offset in _ROWS
]
)
yield now
@pytest.fixture(name="keyless_series", scope="function")
def keyless_series(insert_metrics: Callable[[list[Metrics]], None]) -> Generator[tuple[int, int]]:
"""Inserts three gauge series: GOLD and SILVER carry the metric label,
NONE does not. The `service` label is the identity. Yields the
(start, end) epoch-second window that covers the points."""
start = aligned_epoch(timedelta(minutes=30))
points = 5
def labels(identity: str, tier: str | None) -> dict:
base = {"service": identity}
if tier is not None:
base[METRIC_LABEL] = tier
return base
insert_metrics(
[
Metrics(
metric_name=METRIC_NAME,
labels=labels(identity, tier),
timestamp=datetime.fromtimestamp(start + minute * 60, tz=UTC),
value=10.0,
type_="Gauge",
is_monotonic=False,
)
for identity, tier in ((GOLD, "gold"), (SILVER, "silver"), (NONE, None))
for minute in range(points)
]
)
yield start, start + points * 60

View File

@@ -19,17 +19,14 @@ def teardown(request: pytest.FixtureRequest) -> bool:
def get_cached_resource(pytestconfig: pytest.Config, key: str):
"""Get a resource from pytest cache by key."""
return pytestconfig.cache.get(key, None)
def set_cached_resource(pytestconfig: pytest.Config, key: str, value):
"""Set a resource in pytest cache by key."""
pytestconfig.cache.set(key, value)
def remove_cached_resource(pytestconfig: pytest.Config, key: str):
"""Remove a resource from pytest cache by key (set to None)."""
pytestconfig.cache.set(key, None)

View File

@@ -1,5 +1,3 @@
"""Fixtures and helpers for role tests."""
import json
from collections.abc import Callable
from http import HTTPStatus

View File

@@ -1,11 +1,3 @@
"""Golden dataset fixture — seeds OTel-demo-shaped metrics, traces, and
logs into ClickHouse via the seeder on every test_setup invocation.
Timestamps are rebased to `now` so panels with default time windows
always find data. To refresh the dataset shape on disk, run
`uv run python -m fixtures.seed_golden_dataset regenerate`.
"""
from __future__ import annotations
import datetime

View File

@@ -1,5 +1,3 @@
"""Fixtures and helpers for service account tests."""
from http import HTTPStatus
import requests

View File

@@ -11,7 +11,7 @@ import pytest
import requests
from testcontainers.core.container import DockerContainer, Network
from fixtures import reuse, types
from fixtures import reuse, tls, types
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
@@ -115,6 +115,13 @@ def create_signoz(
"rw",
)
# Trust the integration CA so tests can stand in for real TLS hosts
# (e.g. the fake accounts.google.com); SSL_CERT_FILE replaces the Go
# root pool, which is fine here since every other mocked upstream is
# plain http.
container.with_env("SSL_CERT_FILE", tls.CA_CONTAINER_PATH)
container.with_volume_mapping(str(tls.ensure_ca(pytestconfig) / "ca.pem"), tls.CA_CONTAINER_PATH, "ro")
container.start()
def ready(container: DockerContainer) -> None:
@@ -225,9 +232,6 @@ def signoz( # pylint: disable=too-many-arguments,too-many-positional-arguments
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.SigNoz:
"""
Package-scoped fixture for setting up SigNoz.
"""
return create_signoz(
network=network,
zeus=zeus,

View File

@@ -7,9 +7,6 @@ from fixtures import types
def sqlstore(
request: pytest.FixtureRequest,
) -> types.TestContainerSQL:
"""
Packaged-scoped fixture for creating sql store.
"""
provider = request.config.getoption("--sqlstore-provider")
if provider == "postgres":

View File

@@ -16,9 +16,6 @@ def sqlite(
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.TestContainerSQL:
"""
Package-scoped fixture for SQLite.
"""
def create() -> types.TestContainerSQL:
tmpdir = tmpfs("sqlite")

View File

@@ -1,9 +1,3 @@
"""Shared helpers for the third-party (external) API monitoring domain list.
A translator over v5 builder queries that answers with a UI-formatted scalar table, so the
response is read by column rather than by series.
"""
from typing import Any
import requests
@@ -49,7 +43,9 @@ def make_third_party_apis_request(
def scalar_result(response: requests.Response) -> dict:
"""The single scalar table from a third-party-apis response."""
"""The single scalar table from a third-party-apis response. The endpoint is a
translator over v5 builder queries answering with a UI-formatted scalar table,
so the response is read by column rather than by series."""
return response.json()["data"]["data"]["results"][0]

View File

@@ -6,9 +6,6 @@ import isodate
# parses the given timestamp string from ISO format to datetime.datetime
def parse_timestamp(ts_str: str) -> datetime.datetime:
"""
Parse a timestamp string from ISO format.
"""
if ts_str.endswith("Z"):
ts_str = ts_str[:-1] + "+00:00"
return datetime.datetime.fromisoformat(ts_str)
@@ -16,9 +13,6 @@ def parse_timestamp(ts_str: str) -> datetime.datetime:
# parses the given duration to datetime.timedelta
def parse_duration(duration: Any) -> datetime.timedelta:
"""
Parse a duration string from ISO format.
"""
# if it's string then parse it as iso format
if isinstance(duration, str):
return isodate.parse_duration(duration)

90
tests/fixtures/tls.py vendored Normal file
View File

@@ -0,0 +1,90 @@
import datetime
from pathlib import Path
import pytest
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives.serialization import pkcs12
from cryptography.x509.oid import NameOID
# The integration CA is mounted into every signoz container and trusted via
# SSL_CERT_FILE (which replaces the Go root pool), so mocks that must be
# reached over TLS under a real hostname (e.g. accounts.google.com) can serve
# certificates issued by it. Material is persisted under .pytest_cache so
# --reuse runs keep the chain the running containers already trust.
CA_CONTAINER_PATH = "/etc/signoz-integration/ca.pem"
KEYSTORE_PASSWORD = "password" # noqa: S105
def ensure_ca(pytestconfig: pytest.Config) -> Path:
"""Directory holding the integration CA (ca.pem + ca.key), created once."""
ca_dir = pytestconfig.cache.mkdir("tls")
if (ca_dir / "ca.pem").exists() and (ca_dir / "ca.key").exists():
return ca_dir
now = datetime.datetime.now(datetime.UTC)
ca_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
ca_name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "signoz-integration-ca")])
ca_cert = (
x509.CertificateBuilder()
.subject_name(ca_name)
.issuer_name(ca_name)
.public_key(ca_key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - datetime.timedelta(days=1))
.not_valid_after(now + datetime.timedelta(days=3650))
.add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True)
.sign(ca_key, hashes.SHA256())
)
(ca_dir / "ca.pem").write_bytes(ca_cert.public_bytes(serialization.Encoding.PEM))
(ca_dir / "ca.key").write_bytes(
ca_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
)
return ca_dir
def ensure_server_keystore(pytestconfig: pytest.Config, hostname: str) -> Path:
"""Directory holding a PKCS12 keystore (keystore.p12, password
KEYSTORE_PASSWORD) with a certificate for hostname issued by the
integration CA."""
ca_dir = ensure_ca(pytestconfig)
keystore_dir = pytestconfig.cache.mkdir(f"tls-{hostname}")
if (keystore_dir / "keystore.p12").exists():
return keystore_dir
ca_cert = x509.load_pem_x509_certificate((ca_dir / "ca.pem").read_bytes())
ca_key = serialization.load_pem_private_key((ca_dir / "ca.key").read_bytes(), password=None)
now = datetime.datetime.now(datetime.UTC)
leaf_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
leaf_cert = (
x509.CertificateBuilder()
.subject_name(x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, hostname)]))
.issuer_name(ca_cert.subject)
.public_key(leaf_key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - datetime.timedelta(days=1))
.not_valid_after(now + datetime.timedelta(days=3650))
.add_extension(x509.SubjectAlternativeName([x509.DNSName(hostname)]), critical=False)
.add_extension(x509.ExtendedKeyUsage([x509.oid.ExtendedKeyUsageOID.SERVER_AUTH]), critical=False)
.sign(ca_key, hashes.SHA256())
)
(keystore_dir / "keystore.p12").write_bytes(
pkcs12.serialize_key_and_certificates(
name=hostname.encode(),
key=leaf_key,
cert=leaf_cert,
cas=[ca_cert],
encryption_algorithm=serialization.BestAvailableEncryption(KEYSTORE_PASSWORD.encode()),
)
)
return keystore_dir

View File

@@ -625,7 +625,6 @@ class Traces(ABC):
self.response_status_code = str_value
def np_arr(self) -> np.array:
"""Return span data as numpy array for database insertion"""
return np.array(
[
self.ts_bucket_start,
@@ -669,7 +668,6 @@ class Traces(ABC):
cls,
data: dict,
) -> "Traces":
"""Create a Traces instance from a dict."""
# parse timestamp from iso format
timestamp = parse_timestamp(data["timestamp"])
duration = parse_duration(data.get("duration", "PT1S"))

View File

@@ -1,12 +0,0 @@
{
"note": "Divergences of the /prometheus/api/v1 endpoints, served by the clickhousev2 provider, from the upstream reference engine. 01_prometheus_api_corpus.py enforces this set exactly in both directions. All current entries are the Kahan class recorded in known_divergences_v2.json: the engine sums with Kahan compensation and an overflow-free incremental mean, ClickHouse's aggregates are naive. Only the [instant-coarse] variants appear here: they are range-encoded, so they serve transpiled; the [base] instant evals go through /prometheus/api/v1/query on the exact engine path.",
"divergences": {
"aggregators.test:651[instant-coarse]": "avg over near-max-float64 values: avgForEach overflows to +Inf where the engine's incremental mean does not",
"aggregators.test:654[instant-coarse]": "avg over near-min-float64 values: avgForEach overflows to -Inf",
"aggregators.test:687[instant-coarse]": "sum over {1e100, -1e100, small}: naive summation cancels to 0 where the engine's Kahan sum keeps 10",
"aggregators.test:695[instant-coarse]": "avg over {1e100, -1e100, small}: same cancellation divided by count",
"functions.test:1084[instant-coarse]": "sum_over_time over a ±1e100 window: the disjoint coarse-step form's arraySum cancels to 0",
"functions.test:1087[instant-coarse]": "avg_over_time, same window and cancellation as functions.test:1084",
"functions.test:1149[instant-coarse]": "avg_over_time over ±2.258e220 samples: naive slide summation leaves a ~1e202 residue where the engine cancels to 0"
}
}

View File

@@ -20,9 +20,6 @@ def test_webhook_notification_channel(
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
create_webhook_notification_channel: Callable[[str, str, dict, bool], str],
) -> None:
"""
Tests the creation and delivery of test alerts on the created notification channel
"""
logger.info("Setting up notification channel")
# Prepare notification channel name and webhook endpoint

View File

@@ -50,13 +50,13 @@ def test_create_auth_domain(
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.post(
signoz.self.host_configs["8080"].get("/signoz/api/v1/domains"),
signoz.self.host_configs["8080"].get("/signoz/api/v2/auth_domains"),
json={
"name": "oidc.basepath.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "oidc",
"oidcConfig": {
"kind": "oidc",
"spec": {
"clientId": settings["client_id"],
"clientSecret": settings["client_secret"],
# Change the hostname of the issuer to the internal resolvable hostname of the idp

View File

@@ -48,16 +48,16 @@ def test_create_auth_domain(
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.post(
signoz.self.host_configs["8080"].get("/signoz/api/v1/domains"),
signoz.self.host_configs["8080"].get("/signoz/api/v2/auth_domains"),
json={
"name": "saml.basepath.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": settings["entityID"],
"samlIdp": settings["singleSignOnServiceLocation"],
"samlCert": settings["certificate"],
"kind": "saml",
"spec": {
"entityId": settings["entityID"],
"location": settings["singleSignOnServiceLocation"],
"certificate": settings["certificate"],
},
},
},

View File

@@ -1,6 +1,7 @@
from collections.abc import Callable
from http import HTTPStatus
import pytest
import requests
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
@@ -14,27 +15,33 @@ def test_create_and_get_domain(
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# Get domains which should be an empty list
# Reruns against a reused stack find domains from previous runs; drop them
# all so the suite starts from a clean slate.
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
assert response.json()["status"] == "success"
data = response.json()["data"]
assert len(data) == 0
for domain in response.json()["data"]:
response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT
# Create a domain with google auth config
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "domain-google.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "google_auth",
"googleAuthConfig": {
"kind": "google",
"spec": {
"clientId": "client-id",
"clientSecret": "client-secret",
"redirectURI": "redirect-uri",
@@ -49,16 +56,16 @@ def test_create_and_get_domain(
# Create a domain with saml config
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "domain-saml.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": "saml-entity",
"samlIdp": "saml-idp",
"samlCert": "saml-cert",
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "saml-idp",
"certificate": "saml-cert",
},
},
},
@@ -70,7 +77,7 @@ def test_create_and_get_domain(
# List the domains
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
@@ -86,7 +93,7 @@ def test_create_and_get_domain(
"domain-google.integration.test",
"domain-saml.integration.test",
]
assert domain["config"]["ssoType"] in ["google_auth", "saml"]
assert domain["config"]["kind"] in ["google", "saml"]
def test_create_invalid(
@@ -96,15 +103,15 @@ def test_create_invalid(
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# Create a domain with type saml and body for oidc, this should fail because oidcConfig is not allowed for saml
# Create a domain with kind saml and a spec for oidc, this should fail because the spec does not match the kind
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "domain.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"oidcConfig": {
"kind": "saml",
"spec": {
"clientId": "client-id",
"clientSecret": "client-secret",
"issuer": "issuer",
@@ -117,18 +124,34 @@ def test_create_invalid(
assert response.status_code == HTTPStatus.BAD_REQUEST
# Create a domain with a kind but no spec
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "domain.integration.test",
"enabled": True,
"config": {
"kind": "saml",
},
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.BAD_REQUEST
# Create a domain with invalid name
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "$%^invalid",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": "saml-entity",
"samlIdp": "saml-idp",
"samlCert": "saml-cert",
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "saml-idp",
"certificate": "saml-cert",
},
},
},
@@ -140,17 +163,17 @@ def test_create_invalid(
# Create a domain with no name
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": "saml-entity",
"samlIdp": "saml-idp",
"samlCert": "saml-cert",
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "saml-idp",
"certificate": "saml-cert",
},
}
},
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
@@ -160,7 +183,7 @@ def test_create_invalid(
# Create a domain with no config
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "domain.integration.test",
},
@@ -176,26 +199,25 @@ def test_create_invalid_role_mapping(
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
):
"""Test that invalid role mappings are rejected."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# Create domain with invalid defaultRole
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "invalid-role-test.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": "saml-entity",
"samlIdp": "saml-idp",
"samlCert": "saml-cert",
},
"roleMapping": {
"defaultRole": "SUPERADMIN", # Invalid role
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "saml-idp",
"certificate": "saml-cert",
},
},
"roleMapping": {
"defaultRole": "SUPERADMIN", # Invalid role
},
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
@@ -205,22 +227,22 @@ def test_create_invalid_role_mapping(
# Create domain with invalid role in groupMappings
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "invalid-group-role.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": "saml-entity",
"samlIdp": "saml-idp",
"samlCert": "saml-cert",
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "saml-idp",
"certificate": "saml-cert",
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"admins": "SUPERUSER", # Invalid role
},
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"admins": "SUPERUSER", # Invalid role
},
},
},
@@ -232,23 +254,23 @@ def test_create_invalid_role_mapping(
# Valid role mapping should succeed
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "valid-role-mapping.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": "saml-entity",
"samlIdp": "saml-idp",
"samlCert": "saml-cert",
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "saml-idp",
"certificate": "saml-cert",
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
},
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
},
},
},
@@ -257,3 +279,288 @@ def test_create_invalid_role_mapping(
)
assert response.status_code == HTTPStatus.CREATED
@pytest.mark.parametrize(
("config", "role_mapping", "expected_config", "expected_role_mapping"),
[
pytest.param(
{
"kind": "google",
"spec": {"clientId": "client-id", "clientSecret": "client-secret"},
},
None,
{
"kind": "google",
"spec": {
"clientId": "client-id",
"clientSecret": "client-secret",
"redirectURI": "",
"fetchGroups": False,
"insecureSkipEmailVerified": False,
},
},
None,
id="google_minimal",
),
pytest.param(
{
"kind": "google",
"spec": {
"clientId": "client-id",
"clientSecret": "client-secret",
"redirectURI": "https://redirect.integration.test",
"fetchGroups": True,
"serviceAccountJson": '{"type": "service_account"}',
"domainToAdminEmail": {
"roundtrip.integration.test": "admin@roundtrip.integration.test",
"*": "fallback@roundtrip.integration.test",
},
"fetchTransitiveGroupMembership": True,
"allowedGroups": ["group-one", "group-two"],
"insecureSkipEmailVerified": True,
},
},
None,
{
"kind": "google",
"spec": {
"clientId": "client-id",
"clientSecret": "client-secret",
"redirectURI": "https://redirect.integration.test",
"fetchGroups": True,
"serviceAccountJson": '{"type": "service_account"}',
"domainToAdminEmail": {
"roundtrip.integration.test": "admin@roundtrip.integration.test",
"*": "fallback@roundtrip.integration.test",
},
"fetchTransitiveGroupMembership": True,
"allowedGroups": ["group-one", "group-two"],
"insecureSkipEmailVerified": True,
},
},
None,
id="google_full",
),
pytest.param(
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
},
},
None,
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
"insecureSkipAuthNRequestsSigned": False,
"attributeMapping": {"email": "email", "name": "name", "groups": "groups", "role": "role"},
},
},
None,
id="saml_minimal",
),
pytest.param(
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
"insecureSkipAuthNRequestsSigned": True,
"attributeMapping": {"email": "mail"},
},
},
None,
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
"insecureSkipAuthNRequestsSigned": True,
"attributeMapping": {"email": "mail", "name": "name", "groups": "groups", "role": "role"},
},
},
None,
id="saml_partial_attribute_mapping",
),
pytest.param(
{
"kind": "oidc",
"spec": {
"issuer": "https://issuer.integration.test",
"clientId": "client-id",
"clientSecret": "client-secret",
},
},
None,
{
"kind": "oidc",
"spec": {
"issuer": "https://issuer.integration.test",
"issuerAlias": "",
"clientId": "client-id",
"clientSecret": "client-secret",
"claimMapping": {"email": "email", "name": "name", "groups": "groups", "role": "role"},
"insecureSkipEmailVerified": False,
"getUserInfo": False,
},
},
None,
id="oidc_minimal",
),
pytest.param(
{
"kind": "oidc",
"spec": {
"issuer": "https://issuer.integration.test",
"issuerAlias": "https://alias.integration.test",
"clientId": "client-id",
"clientSecret": "client-secret",
"claimMapping": {"email": "eml", "name": "nm", "groups": "grps", "role": "rl"},
"insecureSkipEmailVerified": True,
"getUserInfo": True,
},
},
None,
{
"kind": "oidc",
"spec": {
"issuer": "https://issuer.integration.test",
"issuerAlias": "https://alias.integration.test",
"clientId": "client-id",
"clientSecret": "client-secret",
"claimMapping": {"email": "eml", "name": "nm", "groups": "grps", "role": "rl"},
"insecureSkipEmailVerified": True,
"getUserInfo": True,
},
},
None,
id="oidc_full",
),
pytest.param(
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
},
},
{
"defaultRole": "EDITOR",
"groupMappings": {"platform-team": "ADMIN"},
"useRoleAttribute": False,
},
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
"insecureSkipAuthNRequestsSigned": False,
"attributeMapping": {"email": "email", "name": "name", "groups": "groups", "role": "role"},
},
},
{
"defaultRole": "signoz-editor",
"groupMappings": {"platform-team": "signoz-admin"},
"useRoleAttribute": False,
},
id="role_mapping_names_normalized",
),
pytest.param(
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
},
},
{"defaultRole": "VIEWER", "useRoleAttribute": True},
{
"kind": "saml",
"spec": {
"entityId": "saml-entity",
"location": "https://idp.integration.test/sso",
"certificate": "saml-cert",
"insecureSkipAuthNRequestsSigned": False,
"attributeMapping": {"email": "email", "name": "name", "groups": "groups", "role": "role"},
},
},
{"defaultRole": "signoz-viewer", "groupMappings": None, "useRoleAttribute": True},
id="role_mapping_null_group_mappings",
),
],
)
def test_domain_roundtrip( # pylint: disable=too-many-arguments,too-many-positional-arguments
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
config: dict,
role_mapping: dict | None,
expected_config: dict,
expected_role_mapping: dict | None,
):
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# Drop a same-named leftover so reruns against a reused stack stay green.
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
for domain in response.json()["data"]:
if domain["name"] == "roundtrip.integration.test":
response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "roundtrip.integration.test",
"enabled": True,
"config": config,
"roleMapping": role_mapping,
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.CREATED
domain_id = response.json()["data"]["id"]
# Clients (e.g. the terraform provider) read state back with a follow-up
# GET after every write, so posted values must round-trip exactly; the
# server-side defaulting and role-name normalization pinned here are part
# of that contract.
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
data = response.json()["data"]
assert data["name"] == "roundtrip.integration.test"
assert data["enabled"] is True
assert data["config"] == expected_config
assert data["roleMapping"] == expected_role_mapping
response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain_id}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT

View File

@@ -50,16 +50,16 @@ def test_create_auth_domain(
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "saml.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": settings["entityID"],
"samlIdp": settings["singleSignOnServiceLocation"],
"samlCert": settings["certificate"],
"kind": "saml",
"spec": {
"entityId": settings["entityID"],
"location": settings["singleSignOnServiceLocation"],
"certificate": settings["certificate"],
},
},
},
@@ -71,7 +71,7 @@ def test_create_auth_domain(
# Get the domains from signoz
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
@@ -174,30 +174,30 @@ def test_saml_update_domain_with_group_mappings(
# update the existing saml domain to have role mappings also
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/domains/{domain['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
json={
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": settings["entityID"],
"samlIdp": settings["singleSignOnServiceLocation"],
"samlCert": settings["certificate"],
"kind": "saml",
"spec": {
"entityId": settings["entityID"],
"location": settings["singleSignOnServiceLocation"],
"certificate": settings["certificate"],
"attributeMapping": {
"name": "givenName",
"groups": "groups",
"role": "signoz_role",
},
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
"signoz-viewers": "VIEWER",
},
"useRoleAttribute": False,
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
"signoz-viewers": "VIEWER",
},
"useRoleAttribute": False,
},
},
headers={"Authorization": f"Bearer {admin_token}"},
@@ -216,9 +216,6 @@ def test_saml_role_mapping_single_group_admin(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""
Test: User in 'signoz-admins' group gets ADMIN role.
"""
email = "admin-group-user@saml.integration.test"
create_user_idp_with_groups(email, "password", True, ["signoz-admins"])
@@ -239,9 +236,6 @@ def test_saml_role_mapping_single_group_editor(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""
Test: User in 'signoz-editors' group gets EDITOR role.
"""
email = "editor-group-user@saml.integration.test"
create_user_idp_with_groups(email, "password", True, ["signoz-editors"])
@@ -311,9 +305,6 @@ def test_saml_role_mapping_unmapped_group_uses_default(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""
Test: User in unmapped group falls back to default role (VIEWER).
"""
email = "unmapped-group-user@saml.integration.test"
create_user_idp_with_groups(email, "password", True, ["some-other-group"])
@@ -338,29 +329,29 @@ def test_saml_update_domain_with_use_role_claim(
settings = get_saml_settings()
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/domains/{domain['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
json={
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "saml",
"samlConfig": {
"samlEntity": settings["entityID"],
"samlIdp": settings["singleSignOnServiceLocation"],
"samlCert": settings["certificate"],
"kind": "saml",
"spec": {
"entityId": settings["entityID"],
"location": settings["singleSignOnServiceLocation"],
"certificate": settings["certificate"],
"attributeMapping": {
"name": "displayName",
"groups": "groups",
"role": "signoz_role",
},
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
},
"useRoleAttribute": True,
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
},
"useRoleAttribute": True,
},
},
headers={"Authorization": f"Bearer {admin_token}"},
@@ -462,7 +453,6 @@ def test_saml_name_mapping(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""Test that user's display name is mapped from SAML displayName attribute."""
email = "named-user@saml.integration.test"
create_user_idp(email, "password", True, "Jane", "Smith")
@@ -485,7 +475,6 @@ def test_saml_empty_name_fallback(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""Test that user without displayName in IDP still gets created."""
email = "no-name@saml.integration.test"
create_user_idp(email, "password", True)

View File

@@ -26,9 +26,6 @@ def test_apply_license(
make_http_mocks: Callable[[TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
) -> None:
"""
This applies a license to the signoz instance.
"""
add_license(signoz, make_http_mocks, get_token)
@@ -40,9 +37,6 @@ def test_create_auth_domain(
create_user_admin: Callable[[], None], # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
"""
This creates an OIDC auth domain in signoz.
"""
client_id = f"oidc.integration.test.{signoz.self.host_configs['8080'].address}:{signoz.self.host_configs['8080'].port}"
# Create a saml client in the idp.
create_oidc_client(client_id, "/api/v1/complete/oidc")
@@ -54,13 +48,13 @@ def test_create_auth_domain(
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/domains"),
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": "oidc.integration.test",
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "oidc",
"oidcConfig": {
"kind": "oidc",
"spec": {
"clientId": settings["client_id"],
"clientSecret": settings["client_secret"],
# Change the hostname of the issuer to the internal resolvable hostname of the idp
@@ -121,21 +115,18 @@ def test_oidc_update_domain_with_group_mappings(
get_token: Callable[[str, str], str],
get_oidc_settings: Callable[[str], dict],
) -> None:
"""
Updates OIDC domain to add role mapping with group mappings and claim mapping.
"""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
domain = get_oidc_domain(signoz, admin_token)
client_id = f"oidc.integration.test.{signoz.self.host_configs['8080'].address}:{signoz.self.host_configs['8080'].port}"
settings = get_oidc_settings(client_id)
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/domains/{domain['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
json={
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "oidc",
"oidcConfig": {
"kind": "oidc",
"spec": {
"clientId": settings["client_id"],
"clientSecret": settings["client_secret"],
"issuer": f"{idp.container.container_configs['6060'].get(urlparse(settings['issuer']).path)}",
@@ -148,15 +139,15 @@ def test_oidc_update_domain_with_group_mappings(
"role": "signoz_role",
},
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
"signoz-viewers": "VIEWER",
},
"useRoleAttribute": False,
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
"signoz-viewers": "VIEWER",
},
"useRoleAttribute": False,
},
},
headers={"Authorization": f"Bearer {admin_token}"},
@@ -175,9 +166,6 @@ def test_oidc_role_mapping_single_group_admin(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""
Test: OIDC user in 'signoz-admins' group gets ADMIN role.
"""
email = "admin-group-user@oidc.integration.test"
create_user_idp_with_groups(email, "password123", True, ["signoz-admins"])
@@ -198,9 +186,6 @@ def test_oidc_role_mapping_single_group_editor(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""
Test: OIDC user in 'signoz-editors' group gets EDITOR role.
"""
email = "editor-group-user@oidc.integration.test"
create_user_idp_with_groups(email, "password123", True, ["signoz-editors"])
@@ -270,9 +255,6 @@ def test_oidc_role_mapping_unmapped_group_uses_default(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], str],
) -> None:
"""
Test: OIDC user in unmapped group falls back to default role.
"""
email = "unmapped-group-user@oidc.integration.test"
create_user_idp_with_groups(email, "password123", True, ["some-other-group"])
@@ -290,21 +272,18 @@ def test_oidc_update_domain_with_use_role_claim(
get_token: Callable[[str, str], str],
get_oidc_settings: Callable[[str], dict],
) -> None:
"""
Updates OIDC domain to enable useRoleClaim.
"""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
domain = get_oidc_domain(signoz, admin_token)
client_id = f"oidc.integration.test.{signoz.self.host_configs['8080'].address}:{signoz.self.host_configs['8080'].port}"
settings = get_oidc_settings(client_id)
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v1/domains/{domain['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
json={
"enabled": True,
"config": {
"ssoEnabled": True,
"ssoType": "oidc",
"oidcConfig": {
"kind": "oidc",
"spec": {
"clientId": settings["client_id"],
"clientSecret": settings["client_secret"],
"issuer": f"{idp.container.container_configs['6060'].get(urlparse(settings['issuer']).path)}",
@@ -317,14 +296,14 @@ def test_oidc_update_domain_with_use_role_claim(
"role": "signoz_role",
},
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
},
"useRoleAttribute": True,
},
"roleMapping": {
"defaultRole": "VIEWER",
"groupMappings": {
"signoz-admins": "ADMIN",
"signoz-editors": "EDITOR",
},
"useRoleAttribute": True,
},
},
headers={"Authorization": f"Bearer {admin_token}"},
@@ -424,7 +403,6 @@ def test_oidc_name_mapping(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], dict],
) -> None:
"""Test that user's display name is mapped from IDP name claim."""
email = "named-user@oidc.integration.test"
# Create user with explicit first/last name
@@ -449,7 +427,6 @@ def test_oidc_empty_name_uses_fallback(
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], dict],
) -> None:
"""Test that user without name in IDP still gets created (may have empty displayName)."""
email = "no-name@oidc.integration.test"
# Create user without first/last name

View File

@@ -0,0 +1,205 @@
from collections.abc import Callable
from http import HTTPStatus
import requests
from wiremock.resources.mappings import Mapping
from fixtures import types
from fixtures.auth import (
USER_ADMIN_EMAIL,
USER_ADMIN_PASSWORD,
assert_user_has_role,
find_user_with_roles_by_email,
)
from fixtures.googleidp import google_oidc_mappings, perform_google_login
from fixtures.types import Operation, SigNoz
GOOGLE_DOMAIN = "google.integration.test"
GOOGLE_CLIENT_ID = "google-client-id.apps.googleusercontent.com"
GOOGLE_CLIENT_SECRET = "google-client-secret"
def test_create_auth_domain(
signoz: SigNoz,
create_user_admin: Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# Reruns against a reused stack find the domain from the previous run;
# drop it so creation always starts from a clean slate.
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
for domain in response.json()["data"]:
if domain["name"] == GOOGLE_DOMAIN:
response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
json={
"name": GOOGLE_DOMAIN,
"enabled": True,
"config": {
"kind": "google",
"spec": {
"clientId": GOOGLE_CLIENT_ID,
"clientSecret": GOOGLE_CLIENT_SECRET,
},
},
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.CREATED
def test_google_authn(
signoz: SigNoz,
googleidp: types.TestContainerDocker,
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], dict],
) -> None:
email = "viewer@google.integration.test"
make_http_mocks(googleidp, google_oidc_mappings(email=email, name="Google Viewer", hd=GOOGLE_DOMAIN, audience=GOOGLE_CLIENT_ID))
redirect_url = perform_google_login(signoz, googleidp, get_session_context, email)
assert "accessToken=" in redirect_url
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
found_user = find_user_with_roles_by_email(signoz, admin_token, email)
assert found_user["displayName"] == "Google Viewer"
assert_user_has_role(found_user, "signoz-viewer")
def test_google_authn_hd_mismatch(
signoz: SigNoz,
googleidp: types.TestContainerDocker,
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], dict],
) -> None:
# The id_token carries a hosted-domain claim for a different workspace than
# the auth domain; the callback must reject it and provision no user.
email = "intruder@google.integration.test"
make_http_mocks(googleidp, google_oidc_mappings(email=email, name="Intruder", hd="other.workspace.test", audience=GOOGLE_CLIENT_ID))
redirect_url = perform_google_login(signoz, googleidp, get_session_context, email)
assert "callbackauthnerr" in redirect_url
assert "accessToken=" not in redirect_url
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/users"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK
assert not any(user["email"] == email for user in response.json()["data"])
def test_google_authn_unverified_email(
signoz: SigNoz,
googleidp: types.TestContainerDocker,
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], dict],
) -> None:
email = "unverified@google.integration.test"
make_http_mocks(googleidp, google_oidc_mappings(email=email, name="Unverified", hd=GOOGLE_DOMAIN, audience=GOOGLE_CLIENT_ID, email_verified=False))
redirect_url = perform_google_login(signoz, googleidp, get_session_context, email)
assert "callbackauthnerr" in redirect_url
# Opting the domain into insecureSkipEmailVerified must let the same
# unverified identity through.
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
domain = next(domain for domain in response.json()["data"] if domain["name"] == GOOGLE_DOMAIN)
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
json={
"enabled": True,
"config": {
"kind": "google",
"spec": {
"clientId": GOOGLE_CLIENT_ID,
"clientSecret": GOOGLE_CLIENT_SECRET,
"insecureSkipEmailVerified": True,
},
},
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT
redirect_url = perform_google_login(signoz, googleidp, get_session_context, email)
assert "accessToken=" in redirect_url
found_user = find_user_with_roles_by_email(signoz, admin_token, email)
assert_user_has_role(found_user, "signoz-viewer")
def test_google_role_mapping_default_role(
signoz: SigNoz,
googleidp: types.TestContainerDocker,
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
get_session_context: Callable[[str], dict],
) -> None:
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v2/auth_domains"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
domain = next(domain for domain in response.json()["data"] if domain["name"] == GOOGLE_DOMAIN)
response = requests.put(
signoz.self.host_configs["8080"].get(f"/api/v2/auth_domains/{domain['id']}"),
json={
"enabled": True,
"config": {
"kind": "google",
"spec": {
"clientId": GOOGLE_CLIENT_ID,
"clientSecret": GOOGLE_CLIENT_SECRET,
},
},
"roleMapping": {
"defaultRole": "EDITOR",
},
},
headers={"Authorization": f"Bearer {admin_token}"},
timeout=2,
)
assert response.status_code == HTTPStatus.NO_CONTENT
email = "editor@google.integration.test"
make_http_mocks(googleidp, google_oidc_mappings(email=email, name="Google Editor", hd=GOOGLE_DOMAIN, audience=GOOGLE_CLIENT_ID))
redirect_url = perform_google_login(signoz, googleidp, get_session_context, email)
assert "accessToken=" in redirect_url
found_user = find_user_with_roles_by_email(signoz, admin_token, email)
assert_user_has_role(found_user, "signoz-editor")

View File

@@ -25,7 +25,6 @@ def test_create_account(
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
) -> None:
"""Test creating a new cloud integration account for AWS."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
cloud_provider = "aws"
@@ -54,7 +53,6 @@ def test_create_account_unsupported_provider(
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
"""Test that creating an account with an unsupported cloud provider returns 400."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
cloud_provider = "unknown"
endpoint = f"/api/v1/cloud_integrations/{cloud_provider}/accounts"

View File

@@ -28,7 +28,6 @@ def test_agent_check_in(
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
) -> None:
"""Test agent check-in with new camelCase fields returns 200 with expected response shape."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, CLOUD_PROVIDER, regions=["us-east-1"])
@@ -71,7 +70,6 @@ def test_agent_check_in_account_not_found(
create_user_admin: types.Operation, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
) -> None:
"""Test that check-in with an unknown cloudIntegrationId returns 404."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
fake_id = str(uuid.uuid4())

View File

@@ -18,9 +18,6 @@ def test_apply_license(
make_http_mocks: Callable[[TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
) -> None:
"""
This applies a license to the signoz instance.
"""
add_license(signoz, make_http_mocks, get_token)

View File

@@ -16,9 +16,6 @@ def test_apply_license(
make_http_mocks: Callable[[TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
) -> None:
"""
This applies a license to the signoz instance.
"""
add_license(signoz, make_http_mocks, get_token)

View File

@@ -6,33 +6,11 @@ import pytest
import requests
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.dashboards import delete_all_dashboards
from fixtures.metrics import Metrics
from fixtures.types import Operation, SigNoz
BASE_URL = "/api/v2/dashboards"
# MaxListLimit caps a single list page, so wiping a shared DB has to drain pages
# until the list comes back empty.
MAX_LIST_LIMIT = 200
def _wipe_all_dashboards(signoz: SigNoz, token: str) -> None:
while True:
response = requests.get(
signoz.self.host_configs["8080"].get(f"{BASE_URL}?limit={MAX_LIST_LIMIT}"),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK, response.text
dashboards = response.json()["data"]["dashboards"]
if not dashboards:
return
for dashboard in dashboards:
del_res = requests.delete(
signoz.self.host_configs["8080"].get(f"{BASE_URL}/{dashboard['id']}"),
headers={"Authorization": f"Bearer {token}"},
timeout=5,
)
assert del_res.status_code == HTTPStatus.NO_CONTENT, del_res.text
# ─── failure cases (create no dashboards) ────────────────────────────────────
@@ -632,7 +610,7 @@ def test_dashboard_v2_lifecycle( # pylint: disable=too-many-locals,too-many-sta
# runs, so start from a clean slate: delete every dashboard (which also clears
# pins via the delete cascade). This test then owns the whole dashboard space
# and asserts on global counts.
_wipe_all_dashboards(signoz, token)
delete_all_dashboards(signoz, token)
dashboard_requests = [
(
@@ -1276,7 +1254,7 @@ def test_dashboard_v2_pin_limit(
# Wipe the dashboard space (see lifecycle) so the per-user pin cap this test
# asserts against starts empty — deleting dashboards clears their pins.
_wipe_all_dashboards(signoz, token)
delete_all_dashboards(signoz, token)
ids: list[str] = []
for i in range(max_pinned + 1):
@@ -1369,7 +1347,7 @@ def test_dashboard_v2_like_escaping(
# Wipe the dashboard space (see lifecycle) so the filter assertions run
# against only the dashboards this test creates.
_wipe_all_dashboards(signoz, token)
delete_all_dashboards(signoz, token)
dashboard_requests = [
("esc-pct", "Cost 50% Report"),
@@ -1447,7 +1425,7 @@ def test_dashboard_v2_get_by_metric_name(
the metric appears only in panel names (the prefilter matches but the parse
rejects it)."""
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
_wipe_all_dashboards(signoz, token)
delete_all_dashboards(signoz, token)
target_metric = "system.network.dropped"
decoy_metric = "system.network.io"

View File

@@ -1,5 +1,3 @@
"""Integration tests for v2 infra-monitoring host endpoints."""
import json
from datetime import UTC, datetime, timedelta
from http import HTTPStatus

View File

@@ -1,5 +1,3 @@
"""Integration tests for v2 infra-monitoring pod endpoints."""
import json
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
@@ -13,51 +11,14 @@ from fixtures.fs import get_testdata_file_path
from fixtures.inframonitoring import STATUS_BUCKETS, STATUS_TO_BUCKET
from fixtures.metrics import Metrics
from fixtures.querier import compare_values, get_all_warnings
from fixtures.time import parse_timestamp
ENDPOINT = "/api/v2/infra_monitoring/pods"
# Placeholder in JSONL labels that gets substituted with a runtime ISO string.
# Placeholder in JSONL labels that gets substituted with a runtime ISO string,
# keeping podAge deterministic across runs.
START_TIME_PLACEHOLDER = "__START_TIME__"
def _load_pods_metrics(
file_relpath: str,
base_time: datetime,
start_time: datetime | None = None,
) -> list[Metrics]:
"""Load pod metrics JSONL with optional k8s.pod.start_time substitution.
Mirrors Metrics.load_from_file's base_time rebase logic but adds a hook
for the start_time label. Lines carrying ``k8s.pod.start_time =
__START_TIME__`` get rewritten to ``start_time.isoformat()`` before
construction, ensuring podAge is deterministic across runs.
"""
path = get_testdata_file_path(file_relpath)
start_time_iso = start_time.isoformat() if start_time else None
rows = []
with open(path, encoding="utf-8") as f:
for line in f:
line = line.strip()
if not line:
continue
data = json.loads(line)
labels = data.get("labels", {})
if start_time_iso and labels.get("k8s.pod.start_time") == START_TIME_PLACEHOLDER:
labels["k8s.pod.start_time"] = start_time_iso
rows.append(data)
if not rows:
return []
earliest = min(parse_timestamp(r["timestamp"]) for r in rows)
offset = base_time - earliest
metrics = []
for r in rows:
ts = parse_timestamp(r["timestamp"]) + offset
r["timestamp"] = ts.isoformat()
metrics.append(Metrics.from_dict(r))
return metrics
def test_pods_accuracy(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
@@ -70,10 +31,10 @@ def test_pods_accuracy(
now = datetime.now(tz=UTC).replace(microsecond=0)
start_time = now - timedelta(minutes=10)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_value_accuracy.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_value_accuracy.jsonl"),
base_time=now - timedelta(minutes=4),
start_time=start_time,
label_substitutions={START_TIME_PLACEHOLDER: start_time.isoformat()},
)
)
@@ -200,8 +161,8 @@ def test_pods_warnings(
once, for hosts, in 01_hosts.py.)"""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
f"inframonitoring/{case['dataset']}",
Metrics.load_from_file(
get_testdata_file_path(f"inframonitoring/{case['dataset']}"),
base_time=now - timedelta(minutes=4),
)
)
@@ -308,8 +269,8 @@ def test_pods_filter(
}
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_filter_dataset.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_filter_dataset.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -356,8 +317,8 @@ def test_pods_filter_invalid(
400 invalid_input with structured errors."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_filter_dataset.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_filter_dataset.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -404,8 +365,8 @@ def test_pods_groupby(
"""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_groupby.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_groupby.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -455,8 +416,8 @@ def test_pods_pagination(
it returns empty records while total still reflects dataset size."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_pagination.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_pagination.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -519,8 +480,8 @@ def test_pods_orderby( # pylint: disable=too-many-arguments,too-many-positional
sort) and records come back sorted by the requested column."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_orderby.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_orderby.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -665,8 +626,8 @@ def test_pods_status_list_mode(
"""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_phases.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_phases.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -721,8 +682,8 @@ def test_pods_restarts_list_mode(
series -> -1 no-data sentinel (kubectl would show 0)."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_phases.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_phases.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -760,8 +721,8 @@ def test_pods_status_latest_wins(
ignored via argMax-by-latest-timestamp per (pod, container, reason)."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_phases_transition.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_phases_transition.jsonl"),
base_time=now - timedelta(minutes=8),
)
)
@@ -797,8 +758,8 @@ def test_pods_restarts_latest_wins(
not be double-counted."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_phases_transition.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_phases_transition.jsonl"),
base_time=now - timedelta(minutes=8),
)
)
@@ -834,8 +795,8 @@ def test_pods_status_grouped_mode(
g-fail-1 Error, g-fail-2 Evicted, g-pend-1 Pending."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_phases_grouped.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_phases_grouped.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -888,8 +849,8 @@ def test_pods_restarts_grouped_mode(
In ns-mixed only g-run-2 has restarts (3); all others 0 -> group total 3."""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_phases_grouped.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_phases_grouped.jsonl"),
base_time=now - timedelta(minutes=4),
)
)
@@ -932,8 +893,8 @@ def test_pods_status_missing_metric_warning(
seeds only k8s.pod.cpu.usage.)"""
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
_load_pods_metrics(
"inframonitoring/pods_missing_metrics.jsonl",
Metrics.load_from_file(
get_testdata_file_path("inframonitoring/pods_missing_metrics.jsonl"),
base_time=now - timedelta(minutes=4),
)
)

Some files were not shown because too many files have changed in this diff Show More