Compare commits

..

61 Commits

Author SHA1 Message Date
swapnil-signoz
42300bd0a0 Merge branch 'main' into feat/cloudintegration-module 2026-03-23 20:57:18 +05:30
swapnil-signoz
6e52f2c8f0 Merge branch 'refactor/cloud-integration-impl-store' into refactor/cloud-integration-handlers 2026-03-22 17:13:53 +05:30
swapnil-signoz
d9f8a4ae5a Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-22 17:13:40 +05:30
swapnil-signoz
eefe3edffd Merge branch 'main' into refactor/cloud-integration-handlers 2026-03-22 17:13:02 +05:30
swapnil-signoz
2051861a03 feat: adding handler skeleton 2026-03-22 17:12:35 +05:30
swapnil-signoz
4b01a40fb9 Merge branch 'refactor/cloud-integration-impl-store' into refactor/cloud-integration-handlers 2026-03-20 20:53:54 +05:30
swapnil-signoz
2d8a00bf18 fix: update error code for service not found 2026-03-20 20:53:33 +05:30
swapnil-signoz
f1b26b310f Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-20 20:51:44 +05:30
swapnil-signoz
2c438b6c32 Merge branch 'refactor/cloud-integration-impl-store' into refactor/cloud-integration-handlers 2026-03-20 20:48:34 +05:30
swapnil-signoz
1814c2d13c Merge branch 'main' into refactor/cloud-integration-handlers 2026-03-20 17:52:31 +05:30
swapnil-signoz
e6cd771f11 Merge origin/main into refactor/cloud-integration-handlers 2026-03-20 16:46:36 +05:30
swapnil-signoz
6b94f87ca0 Merge branch 'main' into refactor/cloud-integration-handlers 2026-03-19 11:43:21 +05:30
swapnil-signoz
bf315253ae fix: lint issues 2026-03-19 11:43:09 +05:30
swapnil-signoz
668ff7bc39 fix: lint and ci issues 2026-03-19 11:34:27 +05:30
swapnil-signoz
07f2aa52fd feat: adding handlers 2026-03-19 01:35:01 +05:30
swapnil-signoz
3416b3ad55 Merge branch 'main' into refactor/cloud-integration-handlers 2026-03-18 21:50:40 +05:30
swapnil-signoz
d6caa4f2c7 Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-18 14:08:14 +05:30
swapnil-signoz
f86371566d refactor: clean up 2026-03-18 13:45:31 +05:30
swapnil-signoz
9115803084 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-18 13:42:43 +05:30
swapnil-signoz
0c14d8f966 refactor: review comments 2026-03-18 13:40:17 +05:30
swapnil-signoz
7afb461af8 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-18 11:14:33 +05:30
swapnil-signoz
a21fbb4ee0 refactor: clean up 2026-03-18 11:14:05 +05:30
swapnil-signoz
0369842f3d refactor: clean up 2026-03-17 23:40:14 +05:30
swapnil-signoz
59cd96562a Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 23:10:54 +05:30
swapnil-signoz
cc4475cab7 refactor: updating store methods 2026-03-17 23:10:15 +05:30
swapnil-signoz
ac8c648420 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 21:09:47 +05:30
swapnil-signoz
bede6be4b8 feat: adding method for service id creation 2026-03-17 21:09:26 +05:30
swapnil-signoz
dd3d60e6df Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 20:49:31 +05:30
swapnil-signoz
538ab686d2 refactor: using serviceID type 2026-03-17 20:49:17 +05:30
swapnil-signoz
936a325cb9 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 17:25:58 +05:30
swapnil-signoz
c6cdcd0143 refactor: renaming service type to service id 2026-03-17 17:25:29 +05:30
swapnil-signoz
cd9211d718 refactor: clean up types 2026-03-17 17:04:27 +05:30
swapnil-signoz
0601c28782 feat: adding integration test 2026-03-17 11:02:46 +05:30
swapnil-signoz
580610dbfa Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-16 23:02:19 +05:30
swapnil-signoz
2d2aa02a81 refactor: split upsert store method 2026-03-16 18:27:42 +05:30
swapnil-signoz
dd9723ad13 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-16 17:42:03 +05:30
swapnil-signoz
3651469416 Merge branch 'main' of https://github.com/SigNoz/signoz into refactor/cloud-integration-types 2026-03-16 17:41:52 +05:30
swapnil-signoz
febce75734 refactor: update Dashboard struct comments and remove unused fields 2026-03-16 17:41:28 +05:30
swapnil-signoz
e1616f3487 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-16 17:36:15 +05:30
swapnil-signoz
4b94287ac7 refactor: add comments for backward compatibility in PostableAgentCheckInRequest 2026-03-16 15:48:20 +05:30
swapnil-signoz
1575c7c54c refactor: streamlining types 2026-03-16 15:39:32 +05:30
swapnil-signoz
8def3f835b refactor: adding comments and removed wrong code 2026-03-16 11:10:53 +05:30
swapnil-signoz
11ed15f4c5 feat: implement cloud integration store 2026-03-14 17:05:02 +05:30
swapnil-signoz
f47877cca9 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-14 17:01:51 +05:30
swapnil-signoz
bb2b9215ba fix: correct GetService signature and remove shadowed Data field 2026-03-14 16:59:07 +05:30
swapnil-signoz
3111904223 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-14 16:36:35 +05:30
swapnil-signoz
003e2c30d8 Merge branch 'main' into refactor/cloud-integration-types 2026-03-14 16:25:35 +05:30
swapnil-signoz
00fe516d10 refactor: update cloud integration types and module interface 2026-03-14 16:25:16 +05:30
swapnil-signoz
0305f4f7db refactor: using struct for map 2026-03-13 16:09:26 +05:30
swapnil-signoz
c60019a6dc Merge branch 'main' into refactor/cloud-integration-types 2026-03-12 23:41:22 +05:30
swapnil-signoz
acde2a37fa feat: adding updated types for cloud integration 2026-03-12 23:40:44 +05:30
swapnil-signoz
945241a52a Merge branch 'main' into refactor/cloud-integration-types 2026-03-12 19:45:50 +05:30
swapnil-signoz
e967f80c86 Merge branch 'main' into refactor/cloud-integration-types 2026-03-02 16:39:42 +05:30
swapnil-signoz
a09dc325de Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-02 16:39:20 +05:30
swapnil-signoz
379b4f7fc4 refactor: removing interface check 2026-03-02 14:50:37 +05:30
swapnil-signoz
5e536ae077 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-02 14:49:35 +05:30
swapnil-signoz
234585e642 Merge branch 'main' into refactor/cloud-integration-types 2026-03-02 14:49:19 +05:30
swapnil-signoz
2cc14f1ad4 Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-02 14:49:00 +05:30
swapnil-signoz
dc4ed4d239 feat: adding sql store implementation 2026-03-02 14:44:56 +05:30
swapnil-signoz
7281c36873 refactor: store interfaces to use local types and error 2026-03-02 13:27:46 +05:30
swapnil-signoz
40288776e8 feat: adding cloud integration type for refactor 2026-02-28 16:59:14 +05:30
2355 changed files with 84576 additions and 178562 deletions

View File

@@ -27,8 +27,8 @@ services:
- ${PWD}/fs/tmp/var/lib/clickhouse/user_scripts/:/var/lib/clickhouse/user_scripts/
- ${PWD}/../../../deploy/common/clickhouse/custom-function.xml:/etc/clickhouse-server/custom-function.xml
ports:
- "127.0.0.1:8123:8123"
- "127.0.0.1:9000:9000"
- '127.0.0.1:8123:8123'
- '127.0.0.1:9000:9000'
tty: true
healthcheck:
test:
@@ -47,16 +47,13 @@ services:
condition: service_healthy
environment:
- CLICKHOUSE_SKIP_USER_SETUP=1
networks:
- default
- signoz-devenv
zookeeper:
image: signoz/zookeeper:3.7.1
container_name: zookeeper
volumes:
- ${PWD}/fs/tmp/zookeeper:/bitnami/zookeeper
ports:
- "127.0.0.1:2181:2181"
- '127.0.0.1:2181:2181'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
healthcheck:
@@ -77,19 +74,12 @@ services:
entrypoint:
- /bin/sh
command:
- -c
- |
/signoz-otel-collector migrate bootstrap &&
/signoz-otel-collector migrate sync up &&
/signoz-otel-collector migrate async up
- -c
- |
/signoz-otel-collector migrate bootstrap &&
/signoz-otel-collector migrate sync up &&
/signoz-otel-collector migrate async up
depends_on:
clickhouse:
condition: service_healthy
restart: on-failure
networks:
- default
- signoz-devenv
networks:
signoz-devenv:
name: signoz-devenv

View File

@@ -3,7 +3,7 @@ services:
image: signoz/signoz-otel-collector:v0.142.0
container_name: signoz-otel-collector-dev
entrypoint:
- /bin/sh
- /bin/sh
command:
- -c
- |
@@ -34,11 +34,4 @@ services:
retries: 3
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- default
- signoz-devenv
networks:
signoz-devenv:
name: signoz-devenv
- "host.docker.internal:host-gateway"

View File

@@ -12,10 +12,10 @@ receivers:
scrape_configs:
- job_name: otel-collector
static_configs:
- targets:
- localhost:8888
labels:
job_name: otel-collector
- targets:
- localhost:8888
labels:
job_name: otel-collector
processors:
batch:
@@ -29,26 +29,7 @@ processors:
signozspanmetrics/delta:
metrics_exporter: signozclickhousemetrics
metrics_flush_interval: 60s
latency_histogram_buckets:
[
100us,
1ms,
2ms,
6ms,
10ms,
50ms,
100ms,
250ms,
500ms,
1000ms,
1400ms,
2000ms,
5s,
10s,
20s,
40s,
60s,
]
latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 50ms, 100ms, 250ms, 500ms, 1000ms, 1400ms, 2000ms, 5s, 10s, 20s, 40s, 60s ]
dimensions_cache_size: 100000
aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA
enable_exp_histogram: true
@@ -79,13 +60,13 @@ extensions:
exporters:
clickhousetraces:
datasource: tcp://clickhouse:9000/signoz_traces
datasource: tcp://host.docker.internal:9000/signoz_traces
low_cardinal_exception_grouping: ${env:LOW_CARDINAL_EXCEPTION_GROUPING}
use_new_schema: true
signozclickhousemetrics:
dsn: tcp://clickhouse:9000/signoz_metrics
dsn: tcp://host.docker.internal:9000/signoz_metrics
clickhouselogsexporter:
dsn: tcp://clickhouse:9000/signoz_logs
dsn: tcp://host.docker.internal:9000/signoz_logs
timeout: 10s
use_new_schema: true
@@ -112,4 +93,4 @@ service:
logs:
receivers: [otlp]
processors: [batch]
exporters: [clickhouselogsexporter]
exporters: [clickhouselogsexporter]

76
.github/CODEOWNERS vendored
View File

@@ -10,45 +10,44 @@
/frontend/src/container/OnboardingV2Container/AddDataSource/AddDataSource.tsx @makeavish
# CI
/deploy/ @therealpandey
.github @therealpandey
go.mod @therealpandey
# Scaffold Owners
/pkg/config/ @therealpandey
/pkg/errors/ @therealpandey
/pkg/factory/ @therealpandey
/pkg/types/ @therealpandey
/pkg/valuer/ @therealpandey
/cmd/ @therealpandey
.golangci.yml @therealpandey
/pkg/config/ @vikrantgupta25
/pkg/errors/ @vikrantgupta25
/pkg/factory/ @vikrantgupta25
/pkg/types/ @vikrantgupta25
/pkg/valuer/ @vikrantgupta25
/cmd/ @vikrantgupta25
.golangci.yml @vikrantgupta25
# Zeus Owners
/pkg/zeus/ @therealpandey
/ee/zeus/ @therealpandey
/pkg/licensing/ @therealpandey
/ee/licensing/ @therealpandey
/pkg/zeus/ @vikrantgupta25
/ee/zeus/ @vikrantgupta25
/pkg/licensing/ @vikrantgupta25
/ee/licensing/ @vikrantgupta25
# SQL Owners
/pkg/sqlmigration/ @therealpandey
/ee/sqlmigration/ @therealpandey
/pkg/sqlschema/ @therealpandey
/ee/sqlschema/ @therealpandey
/pkg/sqlmigration/ @vikrantgupta25
/ee/sqlmigration/ @vikrantgupta25
/pkg/sqlschema/ @vikrantgupta25
/ee/sqlschema/ @vikrantgupta25
# Analytics Owners
/pkg/analytics/ @therealpandey
/pkg/statsreporter/ @therealpandey
/pkg/analytics/ @vikrantgupta25
/pkg/statsreporter/ @vikrantgupta25
# Emailing Owners
/pkg/emailing/ @therealpandey
/pkg/types/emailtypes/ @therealpandey
/templates/email/ @therealpandey
/pkg/emailing/ @vikrantgupta25
/pkg/types/emailtypes/ @vikrantgupta25
/templates/email/ @vikrantgupta25
# Querier Owners
@@ -87,8 +86,6 @@ go.mod @therealpandey
/pkg/types/alertmanagertypes @srikanthccv
/pkg/alertmanager/ @srikanthccv
/pkg/ruler/ @srikanthccv
/pkg/modules/rulestatehistory/ @srikanthccv
/pkg/types/rulestatehistorytypes/ @srikanthccv
# Correlation-adjacent
@@ -98,28 +95,23 @@ go.mod @therealpandey
# AuthN / AuthZ Owners
/pkg/authz/ @therealpandey
/ee/authz/ @therealpandey
/pkg/authn/ @therealpandey
/ee/authn/ @therealpandey
/pkg/modules/user/ @therealpandey
/pkg/modules/session/ @therealpandey
/pkg/modules/organization/ @therealpandey
/pkg/modules/authdomain/ @therealpandey
/pkg/modules/role/ @therealpandey
/pkg/authz/ @vikrantgupta25
/ee/authz/ @vikrantgupta25
/pkg/authn/ @vikrantgupta25
/ee/authn/ @vikrantgupta25
/pkg/modules/user/ @vikrantgupta25
/pkg/modules/session/ @vikrantgupta25
/pkg/modules/organization/ @vikrantgupta25
/pkg/modules/authdomain/ @vikrantgupta25
/pkg/modules/role/ @vikrantgupta25
# IdentN Owners
/pkg/identn/ @therealpandey
/pkg/http/middleware/identn.go @therealpandey
# IdentN Owners
/pkg/identn/ @vikrantgupta25
/pkg/http/middleware/identn.go @vikrantgupta25
# Integration tests
/tests/integration/ @therealpandey
# Flagger Owners
/pkg/flagger/ @therealpandey
/tests/integration/ @vikrantgupta25
# OpenAPI types generator
@@ -140,7 +132,6 @@ go.mod @therealpandey
/frontend/src/container/ListOfDashboard/ @SigNoz/pulse-frontend
# Dashboard Widget Page
/frontend/src/pages/DashboardWidget/ @SigNoz/pulse-frontend
/frontend/src/container/NewWidget/ @SigNoz/pulse-frontend
@@ -156,7 +147,6 @@ go.mod @therealpandey
/frontend/src/container/PublicDashboardContainer/ @SigNoz/pulse-frontend
## Dashboard Libs + Components
/frontend/src/lib/uPlotV2/ @SigNoz/pulse-frontend
/frontend/src/lib/dashboard/ @SigNoz/pulse-frontend
/frontend/src/lib/dashboardVariables/ @SigNoz/pulse-frontend

View File

@@ -1,91 +0,0 @@
name: e2eci
on:
pull_request:
types:
- labeled
pull_request_target:
types:
- labeled
jobs:
fmtlint:
if: |
((github.event_name == 'pull_request' && ! github.event.pull_request.head.repo.fork && github.event.pull_request.user.login != 'dependabot[bot]' && ! contains(github.event.pull_request.labels.*.name, 'safe-to-test')) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test'))) && contains(github.event.pull_request.labels.*.name, 'safe-to-e2e')
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v4
- name: node
uses: actions/setup-node@v4
with:
node-version: lts/*
- name: install
run: |
cd tests/e2e && yarn install --frozen-lockfile
- name: fmt
run: |
cd tests/e2e && yarn fmt:check
- name: lint
run: |
cd tests/e2e && yarn lint
test:
strategy:
fail-fast: false
matrix:
project:
- chromium
if: |
((github.event_name == 'pull_request' && ! github.event.pull_request.head.repo.fork && github.event.pull_request.user.login != 'dependabot[bot]' && ! contains(github.event.pull_request.labels.*.name, 'safe-to-test')) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.labels.*.name, 'safe-to-test'))) && contains(github.event.pull_request.labels.*.name, 'safe-to-e2e')
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: checkout
uses: actions/checkout@v4
- name: python
uses: actions/setup-python@v5
with:
python-version: 3.13
- name: uv
uses: astral-sh/setup-uv@v4
- name: node
uses: actions/setup-node@v4
with:
node-version: lts/*
- name: python-install
run: |
cd tests && uv sync
- name: yarn-install
run: |
cd tests/e2e && yarn install --frozen-lockfile
- name: playwright-browsers
run: |
cd tests/e2e && yarn playwright install --with-deps ${{ matrix.project }}
- name: bring-up-stack
run: |
cd tests && \
uv run pytest \
--basetemp=./tmp/ \
-vv --reuse --with-web \
e2e/bootstrap/setup.py::test_setup
- name: playwright-test
run: |
cd tests/e2e && \
yarn playwright test --project=${{ matrix.project }}
- name: teardown-stack
if: always()
run: |
cd tests && \
uv run pytest \
--basetemp=./tmp/ \
-vv --teardown \
e2e/bootstrap/setup.py::test_teardown
- name: upload-artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: playwright-artifacts-${{ matrix.project }}
path: tests/e2e/artifacts/
retention-days: 5

View File

@@ -25,11 +25,11 @@ jobs:
uses: astral-sh/setup-uv@v4
- name: install
run: |
cd tests && uv sync
cd tests/integration && uv sync
- name: fmt
run: |
make py-fmt
git diff --exit-code -- tests/
git diff --exit-code -- tests/integration/
- name: lint
run: |
make py-lint
@@ -37,26 +37,23 @@ jobs:
strategy:
fail-fast: false
matrix:
suite:
- alerts
src:
- bootstrap
- passwordauthn
- callbackauthn
- cloudintegrations
- dashboard
- ingestionkeys
- logspipelines
- passwordauthn
- preference
- querier
- rawexportdata
- role
- rootuser
- serviceaccount
- ttl
- alerts
- ingestionkeys
- rootuser
sqlstore-provider:
- postgres
- sqlite
sqlite-mode:
- wal
clickhouse-version:
- 25.5.6
- 25.12.5
@@ -79,9 +76,8 @@ jobs:
uses: astral-sh/setup-uv@v4
- name: install
run: |
cd tests && uv sync
cd tests/integration && uv sync
- name: webdriver
if: matrix.suite == 'callbackauthn'
run: |
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
echo "deb http://dl.google.com/linux/chrome/deb/ stable main" | sudo tee -a /etc/apt/sources.list.d/google-chrome.list
@@ -100,12 +96,11 @@ jobs:
google-chrome-stable --version
- name: run
run: |
cd tests && \
cd tests/integration && \
uv run pytest \
--basetemp=./tmp/ \
integration/tests/${{matrix.suite}} \
src/${{matrix.src}} \
--sqlstore-provider ${{matrix.sqlstore-provider}} \
--sqlite-mode ${{matrix.sqlite-mode}} \
--postgres-version ${{matrix.postgres-version}} \
--clickhouse-version ${{matrix.clickhouse-version}} \
--schema-migrator-version ${{matrix.schema-migrator-version}}

62
.github/workflows/run-e2e.yaml vendored Normal file
View File

@@ -0,0 +1,62 @@
name: e2eci
on:
workflow_dispatch:
inputs:
userRole:
description: "Role of the user (ADMIN, EDITOR, VIEWER)"
required: true
type: choice
options:
- ADMIN
- EDITOR
- VIEWER
jobs:
test:
name: Run Playwright Tests
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: lts/*
- name: Mask secrets and input
run: |
echo "::add-mask::${{ secrets.BASE_URL }}"
echo "::add-mask::${{ secrets.LOGIN_USERNAME }}"
echo "::add-mask::${{ secrets.LOGIN_PASSWORD }}"
echo "::add-mask::${{ github.event.inputs.userRole }}"
- name: Install dependencies
working-directory: frontend
run: |
npm install -g yarn
yarn
- name: Install Playwright Browsers
working-directory: frontend
run: yarn playwright install --with-deps
- name: Run Playwright Tests
working-directory: frontend
run: |
BASE_URL="${{ secrets.BASE_URL }}" \
LOGIN_USERNAME="${{ secrets.LOGIN_USERNAME }}" \
LOGIN_PASSWORD="${{ secrets.LOGIN_PASSWORD }}" \
USER_ROLE="${{ github.event.inputs.userRole }}" \
yarn playwright test
- name: Upload Playwright Report
uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
path: frontend/playwright-report/
retention-days: 30

2
.gitignore vendored
View File

@@ -51,8 +51,6 @@ ee/query-service/tests/test-deploy/data/
# local data
*.backup
*.db
*.db-shm
*.db-wal
**/db
/deploy/docker/clickhouse-setup/data/
/deploy/docker-swarm/clickhouse-setup/data/

View File

@@ -6,14 +6,12 @@ linters:
- depguard
- errcheck
- forbidigo
- godot
- govet
- iface
- ineffassign
- misspell
- nilnil
- sloglint
- staticcheck
- wastedassign
- unparam
- unused

39
.vscode/settings.json vendored
View File

@@ -1,22 +1,21 @@
{
"oxc.typeAware": true,
"oxc.tsConfigPath": "./frontend/tsconfig.json",
"editor.formatOnSave": true,
"editor.defaultFormatter": "oxc.oxc-vscode",
"editor.codeActionsOnSave": {
"source.fixAll.oxc": "explicit"
},
"[go]": {
"editor.formatOnSave": true,
"editor.defaultFormatter": "golang.go"
},
"[sql]": {
"editor.defaultFormatter": "adpyke.vscode-sql-formatter"
},
"[html]": {
"editor.defaultFormatter": "vscode.html-language-features"
},
"python-envs.defaultEnvManager": "ms-python.python:system",
"python-envs.pythonProjects": []
"eslint.workingDirectories": [
"./frontend"
],
"editor.formatOnSave": true,
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.codeActionsOnSave": {
"source.fixAll.eslint": "explicit"
},
"prettier.requireConfig": true,
"[go]": {
"editor.formatOnSave": true,
"editor.defaultFormatter": "golang.go"
},
"[sql]": {
"editor.defaultFormatter": "adpyke.vscode-sql-formatter"
},
"[html]": {
"editor.defaultFormatter": "vscode.html-language-features"
}
}

View File

@@ -201,24 +201,26 @@ docker-buildx-enterprise: go-build-enterprise js-build
# python commands
##############################################################
.PHONY: py-fmt
py-fmt: ## Run ruff format across the shared tests project
@cd tests && uv run ruff format .
py-fmt: ## Run black for integration tests
@cd tests/integration && uv run black .
.PHONY: py-lint
py-lint: ## Run ruff check across the shared tests project
@cd tests && uv run ruff check --fix .
py-lint: ## Run lint for integration tests
@cd tests/integration && uv run isort .
@cd tests/integration && uv run autoflake .
@cd tests/integration && uv run pylint .
.PHONY: py-test-setup
py-test-setup: ## Bring up the shared SigNoz backend used by integration and e2e tests
@cd tests && uv run pytest --basetemp=./tmp/ -vv --reuse --capture=no integration/bootstrap/setup.py::test_setup
py-test-setup: ## Runs integration tests
@cd tests/integration && uv run pytest --basetemp=./tmp/ -vv --reuse --capture=no src/bootstrap/setup.py::test_setup
.PHONY: py-test-teardown
py-test-teardown: ## Tear down the shared SigNoz backend
@cd tests && uv run pytest --basetemp=./tmp/ -vv --teardown --capture=no integration/bootstrap/setup.py::test_teardown
py-test-teardown: ## Runs integration tests with teardown
@cd tests/integration && uv run pytest --basetemp=./tmp/ -vv --teardown --capture=no src/bootstrap/setup.py::test_teardown
.PHONY: py-test
py-test: ## Runs integration tests
@cd tests && uv run pytest --basetemp=./tmp/ -vv --capture=no integration/tests/
@cd tests/integration && uv run pytest --basetemp=./tmp/ -vv --capture=no src/
.PHONY: py-clean
py-clean: ## Clear all pycache and pytest cache from tests directory recursively

View File

@@ -7,41 +7,27 @@ import (
"github.com/spf13/cobra"
"github.com/SigNoz/signoz/cmd"
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/analytics"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authn"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/authz/openfgaauthz"
"github.com/SigNoz/signoz/pkg/authz/openfgaschema"
"github.com/SigNoz/signoz/pkg/authz/openfgaserver"
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/gateway/noopgateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/licensing/nooplicensing"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration/implcloudintegration"
"github.com/SigNoz/signoz/pkg/modules/dashboard"
"github.com/SigNoz/signoz/pkg/modules/dashboard/impldashboard"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/query-service/app"
"github.com/SigNoz/signoz/pkg/queryparser"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/ruler/signozruler"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/zeus"
"github.com/SigNoz/signoz/pkg/zeus/noopzeus"
@@ -83,7 +69,7 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
},
signoz.NewEmailingProviderFactories(),
signoz.NewCacheProviderFactories(),
signoz.NewWebProviderFactories(config.Global),
signoz.NewWebProviderFactories(),
func(sqlstore sqlstore.SQLStore) factory.NamedMap[factory.ProviderFactory[sqlschema.SQLSchema, sqlschema.Config]] {
return signoz.NewSQLSchemaProviderFactories(sqlstore)
},
@@ -92,13 +78,8 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(ctx context.Context, providerSettings factory.ProviderSettings, store authtypes.AuthNStore, licensing licensing.Licensing) (map[authtypes.AuthNProvider]authn.AuthN, error) {
return signoz.NewAuthNs(ctx, providerSettings, store, licensing)
},
func(ctx context.Context, sqlstore sqlstore.SQLStore, _ licensing.Licensing, _ dashboard.Module) (factory.ProviderFactory[authz.AuthZ, authz.Config], error) {
openfgaDataStore, err := openfgaserver.NewSQLStore(sqlstore)
if err != nil {
return nil, err
}
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), openfgaDataStore), nil
func(ctx context.Context, sqlstore sqlstore.SQLStore, _ licensing.Licensing, _ dashboard.Module) factory.ProviderFactory[authz.AuthZ, authz.Config] {
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx))
},
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, _ querier.Querier, _ licensing.Licensing) dashboard.Module {
return impldashboard.NewModule(impldashboard.NewStore(store), settings, analytics, orgGetter, queryParser)
@@ -106,18 +87,9 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(_ licensing.Licensing) factory.ProviderFactory[gateway.Gateway, gateway.Config] {
return noopgateway.NewProviderFactory()
},
func(_ licensing.Licensing) factory.NamedMap[factory.ProviderFactory[auditor.Auditor, auditor.Config]] {
return signoz.NewAuditorProviderFactories()
},
func(ps factory.ProviderSettings, q querier.Querier, a analytics.Analytics) querier.Handler {
return querier.NewHandler(ps, q, a)
},
func(_ sqlstore.SQLStore, _ global.Global, _ zeus.Zeus, _ gateway.Gateway, _ licensing.Licensing, _ serviceaccount.Module, _ cloudintegration.Config) (cloudintegration.Module, error) {
return implcloudintegration.NewModule(), nil
},
func(c cache.Cache, am alertmanager.Alertmanager, ss sqlstore.SQLStore, ts telemetrystore.TelemetryStore, ms telemetrytypes.MetadataStore, p prometheus.Prometheus, og organization.Getter, rsh rulestatehistory.Module, q querier.Querier, qp queryparser.QueryParser) factory.NamedMap[factory.ProviderFactory[ruler.Ruler, ruler.Config]] {
return factory.MustNewNamedMap(signozruler.NewFactory(c, am, ss, ts, ms, p, og, rsh, q, qp, nil, nil))
},
)
if err != nil {
logger.ErrorContext(ctx, "failed to create signoz", errors.Attr(err))

View File

@@ -8,56 +8,37 @@ import (
"github.com/spf13/cobra"
"github.com/SigNoz/signoz/cmd"
"github.com/SigNoz/signoz/ee/auditor/otlphttpauditor"
"github.com/SigNoz/signoz/ee/authn/callbackauthn/oidccallbackauthn"
"github.com/SigNoz/signoz/ee/authn/callbackauthn/samlcallbackauthn"
"github.com/SigNoz/signoz/ee/authz/openfgaauthz"
"github.com/SigNoz/signoz/ee/authz/openfgaschema"
"github.com/SigNoz/signoz/ee/authz/openfgaserver"
"github.com/SigNoz/signoz/ee/gateway/httpgateway"
enterpriselicensing "github.com/SigNoz/signoz/ee/licensing"
"github.com/SigNoz/signoz/ee/licensing/httplicensing"
"github.com/SigNoz/signoz/ee/modules/cloudintegration/implcloudintegration"
"github.com/SigNoz/signoz/ee/modules/cloudintegration/implcloudintegration/implcloudprovider"
"github.com/SigNoz/signoz/ee/modules/dashboard/impldashboard"
eequerier "github.com/SigNoz/signoz/ee/querier"
enterpriseapp "github.com/SigNoz/signoz/ee/query-service/app"
eerules "github.com/SigNoz/signoz/ee/query-service/rules"
"github.com/SigNoz/signoz/ee/sqlschema/postgressqlschema"
"github.com/SigNoz/signoz/ee/sqlstore/postgressqlstore"
enterprisezeus "github.com/SigNoz/signoz/ee/zeus"
"github.com/SigNoz/signoz/ee/zeus/httpzeus"
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/analytics"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authn"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
pkgcloudintegration "github.com/SigNoz/signoz/pkg/modules/cloudintegration/implcloudintegration"
"github.com/SigNoz/signoz/pkg/modules/dashboard"
pkgimpldashboard "github.com/SigNoz/signoz/pkg/modules/dashboard/impldashboard"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/queryparser"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/ruler/signozruler"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstorehook"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/zeus"
)
@@ -105,7 +86,7 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
},
signoz.NewEmailingProviderFactories(),
signoz.NewCacheProviderFactories(),
signoz.NewWebProviderFactories(config.Global),
signoz.NewWebProviderFactories(),
func(sqlstore sqlstore.SQLStore) factory.NamedMap[factory.ProviderFactory[sqlschema.SQLSchema, sqlschema.Config]] {
existingFactories := signoz.NewSQLSchemaProviderFactories(sqlstore)
if err := existingFactories.Add(postgressqlschema.NewFactory(sqlstore)); err != nil {
@@ -137,12 +118,8 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
return authNs, nil
},
func(ctx context.Context, sqlstore sqlstore.SQLStore, licensing licensing.Licensing, dashboardModule dashboard.Module) (factory.ProviderFactory[authz.AuthZ, authz.Config], error) {
openfgaDataStore, err := openfgaserver.NewSQLStore(sqlstore)
if err != nil {
return nil, err
}
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), openfgaDataStore, licensing, dashboardModule), nil
func(ctx context.Context, sqlstore sqlstore.SQLStore, licensing licensing.Licensing, dashboardModule dashboard.Module) factory.ProviderFactory[authz.AuthZ, authz.Config] {
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), licensing, dashboardModule)
},
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, querier querier.Querier, licensing licensing.Licensing) dashboard.Module {
return impldashboard.NewModule(pkgimpldashboard.NewStore(store), settings, analytics, orgGetter, queryParser, querier, licensing)
@@ -150,35 +127,12 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(licensing licensing.Licensing) factory.ProviderFactory[gateway.Gateway, gateway.Config] {
return httpgateway.NewProviderFactory(licensing)
},
func(licensing licensing.Licensing) factory.NamedMap[factory.ProviderFactory[auditor.Auditor, auditor.Config]] {
factories := signoz.NewAuditorProviderFactories()
if err := factories.Add(otlphttpauditor.NewFactory(licensing, version.Info)); err != nil {
panic(err)
}
return factories
},
func(ps factory.ProviderSettings, q querier.Querier, a analytics.Analytics) querier.Handler {
communityHandler := querier.NewHandler(ps, q, a)
return eequerier.NewHandler(ps, q, communityHandler)
},
func(sqlStore sqlstore.SQLStore, global global.Global, zeus zeus.Zeus, gateway gateway.Gateway, licensing licensing.Licensing, serviceAccount serviceaccount.Module, config cloudintegration.Config) (cloudintegration.Module, error) {
defStore := pkgcloudintegration.NewServiceDefinitionStore()
awsCloudProviderModule, err := implcloudprovider.NewAWSCloudProvider(defStore)
if err != nil {
return nil, err
}
azureCloudProviderModule := implcloudprovider.NewAzureCloudProvider()
cloudProvidersMap := map[cloudintegrationtypes.CloudProviderType]cloudintegration.CloudProviderModule{
cloudintegrationtypes.CloudProviderTypeAWS: awsCloudProviderModule,
cloudintegrationtypes.CloudProviderTypeAzure: azureCloudProviderModule,
}
return implcloudintegration.NewModule(pkgcloudintegration.NewStore(sqlStore), global, zeus, gateway, licensing, serviceAccount, cloudProvidersMap, config)
},
func(c cache.Cache, am alertmanager.Alertmanager, ss sqlstore.SQLStore, ts telemetrystore.TelemetryStore, ms telemetrytypes.MetadataStore, p prometheus.Prometheus, og organization.Getter, rsh rulestatehistory.Module, q querier.Querier, qp queryparser.QueryParser) factory.NamedMap[factory.ProviderFactory[ruler.Ruler, ruler.Config]] {
return factory.MustNewNamedMap(signozruler.NewFactory(c, am, ss, ts, ms, p, og, rsh, q, qp, eerules.PrepareTaskFunc, eerules.TestNotification))
},
)
if err != nil {
logger.ErrorContext(ctx, "failed to create signoz", errors.Attr(err))
return err

View File

@@ -6,8 +6,6 @@
##################### Global #####################
global:
# the url under which the signoz apiserver is externally reachable.
# the path component (e.g. /signoz in https://example.com/signoz) is used
# as the base path for all HTTP routes (both API and web frontend).
external_url: <unset>
# the url where the SigNoz backend receives telemetry data (traces, metrics, logs) from instrumented applications.
ingestion_url: <unset>
@@ -52,8 +50,8 @@ pprof:
web:
# Whether to enable the web frontend
enabled: true
# The index file to use as the SPA entrypoint.
index: index.html
# The prefix to serve web on
prefix: /
# The directory containing the static build files.
directory: /etc/signoz/web
@@ -84,18 +82,13 @@ sqlstore:
provider: sqlite
# The maximum number of open connections to the database.
max_open_conns: 100
# The maximum amount of time a connection may be reused.
# If max_conn_lifetime == 0, connections are not closed due to a connection's age.
max_conn_lifetime: 0
sqlite:
# The path to the SQLite database file.
path: /var/lib/signoz/signoz.db
# The journal mode for the sqlite database. Supported values: delete, wal.
mode: wal
# The timeout for the sqlite database to wait for a lock.
# Mode is the mode to use for the sqlite database.
mode: delete
# BusyTimeout is the timeout for the sqlite database to wait for a lock.
busy_timeout: 10s
# The default transaction locking behavior. Supported values: deferred, immediate, exclusive.
transaction_mode: deferred
##################### APIServer #####################
apiserver:
@@ -151,8 +144,6 @@ telemetrystore:
##################### Prometheus #####################
prometheus:
# The maximum time a PromQL query is allowed to run before being aborted.
timeout: 2m
active_query_tracker:
# Whether to enable the active query tracker.
enabled: true
@@ -359,51 +350,3 @@ identn:
impersonation:
# toggle impersonation identN, when enabled, all requests will impersonate the root user
enabled: false
##################### Service Account #####################
serviceaccount:
email:
# email domain for the service account principal
domain: signozserviceaccount.com
analytics:
# toggle service account analytics
enabled: true
##################### Auditor #####################
auditor:
# Specifies the auditor provider to use.
# noop: discards all audit events (community default).
# otlphttp: exports audit events via OTLP HTTP (enterprise).
provider: noop
# The async channel capacity for audit events. Events are dropped when full (fail-open).
buffer_size: 1000
# The maximum number of events per export batch.
batch_size: 100
# The maximum time between export flushes.
flush_interval: 1s
otlphttp:
# The target scheme://host:port/path of the OTLP HTTP endpoint.
endpoint: http://localhost:4318/v1/logs
# Whether to use HTTP instead of HTTPS.
insecure: false
# The maximum duration for an export attempt.
timeout: 10s
# Additional HTTP headers sent with every export request.
headers: {}
retry:
# Whether to retry on transient failures.
enabled: true
# The initial wait time before the first retry.
initial_interval: 5s
# The upper bound on backoff interval.
max_interval: 30s
# The total maximum time spent retrying.
max_elapsed_time: 60s
##################### Cloud Integration #####################
cloudintegration:
# cloud integration agent configuration
agent:
# The version of the cloud integration agent.
version: v0.0.8

View File

@@ -190,7 +190,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:v0.120.0
image: signoz/signoz:v0.116.1
ports:
- "8080:8080" # signoz port
# - "6060:6060" # pprof port
@@ -213,7 +213,7 @@ services:
retries: 3
otel-collector:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:v0.144.3
image: signoz/signoz-otel-collector:v0.144.2
entrypoint:
- /bin/sh
command:
@@ -241,7 +241,7 @@ services:
replicas: 3
signoz-telemetrystore-migrator:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:v0.144.3
image: signoz/signoz-otel-collector:v0.144.2
environment:
- SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
- SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster

View File

@@ -117,7 +117,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:v0.120.0
image: signoz/signoz:v0.116.1
ports:
- "8080:8080" # signoz port
volumes:
@@ -139,7 +139,7 @@ services:
retries: 3
otel-collector:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:v0.144.3
image: signoz/signoz-otel-collector:v0.144.2
entrypoint:
- /bin/sh
command:
@@ -167,7 +167,7 @@ services:
replicas: 3
signoz-telemetrystore-migrator:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:v0.144.3
image: signoz/signoz-otel-collector:v0.144.2
environment:
- SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
- SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster

View File

@@ -181,7 +181,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:${VERSION:-v0.120.0}
image: signoz/signoz:${VERSION:-v0.116.1}
container_name: signoz
ports:
- "8080:8080" # signoz port
@@ -204,7 +204,7 @@ services:
retries: 3
otel-collector:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.3}
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.2}
container_name: signoz-otel-collector
entrypoint:
- /bin/sh
@@ -229,7 +229,7 @@ services:
- "4318:4318" # OTLP HTTP receiver
signoz-telemetrystore-migrator:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.3}
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.2}
container_name: signoz-telemetrystore-migrator
environment:
- SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000

View File

@@ -109,7 +109,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:${VERSION:-v0.120.0}
image: signoz/signoz:${VERSION:-v0.116.1}
container_name: signoz
ports:
- "8080:8080" # signoz port
@@ -132,7 +132,7 @@ services:
retries: 3
otel-collector:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.3}
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.2}
container_name: signoz-otel-collector
entrypoint:
- /bin/sh
@@ -157,7 +157,7 @@ services:
- "4318:4318" # OTLP HTTP receiver
signoz-telemetrystore-migrator:
!!merge <<: *db-depend
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.3}
image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.2}
container_name: signoz-telemetrystore-migrator
environment:
- SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000

File diff suppressed because it is too large Load Diff

View File

@@ -123,7 +123,6 @@ if err := router.Handle("/api/v1/things", handler.New(
Description: "This endpoint creates a thing",
Request: new(types.PostableThing),
RequestContentType: "application/json",
RequestQuery: new(types.QueryableThing),
Response: new(types.GettableThing),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
@@ -156,8 +155,6 @@ The `handler.New` function ties the HTTP handler to OpenAPI metadata via `OpenAP
- **Request / RequestContentType**:
- `Request` is a Go type that describes the request body or form.
- `RequestContentType` is usually `"application/json"` or `"application/x-www-form-urlencoded"` (for callbacks like SAML).
- **RequestQuery**:
- `RequestQuery` is a Go type that descirbes query url params.
- **RequestExamples**: An array of `handler.OpenAPIExample` that provide concrete request payloads in the generated spec. See [Adding request examples](#adding-request-examples) below.
- **Response / ResponseContentType**:
- `Response` is the Go type for the successful response payload.

View File

@@ -0,0 +1,215 @@
# Integration Tests
SigNoz uses integration tests to verify that different components work together correctly in a real environment. These tests run against actual services (ClickHouse, PostgreSQL, etc.) to ensure end-to-end functionality.
## How to set up the integration test environment?
### Prerequisites
Before running integration tests, ensure you have the following installed:
- Python 3.13+
- [uv](https://docs.astral.sh/uv/getting-started/installation/)
- Docker (for containerized services)
### Initial Setup
1. Navigate to the integration tests directory:
```bash
cd tests/integration
```
2. Install dependencies using uv:
```bash
uv sync
```
> **_NOTE:_** the build backend could throw an error while installing `psycopg2`, pleae see https://www.psycopg.org/docs/install.html#build-prerequisites
### Starting the Test Environment
To spin up all the containers necessary for writing integration tests and keep them running:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse src/bootstrap/setup.py::test_setup
```
This command will:
- Start all required services (ClickHouse, PostgreSQL, Zookeeper, etc.)
- Keep containers running due to the `--reuse` flag
- Verify that the setup is working correctly
### Stopping the Test Environment
When you're done writing integration tests, clean up the environment:
```bash
uv run pytest --basetemp=./tmp/ -vv --teardown -s src/bootstrap/setup.py::test_teardown
```
This will destroy the running integration test setup and clean up resources.
## Understanding the Integration Test Framework
Python and pytest form the foundation of the integration testing framework. Testcontainers are used to spin up disposable integration environments. Wiremock is used to spin up **test doubles** of other services.
- **Why Python/pytest?** It's expressive, low-boilerplate, and has powerful fixture capabilities that make integration testing straightforward. Extensive libraries for HTTP requests, JSON handling, and data analysis (numpy) make it easier to test APIs and verify data
- **Why testcontainers?** They let us spin up isolated dependencies that match our production environment without complex setup.
- **Why wiremock?** Well maintained, documented and extensible.
```
.
├── conftest.py
├── fixtures
│ ├── __init__.py
│ ├── auth.py
│ ├── clickhouse.py
│ ├── fs.py
│ ├── http.py
│ ├── migrator.py
│ ├── network.py
│ ├── postgres.py
│ ├── signoz.py
│ ├── sql.py
│ ├── sqlite.py
│ ├── types.py
│ └── zookeeper.py
├── uv.lock
├── pyproject.toml
└── src
└── bootstrap
├── __init__.py
├── 01_database.py
├── 02_register.py
└── 03_license.py
```
Each test suite follows some important principles:
1. **Organization**: Test suites live under `src/` in self-contained packages. Fixtures (a pytest concept) live inside `fixtures/`.
2. **Execution Order**: Files are prefixed with two-digit numbers (`01_`, `02_`, `03_`) to ensure sequential execution.
3. **Time Constraints**: Each suite should complete in under 10 minutes (setup takes ~4 mins).
### Test Suite Design
Test suites should target functional domains or subsystems within SigNoz. When designing a test suite, consider these principles:
- **Functional Cohesion**: Group tests around a specific capability or service boundary
- **Data Flow**: Follow the path of data through related components
- **Change Patterns**: Components frequently modified together should be tested together
The exact boundaries for modules are intentionally flexible, allowing teams to define logical groupings based on their specific context and knowledge of the system.
Eg: The **bootstrap** integration test suite validates core system functionality:
- Database initialization
- Version check
Other test suites can be **pipelines, auth, querier.**
## How to write an integration test?
Now start writing an integration test. Create a new file `src/bootstrap/05_version.py` and paste the following:
```python
import requests
from fixtures import types
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
def test_version(signoz: types.SigNoz) -> None:
response = requests.get(signoz.self.host_config.get("/api/v1/version"), timeout=2)
logger.info(response)
```
We have written a simple test which calls the `version` endpoint of the container in step 1. In **order to just run this function, run the following command:**
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse src/bootstrap/05_version.py::test_version
```
> Note: The `--reuse` flag is used to reuse the environment if it is already running. Always use this flag when writing and running integration tests. If you don't use this flag, the environment will be destroyed and recreated every time you run the test.
Here's another example of how to write a more comprehensive integration test:
```python
from http import HTTPStatus
import requests
from fixtures import types
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
def test_user_registration(signoz: types.SigNoz) -> None:
"""Test user registration functionality."""
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/register"),
json={
"name": "testuser",
"orgId": "",
"orgName": "test.org",
"email": "test@example.com",
"password": "password123Z$",
},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
assert response.json()["setupCompleted"] is True
```
## How to run integration tests?
### Running All Tests
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse src/
```
### Running Specific Test Categories
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse src/<suite>
# Run querier tests
uv run pytest --basetemp=./tmp/ -vv --reuse src/querier/
# Run auth tests
uv run pytest --basetemp=./tmp/ -vv --reuse src/auth/
```
### Running Individual Tests
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse src/<suite>/<file>.py::test_name
# Run test_register in file 01_register.py in passwordauthn suite
uv run pytest --basetemp=./tmp/ -vv --reuse src/passwordauthn/01_register.py::test_register
```
## How to configure different options for integration tests?
Tests can be configured using pytest options:
- `--sqlstore-provider` - Choose database provider (default: postgres)
- `--postgres-version` - PostgreSQL version (default: 15)
- `--clickhouse-version` - ClickHouse version (default: 25.5.6)
- `--zookeeper-version` - Zookeeper version (default: 3.7.1)
Example:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse --sqlstore-provider=postgres --postgres-version=14 src/auth/
```
## What should I remember?
- **Always use the `--reuse` flag** when setting up the environment to keep containers running
- **Use the `--teardown` flag** when cleaning up to avoid resource leaks
- **Follow the naming convention** with two-digit numeric prefixes (`01_`, `02_`) for test execution order
- **Use proper timeouts** in HTTP requests to avoid hanging tests
- **Clean up test data** between tests to avoid interference
- **Use descriptive test names** that clearly indicate what is being tested
- **Leverage fixtures** for common setup and authentication
- **Test both success and failure scenarios** to ensure robust functionality

View File

@@ -15,8 +15,8 @@ We **recommend** (almost enforce) reviewing these guides before contributing to
- [Endpoint](endpoint.md) - HTTP endpoint patterns
- [Flagger](flagger.md) - Feature flag patterns
- [Handler](handler.md) - HTTP handler patterns
- [Integration](integration.md) - Integration testing
- [Provider](provider.md) - Dependency injection and provider patterns
- [Packages](packages.md) - Naming, layout, and conventions for `pkg/` packages
- [Service](service.md) - Managed service lifecycle with `factory.Service`
- [SQL](sql.md) - Database and SQL patterns
- [Types](types.md) - Domain types, request/response bodies, and storage rows in `pkg/types/`

View File

@@ -1,152 +0,0 @@
# Types
Domain types in `pkg/types/<domain>/` live on three serialization boundaries — inbound HTTP, outbound HTTP, and SQL — on top of an in-memory domain representation. SigNoz's convention is **core-type-first**: every domain defines a single canonical type `X`, and specialized flavors (`PostableX`, `GettableX`, `UpdatableX`, `StorableX`) are introduced **only when they actually differ from `X`**. This guide spells out when each flavor is warranted and how they relate to each other.
Before reading, make sure you have read [abstractions.md](abstractions.md) — the rules here build on its guidance that every new type must earn its place.
## The core type is required
Every domain package in `pkg/types/<domain>/` defines exactly one core type `X`: `AuthDomain`, `Channel`, `Rule`, `Dashboard`, `Role`, `PlannedMaintenance`. This is the canonical in-memory representation of the domain object. Domain methods, validation invariants, and business logic hang off `X` — not off the flavor types.
Two rules shape how the core type behaves:
- **Conversions can be either `New<Output>From<Input>` or a receiver-style `(x *X) ToY()` method.** Either form is fine; pick whichever reads best at the call site:
```go
// Constructor form
func NewGettableAuthDomainFromAuthDomain(d *AuthDomain, info *AuthNProviderInfo) *GettableAuthDomain
// Receiver form
func (m *PlannedMaintenanceWithRules) ToPlannedMaintenance() *PlannedMaintenance
```
- **`X` can double as the storage row** when the DB shape would be identical. `Channel` embeds `bun.BaseModel` directly, and there is no `StorableChannel`. This is the preferred shape when it works.
Domain packages under `pkg/types/` must not import from other `pkg/` packages. Keep the core type's methods lightweight and push orchestration out to the module layer.
## Add a flavor only when it differs
For each of the four flavors, create it only if its shape diverges from `X`. If a flavor would have the same fields and tags as `X`, reuse `X` directly, or declare a type alias. Every flavor must earn its place per [abstractions.md](abstractions.md) rule 6 ("Wrappers must add semantics, not just rename").
| Flavor | Create it when it differs in… |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PostableX` | JSON shape differs from `X` — typically no `Id`, no audit fields, no server-computed fields. Often owns input validation via `Validate()` or a custom `UnmarshalJSON`. |
| `GettableX` | Response shape adds server-computed fields that are not persisted — e.g., `GettableAuthDomain` adds `AuthNProviderInfo`, which is resolved at read time. |
| `UpdatableX` | Only a strict subset of `PostableX` is replaceable on PUT. If the updatable shape equals `PostableX`, reuse `PostableX`. |
| `StorableX` | DB row shape differs from `X` — usually `X` carries nested typed config while `StorableX` carries a flat `Data string` JSON column, plus bun tags, audit mixins, and an `OrgID`. If `X` already has those, skip the flavor. |
The failure mode this rule exists to prevent: minting all four flavors on reflex for every new resource, even when two or three are structurally identical. Each unnecessary flavor is another type contributors must understand and another conversion that can drift.
## Worked examples
### Channel — core type only
```go
type Channels = []*Channel
type GettableChannels = []*Channel
type Channel struct {
bun.BaseModel `bun:"table:notification_channel"`
types.Identifiable
types.TimeAuditable
Name string `json:"name" required:"true" bun:"name"`
Type string `json:"type" required:"true" bun:"type"`
Data string `json:"data" required:"true" bun:"data"`
OrgID string `json:"orgId" required:"true" bun:"org_id"`
}
```
`Channel` is both the domain type and the bun row. `GettableChannels` is a **type alias** because `*Channel` already serializes correctly as a response. There is no `StorableChannel`, `PostableChannel`, or `UpdatableChannel` — those would be identical to `Channel` and so do not exist. Prefer this shape when it works.
### AuthDomain — all four flavors
```go
type AuthDomain struct {
storableAuthDomain *StorableAuthDomain
authDomainConfig *AuthDomainConfig
}
type StorableAuthDomain struct {
bun.BaseModel `bun:"table:auth_domain"`
types.Identifiable
Name string `bun:"name"`
Data string `bun:"data"` // AuthDomainConfig serialized as JSON
OrgID valuer.UUID `bun:"org_id"`
types.TimeAuditable
}
type PostableAuthDomain struct {
Config AuthDomainConfig `json:"config"`
Name string `json:"name"`
}
type UpdateableAuthDomain struct {
Config AuthDomainConfig `json:"config"` // Name intentionally absent
}
type GettableAuthDomain struct {
*StorableAuthDomain
*AuthDomainConfig
AuthNProviderInfo *AuthNProviderInfo `json:"authNProviderInfo"`
}
```
Each flavor exists for a concrete reason:
- `StorableAuthDomain` stores the typed config as an opaque `Data string` column, so the schema does not need to migrate every time a config field is added.
- `PostableAuthDomain` carries the config as a structured object (not a string) for the request.
- `UpdateableAuthDomain` excludes `Name` because a domain's name cannot change after creation.
- `GettableAuthDomain` adds `AuthNProviderInfo`, which is derived at read time and never persisted.
The core `AuthDomain` holds the two live halves — `storableAuthDomain` and `authDomainConfig` — and owns business methods such as `Update(config)`. Conversions use the `New<Output>From<Input>` form: `NewAuthDomainFromConfig`, `NewAuthDomainFromStorableAuthDomain`, `NewGettableAuthDomainFromAuthDomain`.
## Conventions that tie the flavors together
- **Conversions** use either a `New<Output>From<Input>` constructor — e.g. `NewChannelFromReceiver`, `NewGettableAuthDomainFromAuthDomain` — or a receiver-style `ToY()` method. Both forms coexist in the codebase; use whichever fits the call site.
- **Validation belongs on the core type `X`.** Putting it on `X` means every write path — HTTP create, HTTP update, in-process migration, replay — runs the same checks. `Validate()` on `PostableX` is reserved for checks that are specific to the request shape and do not apply to `X`. `UnmarshalJSON` on `PostableX` is a separate tool that lives there because decoding only happens at the HTTP boundary — `PostableAuthDomain.UnmarshalJSON` rejecting a malformed domain name at decode time is the canonical example.
```go
// Domain invariants: every write path re-runs these.
func (x *X) Validate() error { ... }
// Request-shape-only: checks that do not apply once the value is persisted.
func (p *PostableX) Validate() error { ... }
```
- **Type aliases, not wrappers**, when two shapes are identical. `type GettableChannels = []*Channel` is correct because it adds no semantics beyond the underlying type.
- **Serialization tags** follow [handler.md](handler.md): `required:"true"` means the JSON key must be present, `nullable:"true"` is required on any slice or map that may serialize as `null`, and types with a fixed value set must implement `Enum() []any`.
## A note on `UpdatableX` and `PatchableX`
- `UpdatableX` — the body for PUT (full replace) when the shape is a strict subset of `PostableX`. If the updatable shape equals `PostableX`, reuse `PostableX`.
- `PatchableX` — the body for PATCH (partial update); only the fields a client is allowed to patch. For example, `PatchableRole` carries a single `Description` field even though `Role` has many — clients may patch the description but not anything else.
```go
type PatchableRole struct {
Description string `json:"description"`
}
```
Both are optional. Do not introduce them if `PostableX` already covers the case.
## What to avoid
- **Do not mint a flavor that mirrors the core type.** If `StorableX` would have the same fields as `X`, use `X` directly with `bun.BaseModel` embedded. `Channel` is the canonical example.
- **Do not bolt domain methods onto `StorableX`.** Storage types are data carriers. Domain methods live on `X`.
- **Do not invent new suffixes** (`Creatable`, `Fetchable`, `Savable`). The core type plus `Postable` / `Gettable` / `Updatable` / `Patchable` / `Storable` covers every case that exists today.
- **Spelling — `Updatable`, not `Updateable`.** `Updateable` is a common typo. Prefer the shorter form when introducing new types, and rename any stragglers you come across.
- **Spelling — `Storable`, not `Storeable`.** `Storeable` is a common typo. Prefer the shorter form when introducing new types, and rename any stragglers you come across.
## What should I remember?
- Every domain package defines the core type `X`. Only `X` is mandatory.
- Add `PostableX` / `GettableX` / `UpdatableX` / `StorableX` one at a time, only when the shape actually diverges from `X`.
- Domain logic lives on `X`, not on the flavor types.
- Conversions can be a `New<Output>From<Input>` constructor or a receiver-style `ToY()` method — pick whichever reads best at the call site.
- Use a type alias when two shapes are truly identical.
- `pkg/types/<domain>/` must not import from other `pkg/` packages.
## Further reading
- [abstractions.md](abstractions.md) — when to introduce a new type at all.
- [handler.md](handler.md) — struct tag rules at the HTTP boundary.
- [packages.md](packages.md) — where types live under `pkg/types/`.
- [sql.md](sql.md) — star-schema requirements for `StorableX`.

View File

@@ -7,12 +7,12 @@ This guide explains how to add new data sources to the SigNoz onboarding flow. T
The configuration is located at:
```
frontend/src/container/OnboardingV2Container/onboarding-configs/onboarding-config-with-links.ts
frontend/src/container/OnboardingV2Container/onboarding-configs/onboarding-config-with-links.json
```
## Structure Overview
## JSON Structure Overview
The configuration file exports a TypeScript array (`onboardingConfigWithLinks`) containing data source objects. Each object represents a selectable option in the onboarding flow. SVG logos are imported as ES modules at the top of the file.
The configuration file is a JSON array containing data source objects. Each object represents a selectable option in the onboarding flow.
## Data Source Object Keys
@@ -24,7 +24,7 @@ The configuration file exports a TypeScript array (`onboardingConfigWithLinks`)
| `label` | `string` | Display name shown to users (e.g., `"AWS EC2"`) |
| `tags` | `string[]` | Array of category tags for grouping (e.g., `["AWS"]`, `["database"]`) |
| `module` | `string` | Destination module after onboarding completion |
| `imgUrl` | `string` | Imported SVG URL **(SVG required)** (e.g., `import ec2Url from '@/assets/Logos/ec2.svg'`, then use `ec2Url`) |
| `imgUrl` | `string` | Path to the logo/icon **(SVG required)** (e.g., `"/Logos/ec2.svg"`) |
### Optional Keys
@@ -57,34 +57,36 @@ The `module` key determines where users are redirected after completing onboardi
The `question` object enables multi-step selection flows:
```ts
question: {
desc: 'What would you like to monitor?',
type: 'select',
helpText: 'Choose the telemetry type you want to collect.',
helpLink: '/docs/azure-monitoring/overview/',
helpLinkText: 'Read the guide →',
options: [
{
key: 'logging',
label: 'Logs',
imgUrl: azureVmUrl,
link: '/docs/azure-monitoring/app-service/logging/',
},
{
key: 'metrics',
label: 'Metrics',
imgUrl: azureVmUrl,
link: '/docs/azure-monitoring/app-service/metrics/',
},
{
key: 'tracing',
label: 'Traces',
imgUrl: azureVmUrl,
link: '/docs/azure-monitoring/app-service/tracing/',
},
],
},
```json
{
"question": {
"desc": "What would you like to monitor?",
"type": "select",
"helpText": "Choose the telemetry type you want to collect.",
"helpLink": "/docs/azure-monitoring/overview/",
"helpLinkText": "Read the guide →",
"options": [
{
"key": "logging",
"label": "Logs",
"imgUrl": "/Logos/azure-vm.svg",
"link": "/docs/azure-monitoring/app-service/logging/"
},
{
"key": "metrics",
"label": "Metrics",
"imgUrl": "/Logos/azure-vm.svg",
"link": "/docs/azure-monitoring/app-service/metrics/"
},
{
"key": "tracing",
"label": "Traces",
"imgUrl": "/Logos/azure-vm.svg",
"link": "/docs/azure-monitoring/app-service/tracing/"
}
]
}
}
```
### Question Keys
@@ -104,161 +106,152 @@ Options can be simple (direct link) or nested (with another question):
### Simple Option (Direct Link)
```ts
```json
{
key: 'aws-ec2-logs',
label: 'Logs',
imgUrl: ec2Url,
link: '/docs/userguide/collect_logs_from_file/',
},
"key": "aws-ec2-logs",
"label": "Logs",
"imgUrl": "/Logos/ec2.svg",
"link": "/docs/userguide/collect_logs_from_file/"
}
```
### Option with Internal Redirect
```ts
```json
{
key: 'aws-ec2-metrics-one-click',
label: 'One Click AWS',
imgUrl: ec2Url,
link: '/integrations?integration=aws-integration&service=ec2',
internalRedirect: true,
},
"key": "aws-ec2-metrics-one-click",
"label": "One Click AWS",
"imgUrl": "/Logos/ec2.svg",
"link": "/integrations?integration=aws-integration&service=ec2",
"internalRedirect": true
}
```
> **Important**: Set `internalRedirect: true` only for internal app routes (like `/integrations?...`). Docs links should NOT have this flag.
### Nested Option (Multi-step Flow)
```ts
```json
{
key: 'aws-ec2-metrics',
label: 'Metrics',
imgUrl: ec2Url,
question: {
desc: 'How would you like to set up monitoring?',
helpText: 'Choose your setup method.',
options: [...],
},
},
"key": "aws-ec2-metrics",
"label": "Metrics",
"imgUrl": "/Logos/ec2.svg",
"question": {
"desc": "How would you like to set up monitoring?",
"helpText": "Choose your setup method.",
"options": [...]
}
}
```
## Examples
### Simple Data Source (Direct Link)
```ts
import elbUrl from '@/assets/Logos/elb.svg';
// inside the onboardingConfigWithLinks array:
```json
{
dataSource: 'aws-elb',
label: 'AWS ELB',
tags: ['AWS'],
module: 'logs',
relatedSearchKeywords: [
'aws',
'aws elb',
'elb logs',
'elastic load balancer',
"dataSource": "aws-elb",
"label": "AWS ELB",
"tags": ["AWS"],
"module": "logs",
"relatedSearchKeywords": [
"aws",
"aws elb",
"elb logs",
"elastic load balancer"
],
imgUrl: elbUrl,
link: '/docs/aws-monitoring/elb/',
},
"imgUrl": "/Logos/elb.svg",
"link": "/docs/aws-monitoring/elb/"
}
```
### Data Source with Single Question Level
```ts
import azureVmUrl from '@/assets/Logos/azure-vm.svg';
// inside the onboardingConfigWithLinks array:
```json
{
dataSource: 'app-service',
label: 'App Service',
imgUrl: azureVmUrl,
tags: ['Azure'],
module: 'apm',
relatedSearchKeywords: ['azure', 'app service'],
question: {
desc: 'What telemetry data do you want to visualise?',
type: 'select',
options: [
"dataSource": "app-service",
"label": "App Service",
"imgUrl": "/Logos/azure-vm.svg",
"tags": ["Azure"],
"module": "apm",
"relatedSearchKeywords": ["azure", "app service"],
"question": {
"desc": "What telemetry data do you want to visualise?",
"type": "select",
"options": [
{
key: 'logging',
label: 'Logs',
imgUrl: azureVmUrl,
link: '/docs/azure-monitoring/app-service/logging/',
"key": "logging",
"label": "Logs",
"imgUrl": "/Logos/azure-vm.svg",
"link": "/docs/azure-monitoring/app-service/logging/"
},
{
key: 'metrics',
label: 'Metrics',
imgUrl: azureVmUrl,
link: '/docs/azure-monitoring/app-service/metrics/',
"key": "metrics",
"label": "Metrics",
"imgUrl": "/Logos/azure-vm.svg",
"link": "/docs/azure-monitoring/app-service/metrics/"
},
{
key: 'tracing',
label: 'Traces',
imgUrl: azureVmUrl,
link: '/docs/azure-monitoring/app-service/tracing/',
},
],
},
},
"key": "tracing",
"label": "Traces",
"imgUrl": "/Logos/azure-vm.svg",
"link": "/docs/azure-monitoring/app-service/tracing/"
}
]
}
}
```
### Data Source with Nested Questions (2-3 Levels)
```ts
import ec2Url from '@/assets/Logos/ec2.svg';
// inside the onboardingConfigWithLinks array:
```json
{
dataSource: 'aws-ec2',
label: 'AWS EC2',
tags: ['AWS'],
module: 'logs',
relatedSearchKeywords: ['aws', 'aws ec2', 'ec2 logs', 'ec2 metrics'],
imgUrl: ec2Url,
question: {
desc: 'What would you like to monitor for AWS EC2?',
type: 'select',
helpText: 'Choose the type of telemetry data you want to collect.',
options: [
"dataSource": "aws-ec2",
"label": "AWS EC2",
"tags": ["AWS"],
"module": "logs",
"relatedSearchKeywords": ["aws", "aws ec2", "ec2 logs", "ec2 metrics"],
"imgUrl": "/Logos/ec2.svg",
"question": {
"desc": "What would you like to monitor for AWS EC2?",
"type": "select",
"helpText": "Choose the type of telemetry data you want to collect.",
"options": [
{
key: 'aws-ec2-logs',
label: 'Logs',
imgUrl: ec2Url,
link: '/docs/userguide/collect_logs_from_file/',
"key": "aws-ec2-logs",
"label": "Logs",
"imgUrl": "/Logos/ec2.svg",
"link": "/docs/userguide/collect_logs_from_file/"
},
{
key: 'aws-ec2-metrics',
label: 'Metrics',
imgUrl: ec2Url,
question: {
desc: 'How would you like to set up EC2 Metrics monitoring?',
helpText: 'One Click uses AWS CloudWatch integration. Manual setup uses OpenTelemetry.',
helpLink: '/docs/aws-monitoring/one-click-vs-manual/',
helpLinkText: 'Read the comparison guide →',
options: [
"key": "aws-ec2-metrics",
"label": "Metrics",
"imgUrl": "/Logos/ec2.svg",
"question": {
"desc": "How would you like to set up EC2 Metrics monitoring?",
"helpText": "One Click uses AWS CloudWatch integration. Manual setup uses OpenTelemetry.",
"helpLink": "/docs/aws-monitoring/one-click-vs-manual/",
"helpLinkText": "Read the comparison guide →",
"options": [
{
key: 'aws-ec2-metrics-one-click',
label: 'One Click AWS',
imgUrl: ec2Url,
link: '/integrations?integration=aws-integration&service=ec2',
internalRedirect: true,
"key": "aws-ec2-metrics-one-click",
"label": "One Click AWS",
"imgUrl": "/Logos/ec2.svg",
"link": "/integrations?integration=aws-integration&service=ec2",
"internalRedirect": true
},
{
key: 'aws-ec2-metrics-manual',
label: 'Manual Setup',
imgUrl: ec2Url,
link: '/docs/tutorial/opentelemetry-binary-usage-in-virtual-machine/',
},
],
},
},
],
},
},
"key": "aws-ec2-metrics-manual",
"label": "Manual Setup",
"imgUrl": "/Logos/ec2.svg",
"link": "/docs/tutorial/opentelemetry-binary-usage-in-virtual-machine/"
}
]
}
}
]
}
}
```
## Best Practices
@@ -277,16 +270,10 @@ import ec2Url from '@/assets/Logos/ec2.svg';
### 3. Logos
- Place logo files in `src/assets/Logos/`
- Place logo files in `public/Logos/`
- Use SVG format
- Import the SVG at the top of the file and reference the imported variable:
```ts
import myServiceUrl from '@/assets/Logos/my-service.svg';
// then in the config object:
imgUrl: myServiceUrl,
```
- **Fetching Icons**: New icons can be easily fetched from [OpenBrand](https://openbrand.sh/). Use the pattern `https://openbrand.sh/?url=<TARGET_URL>`, where `<TARGET_URL>` is the URL-encoded link to the service's website. For example, to get Render's logo, use [https://openbrand.sh/?url=https%3A%2F%2Frender.com](https://openbrand.sh/?url=https%3A%2F%2Frender.com).
- **Optimize new SVGs**: Run any newly downloaded SVGs through an optimizer like [SVGOMG (svgo)](https://svgomg.net/) or use `npx svgo src/assets/Logos/your-logo.svg` to minimise their size before committing.
- Reference as `"/Logos/your-logo.svg"`
- **Optimize new SVGs**: Run any newly downloaded SVGs through an optimizer like [SVGOMG (svgo)](https://svgomg.net/) or use `npx svgo public/Logos/your-logo.svg` to minimise their size before committing.
### 4. Links
@@ -302,8 +289,8 @@ import ec2Url from '@/assets/Logos/ec2.svg';
## Adding a New Data Source
1. Add the logo SVG to `src/assets/Logos/` and add a top-level import in the config file (e.g., `import myServiceUrl from '@/assets/Logos/my-service.svg'`)
2. Add your data source object to the `onboardingConfigWithLinks` array, referencing the imported variable for `imgUrl`
1. Add your data source object to the JSON array
2. Ensure the logo exists in `public/Logos/`
3. Test the flow locally with `yarn dev`
4. Validation:
- Navigate to the [onboarding page](http://localhost:3301/get-started-with-signoz-cloud) on your local machine

View File

@@ -1,261 +0,0 @@
# E2E Tests
SigNoz uses end-to-end tests to verify the frontend works correctly against a real backend. These tests use Playwright to drive a real browser against a containerized SigNoz stack that pytest brings up — the same fixture graph integration tests use, with an extra HTTP seeder container for per-spec telemetry seeding.
## How to set up the E2E test environment?
### Prerequisites
Before running E2E tests, ensure you have the following installed:
- Python 3.13+
- [uv](https://docs.astral.sh/uv/getting-started/installation/)
- Docker (for containerized services)
- Node 18+ and Yarn
### Initial Setup
1. Install Python deps for the shared tests project:
```bash
cd tests
uv sync
```
2. Install Node deps and Playwright browsers:
```bash
cd e2e
yarn install
yarn install:browsers # one-time Playwright browser install
```
### Starting the Test Environment
To spin up the backend stack (SigNoz, ClickHouse, Postgres, Zookeeper, Zeus mock, gateway mock, seeder, migrator-with-web) and keep it running:
```bash
cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse --with-web \
e2e/bootstrap/setup.py::test_setup
```
This command will:
- Bring up all containers via pytest fixtures
- Register the admin user (`admin@integration.test` / `password123Z$`)
- Apply the enterprise license (via a WireMock stub of Zeus) and dismiss the org-onboarding prompt so specs can navigate directly to feature pages
- Start the HTTP seeder container (`tests/seeder/` — exposing `/telemetry/{traces,logs,metrics}` POST + DELETE)
- Write backend coordinates to `tests/e2e/.env.local` (loaded by `playwright.config.ts` via dotenv)
- Keep containers running via the `--reuse` flag
The `--with-web` flag builds the frontend into the SigNoz container — required for E2E. The build takes ~4 mins on a cold start.
### Stopping the Test Environment
When you're done writing E2E tests, clean up the environment:
```bash
cd tests
uv run pytest --basetemp=./tmp/ -vv --teardown \
e2e/bootstrap/setup.py::test_teardown
```
## Understanding the E2E Test Framework
Playwright drives a real browser (Chromium / Firefox / WebKit) against the running SigNoz frontend. The backend is brought up by the same pytest fixture graph integration tests use, so both suites share one source of truth for container lifecycle, license seeding, and test-user accounts.
- **Why Playwright?** First-class TypeScript support, network interception, automatic wait-for-visibility, built-in trace viewer that captures every request/response the UI triggers — so specs rarely need separate API probes alongside UI clicks.
- **Why pytest for lifecycle?** The integration suite already owns container bring-up. Reusing it keeps the E2E stack exactly in sync with the integration stack and avoids a parallel lifecycle framework.
- **Why a separate seeder container?** Per-spec telemetry seeding (traces / logs / metrics) needs a thin HTTP wrapper around the ClickHouse insert helpers so a browser spec can POST from inside the test. The seeder lives at `tests/seeder/`, is built from `tests/Dockerfile.seeder`, and reuses the same `fixtures/{traces,logs,metrics}.py` as integration tests.
```
tests/
├── fixtures/ # shared with integration (see integration.md)
├── integration/ # pytest integration suite
├── seeder/ # standalone HTTP seeder container
│ ├── __init__.py
│ ├── Dockerfile
│ └── server.py # FastAPI app wrapping fixtures.{traces,logs,metrics}
└── e2e/
├── package.json
├── playwright.config.ts # loads .env + .env.local via dotenv
├── .env.example # staging-mode template
├── .env.local # generated by bootstrap/setup.py (gitignored)
├── bootstrap/
│ └── setup.py # test_setup / test_teardown — pytest lifecycle
├── fixtures/
│ └── auth.ts # authedPage Playwright fixture + per-worker storageState cache
├── tests/ # Playwright .spec.ts files, one dir per feature area
│ └── alerts/
│ └── alerts.spec.ts
└── artifacts/ # per-run output (gitignored)
├── html/ # HTML reporter output
├── json/ # JSON reporter output
└── results/ # per-test traces / screenshots / videos on failure
```
Each spec follows these principles:
1. **Directory per feature**: `tests/e2e/tests/<feature>/*.spec.ts`. Cross-resource junction concerns (e.g. cascade-delete) go in their own file, not packed into one giant spec.
2. **Test titles use `TC-NN`**: `test('TC-01 alerts page — tabs render', ...)`. Preserves ordering at a glance and maps to external coverage tracking.
3. **UI-first**: drive flows through the UI. Playwright traces capture every BE request/response the UI triggers, so asserting on UI outcomes implicitly validates BE contracts. Reach for direct `page.request.*` only when the test's *purpose* is asserting a response contract (use `page.waitForResponse` on a UI click) or when a specific UI step is structurally flaky (e.g. Ant DatePicker calendar-cell indices) — and even then try UI first.
4. **Self-contained state**: each spec creates what it needs and cleans up in `try/finally`. No global pre-seeding fixtures.
## How to write an E2E test?
Create a new file `tests/e2e/tests/alerts/smoke.spec.ts`:
```typescript
import { test, expect } from '../../fixtures/auth';
test('TC-01 alerts page — tabs render', async ({ authedPage: page }) => {
await page.goto('/alerts');
await expect(page.getByRole('tab', { name: /alert rules/i })).toBeVisible();
await expect(page.getByRole('tab', { name: /configuration/i })).toBeVisible();
});
```
The `authedPage` fixture (from `tests/e2e/fixtures/auth.ts`) gives you a `Page` whose browser context is already authenticated as the admin user. First use per worker triggers one login; the resulting `storageState` is held in memory and reused for later requests.
To run just this test (assuming the stack is up via `test_setup`):
```bash
cd tests/e2e
npx playwright test tests/alerts/smoke.spec.ts --project=chromium
```
Here's a more comprehensive example that exercises a CRUD flow via the UI:
```typescript
import { test, expect } from '../../fixtures/auth';
test.describe.configure({ mode: 'serial' });
test('TC-02 alerts list — create, toggle, delete', async ({ authedPage: page }) => {
await page.goto('/alerts?tab=AlertRules');
const name = 'smoke-rule';
// Seed via UI — click "New Alert", fill form, save.
await page.getByRole('button', { name: /new alert/i }).click();
await page.getByTestId('alert-name-input').fill(name);
// ... fill metric / threshold / save ...
// Find the row and exercise the action menu.
const row = page.locator('tr', { hasText: name });
await expect(row).toBeVisible();
await row.locator('[data-testid="alert-actions"] button').first().click();
// waitForResponse captures the network call the UI triggers — no parallel fetch needed.
const patchWait = page.waitForResponse(
(r) => r.url().includes('/rules/') && r.request().method() === 'PATCH',
);
await page.getByRole('menuitem').filter({ hasText: /^disable$/i }).click();
await patchWait;
await expect(row).toContainText(/disabled/i);
});
```
### Locator priority
1. `getByRole('button', { name: 'Submit' })`
2. `getByLabel('Email')`
3. `getByPlaceholder('...')`
4. `getByText('...')`
5. `getByTestId('...')`
6. `locator('.ant-select')` — last resort (Ant Design dropdowns often have no semantic alternative)
## How to run E2E tests?
### Running All Tests
With the stack already up, from `tests/e2e/`:
```bash
yarn test # headless, all projects
```
### Running Specific Projects
```bash
yarn test:chromium # chromium only
yarn test:firefox
yarn test:webkit
```
### Running Specific Tests
```bash
cd tests/e2e
# Single feature dir
npx playwright test tests/alerts/ --project=chromium
# Single file
npx playwright test tests/alerts/alerts.spec.ts --project=chromium
# Single test by title grep
npx playwright test --project=chromium -g "TC-01"
```
### Iterative modes
```bash
yarn test:ui # Playwright UI mode — watch + step through
yarn test:headed # headed browser
yarn test:debug # Playwright inspector, pause-on-breakpoint
yarn codegen # record-and-replay locator generation
yarn report # open the last HTML report (artifacts/html)
```
### Staging fallback
Point `SIGNOZ_E2E_BASE_URL` at a remote env via `.env` — no local backend bring-up, no `.env.local` generated, Playwright hits the URL directly:
```bash
cd tests/e2e
cp .env.example .env # fill SIGNOZ_E2E_USERNAME / PASSWORD
yarn test:staging
```
## How to configure different options for E2E tests?
### Environment variables
| Variable | Description |
|---|---|
| `SIGNOZ_E2E_BASE_URL` | Base URL the browser targets. Written by `bootstrap/setup.py` for local mode; set manually for staging. |
| `SIGNOZ_E2E_USERNAME` | Admin email. Bootstrap writes `admin@integration.test`. |
| `SIGNOZ_E2E_PASSWORD` | Admin password. Bootstrap writes the integration-test default. |
| `SIGNOZ_E2E_SEEDER_URL` | Seeder HTTP base URL — hit by specs that need per-test telemetry. |
Loading order in `playwright.config.ts`: `.env` first (user-provided, staging), then `.env.local` with `override: true` (bootstrap-generated, local mode). Anything already set in `process.env` at yarn-test time wins because dotenv doesn't touch vars that are already present.
### Playwright options
The full `playwright.config.ts` is the source of truth. Common things to tweak:
- `projects` — Chromium / Firefox / WebKit are enabled by default. Disable to speed up iteration.
- `retries``2` on CI (`process.env.CI`), `0` locally.
- `fullyParallel: true` — files run in parallel by worker; within a file, use `test.describe.configure({ mode: 'serial' })` if tests share list pages / mutate shared state.
- `trace: 'on-first-retry'`, `screenshot: 'only-on-failure'`, `video: 'retain-on-failure'` — default diagnostic artifacts land in `artifacts/results/<test>/`.
### Pytest options (bootstrap side)
The same pytest flags integration tests expose work here, since E2E reuses the shared fixture graph:
- `--reuse` — keep containers warm between runs (required for all iteration).
- `--teardown` — tear everything down.
- `--with-web` — build the frontend into the SigNoz container. **Required for E2E**; integration tests don't need it.
- `--sqlstore-provider`, `--postgres-version`, `--clickhouse-version`, etc. — see `docs/contributing/integration.md`.
## What should I remember?
- **Always use the `--reuse` flag** when setting up the E2E stack. `--with-web` adds a ~4 min frontend build; you only want to pay that once.
- **Don't teardown before setup.** `--reuse` correctly handles partially-set-up state, so chaining teardown → setup wastes time.
- **Prefer UI-driven flows.** Playwright captures BE requests in the trace; a parallel `fetch` probe is almost always redundant. Drop to `page.request.*` only when the UI can't reach what you need.
- **Use `page.waitForResponse` on UI clicks** to assert BE contracts — it still exercises the UI trigger path.
- **Title every test `TC-NN <short description>`** — keeps the suite navigable and reportable.
- **Split by resource, not by regression suite.** One spec per feature resource; cross-resource junction concerns (cascade-delete, linked-edit) get their own file.
- **Use short descriptive resource names** (`alerts-list-rule`, `labels-rule`, `downtime-once`) — no timestamp disambiguation. Each test owns its resources and cleans up in `try/finally`.
- **Never commit `test.only`** — a pre-commit check or CI runs with `forbidOnly: true`.
- **Prefer explicit waits over `page.waitForTimeout(ms)`.** `await expect(locator).toBeVisible()` is always better than `waitForTimeout(5000)`.
- **Unique test names won't save you from shared-tenant state.** When two tests hit the same list page, either serialize (`describe.configure({ mode: 'serial' })`) or isolate cleanup religiously.
- **Artifacts go to `tests/e2e/artifacts/`** — HTML report at `artifacts/html`, traces at `artifacts/results/<test>/`. All gitignored; archive the dir in CI.

View File

@@ -1,251 +0,0 @@
# Integration Tests
SigNoz uses integration tests to verify that different components work together correctly in a real environment. These tests run against actual services (ClickHouse, PostgreSQL, SigNoz, Zeus mock, Keycloak, etc.) spun up as containers, so suites exercise the same code paths production does.
## How to set up the integration test environment?
### Prerequisites
Before running integration tests, ensure you have the following installed:
- Python 3.13+
- [uv](https://docs.astral.sh/uv/getting-started/installation/)
- Docker (for containerized services)
### Initial Setup
1. Navigate to the shared tests project:
```bash
cd tests
```
2. Install dependencies using uv:
```bash
uv sync
```
> **_NOTE:_** the build backend could throw an error while installing `psycopg2`, please see https://www.psycopg.org/docs/install.html#build-prerequisites
### Starting the Test Environment
To spin up all the containers necessary for writing integration tests and keep them running:
```bash
make py-test-setup
```
Under the hood this runs, from `tests/`:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse integration/bootstrap/setup.py::test_setup
```
This command will:
- Start all required services (ClickHouse, PostgreSQL, Zookeeper, SigNoz, Zeus mock, gateway mock)
- Register an admin user
- Keep containers running via the `--reuse` flag
### Stopping the Test Environment
When you're done writing integration tests, clean up the environment:
```bash
make py-test-teardown
```
Which runs:
```bash
uv run pytest --basetemp=./tmp/ -vv --teardown integration/bootstrap/setup.py::test_teardown
```
This destroys the running integration test setup and cleans up resources.
## Understanding the Integration Test Framework
Python and pytest form the foundation of the integration testing framework. Testcontainers are used to spin up disposable integration environments. WireMock is used to spin up **test doubles** of external services (Zeus cloud API, gateway, etc.).
- **Why Python/pytest?** It's expressive, low-boilerplate, and has powerful fixture capabilities that make integration testing straightforward. Extensive libraries for HTTP requests, JSON handling, and data analysis (numpy) make it easier to test APIs and verify data.
- **Why testcontainers?** They let us spin up isolated dependencies that match our production environment without complex setup.
- **Why WireMock?** Well maintained, documented, and extensible.
```
tests/
├── conftest.py # pytest_plugins registration
├── pyproject.toml
├── uv.lock
├── fixtures/ # shared fixture library (flat package)
│ ├── __init__.py
│ ├── auth.py # admin/editor/viewer users, tokens, license
│ ├── clickhouse.py
│ ├── http.py # WireMock helpers
│ ├── keycloak.py # IdP container
│ ├── postgres.py
│ ├── signoz.py # SigNoz-backend container
│ ├── sql.py
│ ├── types.py
│ └── ... # logs, metrics, traces, alerts, dashboards, ...
├── integration/
│ ├── bootstrap/
│ │ └── setup.py # test_setup / test_teardown
│ ├── testdata/ # JSON / JSONL / YAML inputs per suite
│ └── tests/ # one directory per feature area
│ ├── alerts/
│ │ ├── 01_*.py # numbered suite files
│ │ └── conftest.py # optional suite-local fixtures
│ ├── auditquerier/
│ ├── cloudintegrations/
│ ├── dashboard/
│ ├── passwordauthn/
│ ├── querier/
│ └── ...
└── e2e/ # Playwright suite (see docs/contributing/e2e.md)
```
Each test suite follows these principles:
1. **Organization**: Suites live under `tests/integration/tests/` in self-contained packages. Shared fixtures live in the top-level `tests/fixtures/` package so the e2e tree can reuse them.
2. **Execution Order**: Files are prefixed with two-digit numbers (`01_`, `02_`, `03_`) to ensure sequential execution when tests depend on ordering.
3. **Time Constraints**: Each suite should complete in under 10 minutes (setup takes ~4 mins).
### Test Suite Design
Test suites should target functional domains or subsystems within SigNoz. When designing a test suite, consider these principles:
- **Functional Cohesion**: Group tests around a specific capability or service boundary
- **Data Flow**: Follow the path of data through related components
- **Change Patterns**: Components frequently modified together should be tested together
The exact boundaries for suites are intentionally flexible, allowing contributors to define logical groupings based on their domain knowledge. Current suites cover alerts, audit querier, callback authn, cloud integrations, dashboards, ingestion keys, logs pipelines, password authn, preferences, querier, raw export data, roles, root user, service accounts, and TTL.
## How to write an integration test?
Now start writing an integration test. Create a new file `tests/integration/tests/bootstrap/01_version.py` and paste the following:
```python
import requests
from fixtures import types
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
def test_version(signoz: types.SigNoz) -> None:
response = requests.get(
signoz.self.host_configs["8080"].get("/api/v1/version"),
timeout=2,
)
logger.info(response)
```
We have written a simple test which calls the `version` endpoint of the SigNoz backend. **To run just this function, run the following command:**
```bash
cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse \
integration/tests/bootstrap/01_version.py::test_version
```
> **Note:** The `--reuse` flag is used to reuse the environment if it is already running. Always use this flag when writing and running integration tests. Without it the environment is destroyed and recreated every run.
Here's another example of how to write a more comprehensive integration test:
```python
from http import HTTPStatus
import requests
from fixtures import types
from fixtures.logger import setup_logger
logger = setup_logger(__name__)
def test_user_registration(signoz: types.SigNoz) -> None:
"""Test user registration functionality."""
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/register"),
json={
"name": "testuser",
"orgId": "",
"orgName": "test.org",
"email": "test@example.com",
"password": "password123Z$",
},
timeout=2,
)
assert response.status_code == HTTPStatus.OK
assert response.json()["setupCompleted"] is True
```
Test inputs (JSON fixtures, expected payloads) go under `tests/integration/testdata/<suite>/` and are loaded via `fixtures.fs.get_testdata_file_path`.
## How to run integration tests?
### Running All Tests
```bash
make py-test
```
Which runs:
```bash
uv run pytest --basetemp=./tmp/ -vv integration/tests/
```
### Running Specific Test Categories
```bash
cd tests
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/<suite>/
# Run querier tests
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/querier/
# Run passwordauthn tests
uv run pytest --basetemp=./tmp/ -vv --reuse integration/tests/passwordauthn/
```
### Running Individual Tests
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse \
integration/tests/<suite>/<file>.py::test_name
# Run test_register in 01_register.py in the passwordauthn suite
uv run pytest --basetemp=./tmp/ -vv --reuse \
integration/tests/passwordauthn/01_register.py::test_register
```
## How to configure different options for integration tests?
Tests can be configured using pytest options:
- `--sqlstore-provider` — Choose the SQL store provider (default: `postgres`)
- `--sqlite-mode` — SQLite journal mode: `delete` or `wal` (default: `delete`). Only relevant when `--sqlstore-provider=sqlite`.
- `--postgres-version` — PostgreSQL version (default: `15`)
- `--clickhouse-version` — ClickHouse version (default: `25.5.6`)
- `--zookeeper-version` — Zookeeper version (default: `3.7.1`)
- `--schema-migrator-version` — SigNoz schema migrator version (default: `v0.144.2`)
Example:
```bash
uv run pytest --basetemp=./tmp/ -vv --reuse \
--sqlstore-provider=postgres --postgres-version=14 \
integration/tests/passwordauthn/
```
## What should I remember?
- **Always use the `--reuse` flag** when setting up the environment or running tests to keep containers warm. Without it every run rebuilds the stack (~4 mins).
- **Use the `--teardown` flag** only when cleaning up — mixing `--teardown` with `--reuse` is a contradiction.
- **Do not pre-emptively teardown before setup.** If the stack is partially up, `--reuse` picks up from wherever it is. `make py-test-teardown` then `make py-test-setup` wastes minutes.
- **Follow the naming convention** with two-digit numeric prefixes (`01_`, `02_`) for ordered test execution within a suite.
- **Use proper timeouts** in HTTP requests to avoid hanging tests (`timeout=5` is typical).
- **Clean up test data** between tests in the same suite to avoid interference — or rely on a fresh SigNoz container if you need full isolation.
- **Use descriptive test names** that clearly indicate what is being tested.
- **Leverage fixtures** for common setup. The shared fixture package is at `tests/fixtures/` — reuse before adding new ones.
- **Test both success and failure scenarios** (4xx / 5xx paths) to ensure robust functionality.
- **Run `make py-fmt` and `make py-lint` before committing** Python changes — black + isort + autoflake + pylint.
- **`--sqlite-mode=wal` does not work on macOS.** The integration test environment runs SigNoz inside a Linux container with the SQLite database file mounted from the macOS host. WAL mode requires shared memory between connections, and connections crossing the VM boundary (macOS host ↔ Linux container) cannot share the WAL index, resulting in `SQLITE_IOERR_SHORT_READ`. WAL mode is tested in CI on Linux only.

View File

@@ -16,7 +16,7 @@ func (hp *HourlyProvider) GetBaseSeasonalProvider() *BaseSeasonalProvider {
return &hp.BaseSeasonalProvider
}
// NewHourlyProvider now uses the generic option type.
// NewHourlyProvider now uses the generic option type
func NewHourlyProvider(opts ...GenericProviderOption[*HourlyProvider]) *HourlyProvider {
hp := &HourlyProvider{
BaseSeasonalProvider: BaseSeasonalProvider{},

View File

@@ -32,7 +32,7 @@ func (s Seasonality) IsValid() bool {
}
type AnomaliesRequest struct {
Params *qbtypes.QueryRangeRequest
Params qbtypes.QueryRangeRequest
Seasonality Seasonality
}
@@ -47,7 +47,7 @@ type AnomaliesResponse struct {
// | |
// (rounded value for past peiod) + (seasonal growth)
//
// score = abs(value - prediction) / stddev (current_season_query).
// score = abs(value - prediction) / stddev (current_season_query)
type anomalyQueryParams struct {
// CurrentPeriodQuery is the query range params for period user is looking at or eval window
// Example: (now-5m, now), (now-30m, now), (now-1h, now)
@@ -81,7 +81,7 @@ type anomalyQueryParams struct {
Past3SeasonQuery qbtypes.QueryRangeRequest
}
func prepareAnomalyQueryParams(req *qbtypes.QueryRangeRequest, seasonality Seasonality) *anomalyQueryParams {
func prepareAnomalyQueryParams(req qbtypes.QueryRangeRequest, seasonality Seasonality) *anomalyQueryParams {
start := req.Start
end := req.End

View File

@@ -18,12 +18,12 @@ var (
movingAvgWindowSize = 7
)
// BaseProvider is an interface that includes common methods for all provider types.
// BaseProvider is an interface that includes common methods for all provider types
type BaseProvider interface {
GetBaseSeasonalProvider() *BaseSeasonalProvider
}
// GenericProviderOption is a generic type for provider options.
// GenericProviderOption is a generic type for provider options
type GenericProviderOption[T BaseProvider] func(T)
func WithQuerier[T BaseProvider](querier querier.Querier) GenericProviderOption[T] {
@@ -121,7 +121,7 @@ func (p *BaseSeasonalProvider) getResults(ctx context.Context, orgID valuer.UUID
}
// getMatchingSeries gets the matching series from the query result
// for the given series.
// for the given series
func (p *BaseSeasonalProvider) getMatchingSeries(_ context.Context, queryResult *qbtypes.TimeSeriesData, series *qbtypes.TimeSeries) *qbtypes.TimeSeries {
if queryResult == nil || len(queryResult.Aggregations) == 0 || len(queryResult.Aggregations[0].Series) == 0 {
return nil
@@ -155,14 +155,13 @@ func (p *BaseSeasonalProvider) getStdDev(series *qbtypes.TimeSeries) float64 {
avg := p.getAvg(series)
var sum float64
for _, smpl := range series.Values {
d := smpl.Value - avg
sum += d * d
sum += math.Pow(smpl.Value-avg, 2)
}
return math.Sqrt(sum / float64(len(series.Values)))
}
// getMovingAvg gets the moving average for the given series
// for the given window size and start index.
// for the given window size and start index
func (p *BaseSeasonalProvider) getMovingAvg(series *qbtypes.TimeSeries, movingAvgWindowSize, startIdx int) float64 {
if series == nil || len(series.Values) == 0 {
return 0
@@ -237,7 +236,7 @@ func (p *BaseSeasonalProvider) getPredictedSeries(
// getBounds gets the upper and lower bounds for the given series
// for the given z score threshold
// moving avg of the previous period series + z score threshold * std dev of the series
// moving avg of the previous period series - z score threshold * std dev of the series.
// moving avg of the previous period series - z score threshold * std dev of the series
func (p *BaseSeasonalProvider) getBounds(
series, predictedSeries, weekSeries *qbtypes.TimeSeries,
zScoreThreshold float64,
@@ -270,7 +269,7 @@ func (p *BaseSeasonalProvider) getBounds(
// getExpectedValue gets the expected value for the given series
// for the given index
// prevSeriesAvg + currentSeasonSeriesAvg - mean of past season series, past2 season series and past3 season series.
// prevSeriesAvg + currentSeasonSeriesAvg - mean of past season series, past2 season series and past3 season series
func (p *BaseSeasonalProvider) getExpectedValue(
_, prevSeries, currentSeasonSeries, pastSeasonSeries, past2SeasonSeries, past3SeasonSeries *qbtypes.TimeSeries, idx int,
) float64 {
@@ -284,7 +283,7 @@ func (p *BaseSeasonalProvider) getExpectedValue(
// getScore gets the anomaly score for the given series
// for the given index
// (value - expectedValue) / std dev of the series.
// (value - expectedValue) / std dev of the series
func (p *BaseSeasonalProvider) getScore(
series, prevSeries, weekSeries, weekPrevSeries, past2SeasonSeries, past3SeasonSeries *qbtypes.TimeSeries, value float64, idx int,
) float64 {
@@ -297,7 +296,7 @@ func (p *BaseSeasonalProvider) getScore(
// getAnomalyScores gets the anomaly scores for the given series
// for the given index
// (value - expectedValue) / std dev of the series.
// (value - expectedValue) / std dev of the series
func (p *BaseSeasonalProvider) getAnomalyScores(
series, prevSeries, currentSeasonSeries, pastSeasonSeries, past2SeasonSeries, past3SeasonSeries *qbtypes.TimeSeries,
) *qbtypes.TimeSeries {

View File

@@ -1,143 +0,0 @@
package otlphttpauditor
import (
"bytes"
"context"
"io"
"log/slog"
"net/http"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/audittypes"
collogspb "go.opentelemetry.io/proto/otlp/collector/logs/v1"
"google.golang.org/protobuf/proto"
spb "google.golang.org/genproto/googleapis/rpc/status"
)
const (
maxHTTPResponseReadBytes int64 = 64 * 1024
protobufContentType string = "application/x-protobuf"
)
func (provider *provider) export(ctx context.Context, events []audittypes.AuditEvent) error {
logs := audittypes.NewPLogsFromAuditEvents(events, "signoz", provider.build.Version(), "signoz.audit")
request, err := provider.marshaler.MarshalLogs(logs)
if err != nil {
return errors.Wrapf(err, errors.TypeInternal, auditor.ErrCodeAuditExportFailed, "failed to marshal audit logs")
}
if err := provider.send(ctx, request); err != nil {
provider.settings.Logger().ErrorContext(ctx, "audit export failed", errors.Attr(err), slog.Int("dropped_log_records", len(events)))
return err
}
return nil
}
// Posts a protobuf-encoded OTLP request to the configured endpoint.
// Retries are handled by the underlying heimdall HTTP client.
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlphttpexporter/otlp.go
func (provider *provider) send(ctx context.Context, body []byte) error {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.config.OTLPHTTP.Endpoint.String(), bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", protobufContentType)
res, err := provider.httpClient.Do(req)
if err != nil {
return err
}
defer func() {
_, _ = io.CopyN(io.Discard, res.Body, maxHTTPResponseReadBytes)
res.Body.Close()
}()
if res.StatusCode >= 200 && res.StatusCode <= 299 {
provider.onSuccess(ctx, res)
return nil
}
return provider.onErr(res)
}
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/01b07fcbb7a253bd996c290dcae6166e71d13732/exporter/otlphttpexporter/otlp.go#L403.
func (provider *provider) onSuccess(ctx context.Context, res *http.Response) {
resBytes, err := readResponseBody(res)
if err != nil || resBytes == nil {
return
}
exportResponse := &collogspb.ExportLogsServiceResponse{}
if err := proto.Unmarshal(resBytes, exportResponse); err != nil {
return
}
ps := exportResponse.GetPartialSuccess()
if ps == nil {
return
}
if ps.GetErrorMessage() != "" || ps.GetRejectedLogRecords() != 0 {
provider.settings.Logger().WarnContext(ctx, "partial success response", slog.String("message", ps.GetErrorMessage()), slog.Int64("dropped_log_records", ps.GetRejectedLogRecords()))
}
}
func (provider *provider) onErr(res *http.Response) error {
status := readResponseStatus(res)
if status != nil {
return errors.Newf(errors.TypeInternal, auditor.ErrCodeAuditExportFailed, "request to %s responded with status code %d, Message=%s, Details=%v", provider.config.OTLPHTTP.Endpoint.String(), res.StatusCode, status.Message, status.Details)
}
return errors.Newf(errors.TypeInternal, auditor.ErrCodeAuditExportFailed, "request to %s responded with status code %d", provider.config.OTLPHTTP.Endpoint.String(), res.StatusCode)
}
// Reads at most maxHTTPResponseReadBytes from the response body.
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/01b07fcbb7a253bd996c290dcae6166e71d13732/exporter/otlphttpexporter/otlp.go#L275.
func readResponseBody(resp *http.Response) ([]byte, error) {
if resp.ContentLength == 0 {
return nil, nil
}
maxRead := resp.ContentLength
if maxRead == -1 || maxRead > maxHTTPResponseReadBytes {
maxRead = maxHTTPResponseReadBytes
}
protoBytes := make([]byte, maxRead)
n, err := io.ReadFull(resp.Body, protoBytes)
if n == 0 && (err == nil || errors.Is(err, io.EOF)) {
return nil, nil
}
if err != nil && !errors.Is(err, io.ErrUnexpectedEOF) {
return nil, err
}
return protoBytes[:n], nil
}
// Decodes a protobuf-encoded Status from 4xx/5xx response bodies. Returns nil if the response is empty or cannot be decoded.
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/01b07fcbb7a253bd996c290dcae6166e71d13732/exporter/otlphttpexporter/otlp.go#L310.
func readResponseStatus(resp *http.Response) *spb.Status {
if resp.StatusCode < 400 || resp.StatusCode > 599 {
return nil
}
respBytes, err := readResponseBody(resp)
if err != nil || respBytes == nil {
return nil
}
respStatus := &spb.Status{}
if err := proto.Unmarshal(respBytes, respStatus); err != nil {
return nil
}
return respStatus
}

View File

@@ -1,97 +0,0 @@
package otlphttpauditor
import (
"context"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/auditor/auditorserver"
"github.com/SigNoz/signoz/pkg/factory"
client "github.com/SigNoz/signoz/pkg/http/client"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/types/audittypes"
"github.com/SigNoz/signoz/pkg/version"
"go.opentelemetry.io/collector/pdata/plog"
)
var _ auditor.Auditor = (*provider)(nil)
type provider struct {
settings factory.ScopedProviderSettings
config auditor.Config
licensing licensing.Licensing
build version.Build
server *auditorserver.Server
marshaler plog.ProtoMarshaler
httpClient *client.Client
}
func NewFactory(licensing licensing.Licensing, build version.Build) factory.ProviderFactory[auditor.Auditor, auditor.Config] {
return factory.NewProviderFactory(factory.MustNewName("otlphttp"), func(ctx context.Context, providerSettings factory.ProviderSettings, config auditor.Config) (auditor.Auditor, error) {
return newProvider(ctx, providerSettings, config, licensing, build)
})
}
func newProvider(_ context.Context, providerSettings factory.ProviderSettings, config auditor.Config, licensing licensing.Licensing, build version.Build) (auditor.Auditor, error) {
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/ee/auditor/otlphttpauditor")
httpClient, err := client.New(
settings.Logger(),
providerSettings.TracerProvider,
providerSettings.MeterProvider,
client.WithTimeout(config.OTLPHTTP.Timeout),
client.WithRetryCount(retryCountFromConfig(config.OTLPHTTP.Retry)),
retrierOption(config.OTLPHTTP.Retry),
)
if err != nil {
return nil, err
}
provider := &provider{
settings: settings,
config: config,
licensing: licensing,
build: build,
marshaler: plog.ProtoMarshaler{},
httpClient: httpClient,
}
server, err := auditorserver.New(settings,
auditorserver.Config{
BufferSize: config.BufferSize,
BatchSize: config.BatchSize,
FlushInterval: config.FlushInterval,
},
provider.export,
)
if err != nil {
return nil, err
}
provider.server = server
return provider, nil
}
func (provider *provider) Start(ctx context.Context) error {
return provider.server.Start(ctx)
}
func (provider *provider) Audit(ctx context.Context, event audittypes.AuditEvent) {
if event.PrincipalAttributes.PrincipalOrgID.IsZero() {
provider.settings.Logger().WarnContext(ctx, "audit event dropped as org_id is zero")
return
}
if _, err := provider.licensing.GetActive(ctx, event.PrincipalAttributes.PrincipalOrgID); err != nil {
return
}
provider.server.Add(ctx, event)
}
func (provider *provider) Healthy() <-chan struct{} {
return provider.server.Healthy()
}
func (provider *provider) Stop(ctx context.Context) error {
return provider.server.Stop(ctx)
}

View File

@@ -1,52 +0,0 @@
package otlphttpauditor
import (
"time"
"github.com/SigNoz/signoz/pkg/auditor"
client "github.com/SigNoz/signoz/pkg/http/client"
)
// retrier implements client.Retriable with exponential backoff
// derived from auditor.RetryConfig.
type retrier struct {
initialInterval time.Duration
maxInterval time.Duration
}
func newRetrier(cfg auditor.RetryConfig) *retrier {
return &retrier{
initialInterval: cfg.InitialInterval,
maxInterval: cfg.MaxInterval,
}
}
// NextInterval returns the backoff duration for the given retry attempt.
// Uses exponential backoff: initialInterval * 2^retry, capped at maxInterval.
func (r *retrier) NextInterval(retry int) time.Duration {
interval := r.initialInterval
for range retry {
interval *= 2
}
return min(interval, r.maxInterval)
}
func retrierOption(cfg auditor.RetryConfig) client.Option {
return client.WithRetriable(newRetrier(cfg))
}
func retryCountFromConfig(cfg auditor.RetryConfig) int {
if !cfg.Enabled || cfg.MaxElapsedTime <= 0 {
return 0
}
count := 0
elapsed := time.Duration(0)
interval := cfg.InitialInterval
for elapsed < cfg.MaxElapsedTime {
elapsed += interval
interval = min(interval*2, cfg.MaxInterval)
count++
}
return count
}

View File

@@ -16,7 +16,6 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
openfgav1 "github.com/openfga/api/proto/openfga/v1"
openfgapkgtransformer "github.com/openfga/language/pkg/go/transformer"
"github.com/openfga/openfga/pkg/storage"
)
type provider struct {
@@ -27,14 +26,14 @@ type provider struct {
registry []authz.RegisterTypeable
}
func NewProviderFactory(sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, openfgaDataStore storage.OpenFGADatastore, licensing licensing.Licensing, registry ...authz.RegisterTypeable) factory.ProviderFactory[authz.AuthZ, authz.Config] {
func NewProviderFactory(sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, licensing licensing.Licensing, registry ...authz.RegisterTypeable) factory.ProviderFactory[authz.AuthZ, authz.Config] {
return factory.NewProviderFactory(factory.MustNewName("openfga"), func(ctx context.Context, ps factory.ProviderSettings, config authz.Config) (authz.AuthZ, error) {
return newOpenfgaProvider(ctx, ps, config, sqlstore, openfgaSchema, openfgaDataStore, licensing, registry)
return newOpenfgaProvider(ctx, ps, config, sqlstore, openfgaSchema, licensing, registry)
})
}
func newOpenfgaProvider(ctx context.Context, settings factory.ProviderSettings, config authz.Config, sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, openfgaDataStore storage.OpenFGADatastore, licensing licensing.Licensing, registry []authz.RegisterTypeable) (authz.AuthZ, error) {
pkgOpenfgaAuthzProvider := pkgopenfgaauthz.NewProviderFactory(sqlstore, openfgaSchema, openfgaDataStore)
func newOpenfgaProvider(ctx context.Context, settings factory.ProviderSettings, config authz.Config, sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, licensing licensing.Licensing, registry []authz.RegisterTypeable) (authz.AuthZ, error) {
pkgOpenfgaAuthzProvider := pkgopenfgaauthz.NewProviderFactory(sqlstore, openfgaSchema)
pkgAuthzService, err := pkgOpenfgaAuthzProvider.New(ctx, settings, config)
if err != nil {
return nil, err
@@ -58,10 +57,6 @@ func (provider *provider) Start(ctx context.Context) error {
return provider.openfgaServer.Start(ctx)
}
func (provider *provider) Healthy() <-chan struct{} {
return provider.openfgaServer.Healthy()
}
func (provider *provider) Stop(ctx context.Context) error {
return provider.openfgaServer.Stop(ctx)
}

View File

@@ -16,6 +16,7 @@ type Server struct {
}
func NewOpenfgaServer(ctx context.Context, pkgAuthzService authz.AuthZ) (*Server, error) {
return &Server{
pkgAuthzService: pkgAuthzService,
}, nil
@@ -25,31 +26,14 @@ func (server *Server) Start(ctx context.Context) error {
return server.pkgAuthzService.Start(ctx)
}
func (server *Server) Healthy() <-chan struct{} {
return server.pkgAuthzService.Healthy()
}
func (server *Server) Stop(ctx context.Context) error {
return server.pkgAuthzService.Stop(ctx)
}
func (server *Server) CheckWithTupleCreation(ctx context.Context, claims authtypes.Claims, orgID valuer.UUID, relation authtypes.Relation, typeable authtypes.Typeable, selectors []authtypes.Selector, _ []authtypes.Selector) error {
subject := ""
switch claims.Principal {
case authtypes.PrincipalUser:
user, err := authtypes.NewSubject(authtypes.TypeableUser, claims.UserID, orgID, nil)
if err != nil {
return err
}
subject = user
case authtypes.PrincipalServiceAccount:
serviceAccount, err := authtypes.NewSubject(authtypes.TypeableServiceAccount, claims.ServiceAccountID, orgID, nil)
if err != nil {
return err
}
subject = serviceAccount
subject, err := authtypes.NewSubject(authtypes.TypeableUser, claims.UserID, orgID, nil)
if err != nil {
return err
}
tupleSlice, err := typeable.Tuples(subject, relation, selectors, orgID)

View File

@@ -1,32 +0,0 @@
package openfgaserver
import (
"github.com/SigNoz/signoz/ee/sqlstore/postgressqlstore"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/openfga/openfga/pkg/storage"
"github.com/openfga/openfga/pkg/storage/postgres"
"github.com/openfga/openfga/pkg/storage/sqlcommon"
"github.com/openfga/openfga/pkg/storage/sqlite"
)
func NewSQLStore(store sqlstore.SQLStore) (storage.OpenFGADatastore, error) {
switch store.BunDB().Dialect().Name().String() {
case "sqlite":
return sqlite.NewWithDB(store.SQLDB(), &sqlcommon.Config{
MaxTuplesPerWriteField: 100,
MaxTypesPerModelField: 100,
})
case "pg":
pgStore, ok := store.(postgressqlstore.Pooler)
if !ok {
panic(errors.New(errors.TypeInternal, errors.CodeInternal, "postgressqlstore should implement Pooler"))
}
return postgres.NewWithDB(pgStore.Pool(), nil, &sqlcommon.Config{
MaxTuplesPerWriteField: 100,
MaxTypesPerModelField: 100,
})
}
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid store type: %s", store.BunDB().Dialect().Name().String())
}

View File

@@ -13,7 +13,7 @@ var (
once sync.Once
)
// Config initializes the licensing configuration.
// initializes the licensing configuration
func Config(pollInterval time.Duration, failureThreshold int) licensing.Config {
once.Do(func() {
config = licensing.Config{PollInterval: pollInterval, FailureThreshold: failureThreshold}

View File

@@ -1,127 +0,0 @@
package implcloudprovider
import (
"context"
"fmt"
"net/url"
"sort"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
)
type awscloudprovider struct {
serviceDefinitions cloudintegrationtypes.ServiceDefinitionStore
}
func NewAWSCloudProvider(defStore cloudintegrationtypes.ServiceDefinitionStore) (cloudintegration.CloudProviderModule, error) {
return &awscloudprovider{serviceDefinitions: defStore}, nil
}
func (provider *awscloudprovider) GetConnectionArtifact(ctx context.Context, account *cloudintegrationtypes.Account, req *cloudintegrationtypes.GetConnectionArtifactRequest) (*cloudintegrationtypes.ConnectionArtifact, error) {
baseURL := fmt.Sprintf(cloudintegrationtypes.CloudFormationQuickCreateBaseURL.StringValue(), req.Config.AWS.DeploymentRegion)
u, _ := url.Parse(baseURL)
q := u.Query()
q.Set("region", req.Config.AWS.DeploymentRegion)
u.Fragment = "/stacks/quickcreate"
u.RawQuery = q.Encode()
q = u.Query()
q.Set("stackName", cloudintegrationtypes.AgentCloudFormationBaseStackName.StringValue())
q.Set("templateURL", fmt.Sprintf(cloudintegrationtypes.AgentCloudFormationTemplateS3Path.StringValue(), req.Config.AgentVersion))
q.Set("param_SigNozIntegrationAgentVersion", req.Config.AgentVersion)
q.Set("param_SigNozApiUrl", req.Credentials.SigNozAPIURL)
q.Set("param_SigNozApiKey", req.Credentials.SigNozAPIKey)
q.Set("param_SigNozAccountId", account.ID.StringValue())
q.Set("param_IngestionUrl", req.Credentials.IngestionURL)
q.Set("param_IngestionKey", req.Credentials.IngestionKey)
return &cloudintegrationtypes.ConnectionArtifact{
AWS: cloudintegrationtypes.NewAWSConnectionArtifact(u.String() + "?&" + q.Encode()), // this format is required by AWS
}, nil
}
func (provider *awscloudprovider) ListServiceDefinitions(ctx context.Context) ([]*cloudintegrationtypes.ServiceDefinition, error) {
return provider.serviceDefinitions.List(ctx, cloudintegrationtypes.CloudProviderTypeAWS)
}
func (provider *awscloudprovider) GetServiceDefinition(ctx context.Context, serviceID cloudintegrationtypes.ServiceID) (*cloudintegrationtypes.ServiceDefinition, error) {
serviceDef, err := provider.serviceDefinitions.Get(ctx, cloudintegrationtypes.CloudProviderTypeAWS, serviceID)
if err != nil {
return nil, err
}
// override cloud integration dashboard id
for index, dashboard := range serviceDef.Assets.Dashboards {
serviceDef.Assets.Dashboards[index].ID = cloudintegrationtypes.GetCloudIntegrationDashboardID(cloudintegrationtypes.CloudProviderTypeAWS, serviceID.StringValue(), dashboard.ID)
}
return serviceDef, nil
}
func (provider *awscloudprovider) BuildIntegrationConfig(
ctx context.Context,
account *cloudintegrationtypes.Account,
services []*cloudintegrationtypes.StorableCloudIntegrationService,
) (*cloudintegrationtypes.ProviderIntegrationConfig, error) {
// Sort services for deterministic output
sort.Slice(services, func(i, j int) bool {
return services[i].Type.StringValue() < services[j].Type.StringValue()
})
compiledMetrics := new(cloudintegrationtypes.AWSMetricsCollectionStrategy)
compiledLogs := new(cloudintegrationtypes.AWSLogsCollectionStrategy)
var compiledS3Buckets map[string][]string
for _, storedSvc := range services {
svcCfg, err := cloudintegrationtypes.NewServiceConfigFromJSON(cloudintegrationtypes.CloudProviderTypeAWS, storedSvc.Config)
if err != nil {
return nil, err
}
svcDef, err := provider.GetServiceDefinition(ctx, storedSvc.Type)
if err != nil {
return nil, err
}
strategy := svcDef.TelemetryCollectionStrategy.AWS
logsEnabled := svcCfg.IsLogsEnabled(cloudintegrationtypes.CloudProviderTypeAWS)
// S3Sync: logs come directly from configured S3 buckets, not CloudWatch subscriptions
if storedSvc.Type == cloudintegrationtypes.AWSServiceS3Sync {
if logsEnabled && svcCfg.AWS.Logs.S3Buckets != nil {
compiledS3Buckets = svcCfg.AWS.Logs.S3Buckets
}
// no need to go ahead as the code block specifically checks for the S3Sync service
continue
}
if logsEnabled && strategy.Logs != nil {
compiledLogs.Subscriptions = append(compiledLogs.Subscriptions, strategy.Logs.Subscriptions...)
}
metricsEnabled := svcCfg.IsMetricsEnabled(cloudintegrationtypes.CloudProviderTypeAWS)
if metricsEnabled && strategy.Metrics != nil {
compiledMetrics.StreamFilters = append(compiledMetrics.StreamFilters, strategy.Metrics.StreamFilters...)
}
}
collectionStrategy := new(cloudintegrationtypes.AWSTelemetryCollectionStrategy)
if len(compiledMetrics.StreamFilters) > 0 {
collectionStrategy.Metrics = compiledMetrics
}
if len(compiledLogs.Subscriptions) > 0 {
collectionStrategy.Logs = compiledLogs
}
if compiledS3Buckets != nil {
collectionStrategy.S3Buckets = compiledS3Buckets
}
return &cloudintegrationtypes.ProviderIntegrationConfig{
AWS: cloudintegrationtypes.NewAWSIntegrationConfig(account.Config.AWS.Regions, collectionStrategy),
}, nil
}

View File

@@ -1,34 +0,0 @@
package implcloudprovider
import (
"context"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
)
type azurecloudprovider struct{}
func NewAzureCloudProvider() cloudintegration.CloudProviderModule {
return &azurecloudprovider{}
}
func (provider *azurecloudprovider) GetConnectionArtifact(ctx context.Context, account *cloudintegrationtypes.Account, req *cloudintegrationtypes.GetConnectionArtifactRequest) (*cloudintegrationtypes.ConnectionArtifact, error) {
panic("implement me")
}
func (provider *azurecloudprovider) ListServiceDefinitions(ctx context.Context) ([]*cloudintegrationtypes.ServiceDefinition, error) {
panic("implement me")
}
func (provider *azurecloudprovider) GetServiceDefinition(ctx context.Context, serviceID cloudintegrationtypes.ServiceID) (*cloudintegrationtypes.ServiceDefinition, error) {
panic("implement me")
}
func (provider *azurecloudprovider) BuildIntegrationConfig(
ctx context.Context,
account *cloudintegrationtypes.Account,
services []*cloudintegrationtypes.StorableCloudIntegrationService,
) (*cloudintegrationtypes.ProviderIntegrationConfig, error) {
panic("implement me")
}

View File

@@ -1,540 +0,0 @@
package implcloudintegration
import (
"context"
"fmt"
"sort"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/serviceaccounttypes"
"github.com/SigNoz/signoz/pkg/types/zeustypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/SigNoz/signoz/pkg/zeus"
)
type module struct {
store cloudintegrationtypes.Store
gateway gateway.Gateway
zeus zeus.Zeus
licensing licensing.Licensing
global global.Global
serviceAccount serviceaccount.Module
cloudProvidersMap map[cloudintegrationtypes.CloudProviderType]cloudintegration.CloudProviderModule
config cloudintegration.Config
}
func NewModule(
store cloudintegrationtypes.Store,
global global.Global,
zeus zeus.Zeus,
gateway gateway.Gateway,
licensing licensing.Licensing,
serviceAccount serviceaccount.Module,
cloudProvidersMap map[cloudintegrationtypes.CloudProviderType]cloudintegration.CloudProviderModule,
config cloudintegration.Config,
) (cloudintegration.Module, error) {
return &module{
store: store,
global: global,
zeus: zeus,
gateway: gateway,
licensing: licensing,
serviceAccount: serviceAccount,
cloudProvidersMap: cloudProvidersMap,
config: config,
}, nil
}
// GetConnectionCredentials returns credentials required to generate connection artifact. eg. apiKey, ingestionKey etc.
// It will return creds it can deduce and return empty value for others.
func (module *module) GetConnectionCredentials(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (*cloudintegrationtypes.Credentials, error) {
// get license to get the deployment details
license, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
// get deployment details from zeus, ignore error
respBytes, _ := module.zeus.GetDeployment(ctx, license.Key)
var signozAPIURL string
if len(respBytes) > 0 {
// parse deployment details, ignore error, if client is asking api url every time check for possible error
deployment, _ := zeustypes.NewGettableDeployment(respBytes)
if deployment != nil {
signozAPIURL, _ = cloudintegrationtypes.GetSigNozAPIURLFromDeployment(deployment)
}
}
// ignore error
apiKey, _ := module.getOrCreateAPIKey(ctx, orgID, provider)
// ignore error
ingestionKey, _ := module.getOrCreateIngestionKey(ctx, orgID, provider)
return cloudintegrationtypes.NewCredentials(
signozAPIURL,
apiKey,
module.global.GetConfig(ctx).IngestionURL,
ingestionKey,
), nil
}
func (module *module) CreateAccount(ctx context.Context, account *cloudintegrationtypes.Account) error {
_, err := module.licensing.GetActive(ctx, account.OrgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableCloudIntegration, err := cloudintegrationtypes.NewStorableCloudIntegration(account)
if err != nil {
return err
}
return module.store.CreateAccount(ctx, storableCloudIntegration)
}
func (module *module) GetConnectionArtifact(ctx context.Context, account *cloudintegrationtypes.Account, req *cloudintegrationtypes.GetConnectionArtifactRequest) (*cloudintegrationtypes.ConnectionArtifact, error) {
_, err := module.licensing.GetActive(ctx, account.OrgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProviderModule, err := module.getCloudProvider(account.Provider)
if err != nil {
return nil, err
}
req.Config.SetAgentVersion(module.config.Agent.Version)
return cloudProviderModule.GetConnectionArtifact(ctx, account, req)
}
func (module *module) GetAccount(ctx context.Context, orgID valuer.UUID, accountID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (*cloudintegrationtypes.Account, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccount, err := module.store.GetAccountByID(ctx, orgID, accountID, provider)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAccountFromStorable(storableAccount)
}
func (module *module) GetConnectedAccount(ctx context.Context, orgID, accountID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (*cloudintegrationtypes.Account, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccount, err := module.store.GetConnectedAccount(ctx, orgID, accountID, provider)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAccountFromStorable(storableAccount)
}
// ListAccounts return only agent connected accounts.
func (module *module) ListAccounts(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) ([]*cloudintegrationtypes.Account, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccounts, err := module.store.ListConnectedAccounts(ctx, orgID, provider)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAccountsFromStorables(storableAccounts)
}
func (module *module) AgentCheckIn(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType, req *cloudintegrationtypes.AgentCheckInRequest) (*cloudintegrationtypes.AgentCheckInResponse, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
connectedAccount, err := module.store.GetConnectedAccountByProviderAccountID(ctx, orgID, req.ProviderAccountID, provider)
if err != nil && !errors.Ast(err, errors.TypeNotFound) {
return nil, err
}
// If a different integration is already connected to this provider account ID, reject the check-in.
// Allow re-check-in from the same integration (e.g. agent restarting).
if connectedAccount != nil && connectedAccount.ID != req.CloudIntegrationID {
errMessage := fmt.Sprintf("provider account id %s is already connected to cloud integration id %s", req.ProviderAccountID, connectedAccount.ID)
return nil, errors.New(errors.TypeAlreadyExists, cloudintegrationtypes.ErrCodeCloudIntegrationAlreadyConnected, errMessage)
}
account, err := module.store.GetAccountByID(ctx, orgID, req.CloudIntegrationID, provider)
if err != nil {
return nil, err
}
// If account has been removed (disconnected), return a minimal response with empty integration config.
// The agent uses this response to clean up resources
if account.RemovedAt != nil {
return cloudintegrationtypes.NewAgentCheckInResponse(
req.ProviderAccountID,
account.ID.StringValue(),
new(cloudintegrationtypes.ProviderIntegrationConfig),
account.RemovedAt,
), nil
}
// update account with cloud provider account id and agent report (heartbeat)
account.Update(&req.ProviderAccountID, cloudintegrationtypes.NewAgentReport(req.Data))
err = module.store.UpdateAccount(ctx, account)
if err != nil {
return nil, err
}
// Get account as domain object for config access (enabled regions, etc.)
domainAccount, err := cloudintegrationtypes.NewAccountFromStorable(account)
if err != nil {
return nil, err
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
storedServices, err := module.store.ListServices(ctx, req.CloudIntegrationID)
if err != nil {
return nil, err
}
// Delegate integration config building entirely to the provider module
integrationConfig, err := cloudProvider.BuildIntegrationConfig(ctx, domainAccount, storedServices)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAgentCheckInResponse(
req.ProviderAccountID,
account.ID.StringValue(),
integrationConfig,
account.RemovedAt,
), nil
}
func (module *module) UpdateAccount(ctx context.Context, account *cloudintegrationtypes.Account) error {
_, err := module.licensing.GetActive(ctx, account.OrgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccount, err := cloudintegrationtypes.NewStorableCloudIntegration(account)
if err != nil {
return err
}
return module.store.UpdateAccount(ctx, storableAccount)
}
func (module *module) DisconnectAccount(ctx context.Context, orgID valuer.UUID, accountID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) error {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
return module.store.RemoveAccount(ctx, orgID, accountID, provider)
}
func (module *module) ListServicesMetadata(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType, integrationID valuer.UUID) ([]*cloudintegrationtypes.ServiceMetadata, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
serviceDefinitions, err := cloudProvider.ListServiceDefinitions(ctx)
if err != nil {
return nil, err
}
enabledServiceIDs := map[string]bool{}
if !integrationID.IsZero() {
storedServices, err := module.store.ListServices(ctx, integrationID)
if err != nil {
return nil, err
}
for _, svc := range storedServices {
serviceConfig, err := cloudintegrationtypes.NewServiceConfigFromJSON(provider, svc.Config)
if err != nil {
return nil, err
}
if serviceConfig.IsServiceEnabled(provider) {
enabledServiceIDs[svc.Type.StringValue()] = true
}
}
}
resp := make([]*cloudintegrationtypes.ServiceMetadata, 0, len(serviceDefinitions))
for _, serviceDefinition := range serviceDefinitions {
resp = append(resp, cloudintegrationtypes.NewServiceMetadata(*serviceDefinition, enabledServiceIDs[serviceDefinition.ID]))
}
return resp, nil
}
func (module *module) GetService(ctx context.Context, orgID valuer.UUID, serviceID cloudintegrationtypes.ServiceID, provider cloudintegrationtypes.CloudProviderType, cloudIntegrationID valuer.UUID) (*cloudintegrationtypes.Service, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
serviceDefinition, err := cloudProvider.GetServiceDefinition(ctx, serviceID)
if err != nil {
return nil, err
}
var integrationService *cloudintegrationtypes.CloudIntegrationService
if !cloudIntegrationID.IsZero() {
storedService, err := module.store.GetServiceByServiceID(ctx, cloudIntegrationID, serviceID)
if err != nil && !errors.Ast(err, errors.TypeNotFound) {
return nil, err
}
if storedService != nil {
serviceConfig, err := cloudintegrationtypes.NewServiceConfigFromJSON(provider, storedService.Config)
if err != nil {
return nil, err
}
integrationService = cloudintegrationtypes.NewCloudIntegrationServiceFromStorable(storedService, serviceConfig)
}
}
return cloudintegrationtypes.NewService(*serviceDefinition, integrationService), nil
}
func (module *module) CreateService(ctx context.Context, orgID valuer.UUID, service *cloudintegrationtypes.CloudIntegrationService, provider cloudintegrationtypes.CloudProviderType) error {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return err
}
serviceDefinition, err := cloudProvider.GetServiceDefinition(ctx, service.Type)
if err != nil {
return err
}
configJSON, err := service.Config.ToJSON(provider, service.Type, &serviceDefinition.SupportedSignals)
if err != nil {
return err
}
return module.store.CreateService(ctx, cloudintegrationtypes.NewStorableCloudIntegrationService(service, string(configJSON)))
}
func (module *module) UpdateService(ctx context.Context, orgID valuer.UUID, integrationService *cloudintegrationtypes.CloudIntegrationService, provider cloudintegrationtypes.CloudProviderType) error {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return err
}
serviceDefinition, err := cloudProvider.GetServiceDefinition(ctx, integrationService.Type)
if err != nil {
return err
}
configJSON, err := integrationService.Config.ToJSON(provider, integrationService.Type, &serviceDefinition.SupportedSignals)
if err != nil {
return err
}
storableService := cloudintegrationtypes.NewStorableCloudIntegrationService(integrationService, string(configJSON))
return module.store.UpdateService(ctx, storableService)
}
func (module *module) GetDashboardByID(ctx context.Context, orgID valuer.UUID, id string) (*dashboardtypes.Dashboard, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
_, _, _, err = cloudintegrationtypes.ParseCloudIntegrationDashboardID(id)
if err != nil {
return nil, err
}
allDashboards, err := module.listDashboards(ctx, orgID)
if err != nil {
return nil, err
}
for _, d := range allDashboards {
if d.ID == id {
return d, nil
}
}
return nil, errors.New(errors.TypeNotFound, cloudintegrationtypes.ErrCodeCloudIntegrationNotFound, "cloud integration dashboard not found")
}
func (module *module) ListDashboards(ctx context.Context, orgID valuer.UUID) ([]*dashboardtypes.Dashboard, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
return module.listDashboards(ctx, orgID)
}
func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
stats := make(map[string]any)
// get connected accounts for AWS
awsAccountsCount, err := module.store.CountConnectedAccounts(ctx, orgID, cloudintegrationtypes.CloudProviderTypeAWS)
if err == nil {
stats["cloudintegration.aws.connectedaccounts.count"] = awsAccountsCount
}
// NOTE: not adding stats for services for now.
// TODO: add more cloud providers when supported
return stats, nil
}
func (module *module) getCloudProvider(provider cloudintegrationtypes.CloudProviderType) (cloudintegration.CloudProviderModule, error) {
if cloudProviderModule, ok := module.cloudProvidersMap[provider]; ok {
return cloudProviderModule, nil
}
return nil, errors.NewInvalidInputf(cloudintegrationtypes.ErrCodeCloudProviderInvalidInput, "invalid cloud provider: %s", provider.StringValue())
}
func (module *module) getOrCreateIngestionKey(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (string, error) {
keyName := cloudintegrationtypes.NewIngestionKeyName(provider)
result, err := module.gateway.SearchIngestionKeysByName(ctx, orgID, keyName, 1, 10)
if err != nil {
return "", errors.WrapInternalf(err, errors.CodeInternal, "couldn't search ingestion keys")
}
// ideally there should be only one key per cloud integration provider
if len(result.Keys) > 0 {
return result.Keys[0].Value, nil
}
createdIngestionKey, err := module.gateway.CreateIngestionKey(ctx, orgID, keyName, []string{"integration"}, time.Time{})
if err != nil {
return "", errors.WrapInternalf(err, errors.CodeInternal, "couldn't create ingestion key")
}
return createdIngestionKey.Value, nil
}
func (module *module) getOrCreateAPIKey(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (string, error) {
domain := module.serviceAccount.Config().Email.Domain
serviceAccount := serviceaccounttypes.NewServiceAccount("integration", domain, serviceaccounttypes.ServiceAccountStatusActive, orgID)
serviceAccount, err := module.serviceAccount.GetOrCreate(ctx, orgID, serviceAccount)
if err != nil {
return "", err
}
err = module.serviceAccount.SetRoleByName(ctx, orgID, serviceAccount.ID, authtypes.SigNozViewerRoleName)
if err != nil {
return "", err
}
factorAPIKey, err := serviceAccount.NewFactorAPIKey(provider.StringValue(), 0)
if err != nil {
return "", err
}
factorAPIKey, err = module.serviceAccount.GetOrCreateFactorAPIKey(ctx, factorAPIKey)
if err != nil {
return "", err
}
return factorAPIKey.Key, nil
}
func (module *module) listDashboards(ctx context.Context, orgID valuer.UUID) ([]*dashboardtypes.Dashboard, error) {
var allDashboards []*dashboardtypes.Dashboard
for provider := range module.cloudProvidersMap {
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
connectedAccounts, err := module.store.ListConnectedAccounts(ctx, orgID, provider)
if err != nil {
return nil, err
}
for _, storableAccount := range connectedAccounts {
storedServices, err := module.store.ListServices(ctx, storableAccount.ID)
if err != nil {
return nil, err
}
for _, storedSvc := range storedServices {
serviceConfig, err := cloudintegrationtypes.NewServiceConfigFromJSON(provider, storedSvc.Config)
if err != nil || !serviceConfig.IsMetricsEnabled(provider) {
continue
}
svcDef, err := cloudProvider.GetServiceDefinition(ctx, storedSvc.Type)
if err != nil || svcDef == nil {
continue
}
dashboards := cloudintegrationtypes.GetDashboardsFromAssets(
storedSvc.Type.StringValue(),
orgID,
provider,
storableAccount.CreatedAt,
svcDef.Assets,
)
allDashboards = append(allDashboards, dashboards...)
}
}
}
sort.Slice(allDashboards, func(i, j int) bool {
return allDashboards[i].ID < allDashboards[j].ID
})
return allDashboards, nil
}

View File

@@ -213,8 +213,8 @@ func (module *module) Update(ctx context.Context, orgID valuer.UUID, id valuer.U
return module.pkgDashboardModule.Update(ctx, orgID, id, updatedBy, data, diff)
}
func (module *module) LockUnlock(ctx context.Context, orgID valuer.UUID, id valuer.UUID, updatedBy string, isAdmin bool, lock bool) error {
return module.pkgDashboardModule.LockUnlock(ctx, orgID, id, updatedBy, isAdmin, lock)
func (module *module) LockUnlock(ctx context.Context, orgID valuer.UUID, id valuer.UUID, updatedBy string, role types.Role, lock bool) error {
return module.pkgDashboardModule.LockUnlock(ctx, orgID, id, updatedBy, role, lock)
}
func (module *module) MustGetTypeables() []authtypes.Typeable {

View File

@@ -65,7 +65,7 @@ func (h *handler) QueryRange(rw http.ResponseWriter, req *http.Request) {
}
if anomalyQuery, ok := queryRangeRequest.IsAnomalyRequest(); ok {
anomalies, err := h.handleAnomalyQuery(ctx, orgID, anomalyQuery, &queryRangeRequest)
anomalies, err := h.handleAnomalyQuery(ctx, orgID, anomalyQuery, queryRangeRequest)
if err != nil {
render.Error(rw, errors.NewInternalf(errors.CodeInternal, "failed to get anomalies: %v", err))
return
@@ -79,7 +79,7 @@ func (h *handler) QueryRange(rw http.ResponseWriter, req *http.Request) {
// Build step intervals from the anomaly query
stepIntervals := make(map[string]uint64)
if anomalyQuery.StepInterval.Duration > 0 {
stepIntervals[anomalyQuery.Name] = uint64(anomalyQuery.StepInterval.Seconds())
stepIntervals[anomalyQuery.Name] = uint64(anomalyQuery.StepInterval.Duration.Seconds())
}
finalResp := &qbtypes.QueryRangeResponse{
@@ -149,7 +149,7 @@ func (h *handler) createAnomalyProvider(seasonality anomalyV2.Seasonality) anoma
}
}
func (h *handler) handleAnomalyQuery(ctx context.Context, orgID valuer.UUID, anomalyQuery *qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation], queryRangeRequest *qbtypes.QueryRangeRequest) (*anomalyV2.AnomaliesResponse, error) {
func (h *handler) handleAnomalyQuery(ctx context.Context, orgID valuer.UUID, anomalyQuery *qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation], queryRangeRequest qbtypes.QueryRangeRequest) (*anomalyV2.AnomaliesResponse, error) {
seasonality := extractSeasonality(anomalyQuery)
provider := h.createAnomalyProvider(seasonality)

View File

@@ -6,6 +6,7 @@ import (
"github.com/SigNoz/signoz/ee/licensing/httplicensing"
"github.com/SigNoz/signoz/ee/query-service/usage"
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/http/middleware"
baseapp "github.com/SigNoz/signoz/pkg/query-service/app"
@@ -14,6 +15,7 @@ import (
"github.com/SigNoz/signoz/pkg/query-service/app/logparsingpipeline"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
basemodel "github.com/SigNoz/signoz/pkg/query-service/model"
rules "github.com/SigNoz/signoz/pkg/query-service/rules"
"github.com/SigNoz/signoz/pkg/queryparser"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/version"
@@ -22,6 +24,7 @@ import (
type APIHandlerOptions struct {
DataConnector interfaces.Reader
RulesManager *rules.Manager
UsageManager *usage.Manager
IntegrationsController *integrations.Controller
CloudIntegrationsController *cloudintegrations.Controller
@@ -41,10 +44,12 @@ type APIHandler struct {
func NewAPIHandler(opts APIHandlerOptions, signoz *signoz.SigNoz, config signoz.Config) (*APIHandler, error) {
baseHandler, err := baseapp.NewAPIHandler(baseapp.APIHandlerOpts{
Reader: opts.DataConnector,
RuleManager: opts.RulesManager,
IntegrationsController: opts.IntegrationsController,
CloudIntegrationsController: opts.CloudIntegrationsController,
LogsParsingPipelineController: opts.LogsParsingPipelineController,
FluxInterval: opts.FluxInterval,
AlertmanagerAPI: alertmanager.NewAPI(signoz.Alertmanager),
LicensingAPI: httplicensing.NewLicensingAPI(signoz.Licensing),
Signoz: signoz,
QueryParserAPI: queryparser.NewAPI(signoz.Instrumentation.ToProviderSettings(), signoz.QueryParser),
@@ -61,6 +66,10 @@ func NewAPIHandler(opts APIHandlerOptions, signoz *signoz.SigNoz, config signoz.
return ah, nil
}
func (ah *APIHandler) RM() *rules.Manager {
return ah.opts.RulesManager
}
func (ah *APIHandler) UM() *usage.Manager {
return ah.opts.UsageManager
}

View File

@@ -14,9 +14,10 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/modules/user"
basemodel "github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/serviceaccounttypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
)
@@ -49,7 +50,7 @@ func (ah *APIHandler) CloudIntegrationsGenerateConnectionParams(w http.ResponseW
return
}
apiKey, apiErr := ah.getOrCreateCloudIntegrationFactorAPIKey(r.Context(), valuer.MustNewUUID(claims.OrgID), cloudProvider)
apiKey, apiErr := ah.getOrCreateCloudIntegrationPAT(r.Context(), claims.OrgID, cloudProvider)
if apiErr != nil {
RespondError(w, basemodel.WrapApiError(
apiErr, "couldn't provision PAT for cloud integration:",
@@ -109,44 +110,84 @@ func (ah *APIHandler) CloudIntegrationsGenerateConnectionParams(w http.ResponseW
ah.Respond(w, result)
}
func (ah *APIHandler) getOrCreateCloudIntegrationFactorAPIKey(ctx context.Context, orgID valuer.UUID, cloudProvider string) (
func (ah *APIHandler) getOrCreateCloudIntegrationPAT(ctx context.Context, orgId string, cloudProvider string) (
string, *basemodel.ApiError,
) {
integrationPATName := fmt.Sprintf("%s", cloudProvider)
serviceAccount, apiErr := ah.getOrCreateCloudIntegrationServiceAccount(ctx, orgID)
integrationPATName := fmt.Sprintf("%s integration", cloudProvider)
integrationUser, apiErr := ah.getOrCreateCloudIntegrationUser(ctx, orgId, cloudProvider)
if apiErr != nil {
return "", apiErr
}
factorAPIKey, err := serviceAccount.NewFactorAPIKey(integrationPATName, 0)
orgIdUUID, err := valuer.NewUUID(orgId)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't parse orgId: %w", err,
))
}
allPats, err := ah.Signoz.Modules.UserSetter.ListAPIKeys(ctx, orgIdUUID)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't list PATs: %w", err,
))
}
for _, p := range allPats {
if p.UserID == integrationUser.ID && p.Name == integrationPATName {
return p.Token, nil
}
}
slog.InfoContext(ctx, "no PAT found for cloud integration, creating a new one",
"cloud_provider", cloudProvider,
)
newPAT, err := types.NewStorableAPIKey(
integrationPATName,
integrationUser.ID,
types.RoleViewer,
0,
)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't create cloud integration PAT: %w", err,
))
}
factorAPIKey, err = ah.Signoz.Modules.ServiceAccount.GetOrCreateFactorAPIKey(ctx, factorAPIKey)
err = ah.Signoz.Modules.UserSetter.CreateAPIKey(ctx, newPAT)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't create cloud integration PAT: %w", err,
))
}
return factorAPIKey.Key, nil
return newPAT.Token, nil
}
func (ah *APIHandler) getOrCreateCloudIntegrationServiceAccount(ctx context.Context, orgId valuer.UUID) (*serviceaccounttypes.ServiceAccount, *basemodel.ApiError) {
domain := ah.Signoz.Modules.ServiceAccount.Config().Email.Domain
cloudIntegrationServiceAccount := serviceaccounttypes.NewServiceAccount("integration", domain, serviceaccounttypes.ServiceAccountStatusActive, orgId)
cloudIntegrationServiceAccount, err := ah.Signoz.Modules.ServiceAccount.GetOrCreate(ctx, orgId, cloudIntegrationServiceAccount)
func (ah *APIHandler) getOrCreateCloudIntegrationUser(
ctx context.Context, orgId string, cloudProvider string,
) (*types.User, *basemodel.ApiError) {
cloudIntegrationUserName := fmt.Sprintf("%s-integration", cloudProvider)
email := valuer.MustNewEmail(fmt.Sprintf("%s@signoz.io", cloudIntegrationUserName))
cloudIntegrationUser, err := types.NewUser(cloudIntegrationUserName, email, valuer.MustNewUUID(orgId), types.UserStatusActive)
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration service account: %w", err))
}
err = ah.Signoz.Modules.ServiceAccount.SetRoleByName(ctx, orgId, cloudIntegrationServiceAccount.ID, authtypes.SigNozViewerRoleName)
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration service account: %w", err))
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration user: %w", err))
}
return cloudIntegrationServiceAccount, nil
password := types.MustGenerateFactorPassword(cloudIntegrationUser.ID.StringValue())
cloudIntegrationUser, err = ah.Signoz.Modules.UserSetter.GetOrCreateUser(
ctx,
cloudIntegrationUser,
user.WithFactorPassword(password),
user.WithRoleNames([]string{authtypes.SigNozViewerRoleName}),
)
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't look for integration user: %w", err))
}
return cloudIntegrationUser, nil
}
func (ah *APIHandler) getIngestionUrlAndSigNozAPIUrl(ctx context.Context, licenseKey string) (

View File

@@ -71,15 +71,6 @@ func (ah *APIHandler) getFeatureFlags(w http.ResponseWriter, r *http.Request) {
Route: "",
})
bodyJSONQuery := ah.Signoz.Flagger.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureBodyJSONQuery.String()),
Active: bodyJSONQuery,
Usage: 0,
UsageLimit: -1,
Route: "",
})
if constants.IsDotMetricsEnabled {
for idx, feature := range featureSet {
if feature.Name == licensetypes.DotMetricsEnabled {

View File

@@ -7,10 +7,6 @@ import (
"github.com/SigNoz/signoz/ee/query-service/constants"
"github.com/SigNoz/signoz/ee/query-service/model"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type DayWiseBreakdown struct {
@@ -49,17 +45,15 @@ type details struct {
BillTotal float64 `json:"billTotal"`
}
type billingData struct {
BillingPeriodStart int64 `json:"billingPeriodStart"`
BillingPeriodEnd int64 `json:"billingPeriodEnd"`
Details details `json:"details"`
Discount float64 `json:"discount"`
SubscriptionStatus string `json:"subscriptionStatus"`
}
type billingDetails struct {
Status string `json:"status"`
Data billingData `json:"data"`
Status string `json:"status"`
Data struct {
BillingPeriodStart int64 `json:"billingPeriodStart"`
BillingPeriodEnd int64 `json:"billingPeriodEnd"`
Details details `json:"details"`
Discount float64 `json:"discount"`
SubscriptionStatus string `json:"subscriptionStatus"`
} `json:"data"`
}
func (ah *APIHandler) getBilling(w http.ResponseWriter, r *http.Request) {
@@ -70,33 +64,6 @@ func (ah *APIHandler) getBilling(w http.ResponseWriter, r *http.Request) {
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)
useZeus := ah.Signoz.Flagger.BooleanOrEmpty(r.Context(), flagger.FeatureGetMetersFromZeus, evalCtx)
if useZeus {
data, err := ah.Signoz.Zeus.GetMeters(r.Context(), licenseKey)
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
var billing billingData
if err := json.Unmarshal(data, &billing); err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
ah.Respond(w, billing)
return
}
billingURL := fmt.Sprintf("%s/usage?licenseKey=%s", constants.LicenseSignozIo, licenseKey)
hClient := &http.Client{}
@@ -112,11 +79,13 @@ func (ah *APIHandler) getBilling(w http.ResponseWriter, r *http.Request) {
return
}
// decode response body
var billingResponse billingDetails
if err := json.NewDecoder(billingResp.Body).Decode(&billingResponse); err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
// TODO(srikanthccv):Fetch the current day usage and add it to the response
ah.Respond(w, billingResponse.Data)
}

View File

@@ -12,6 +12,10 @@ import (
"github.com/SigNoz/signoz/pkg/cache/memorycache"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/queryparser"
"github.com/SigNoz/signoz/pkg/ruler/rulestore/sqlrulestore"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/gorilla/handlers"
@@ -19,10 +23,17 @@ import (
"github.com/soheilhy/cmux"
"github.com/SigNoz/signoz/ee/query-service/app/api"
"github.com/SigNoz/signoz/ee/query-service/rules"
"github.com/SigNoz/signoz/ee/query-service/usage"
"github.com/SigNoz/signoz/pkg/alertmanager"
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/http/middleware"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/signoz"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/web"
"log/slog"
@@ -37,13 +48,16 @@ import (
opAmpModel "github.com/SigNoz/signoz/pkg/query-service/app/opamp/model"
baseconst "github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/healthcheck"
baseint "github.com/SigNoz/signoz/pkg/query-service/interfaces"
baserules "github.com/SigNoz/signoz/pkg/query-service/rules"
"github.com/SigNoz/signoz/pkg/query-service/utils"
)
// Server runs HTTP, Mux and a grpc server
type Server struct {
config signoz.Config
signoz *signoz.SigNoz
config signoz.Config
signoz *signoz.SigNoz
ruleManager *baserules.Manager
// public http router
httpConn net.Listener
@@ -83,6 +97,24 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
nil,
)
rm, err := makeRulesManager(
reader,
signoz.Cache,
signoz.Alertmanager,
signoz.SQLStore,
signoz.TelemetryStore,
signoz.TelemetryMetadataStore,
signoz.Prometheus,
signoz.Modules.OrgGetter,
signoz.Querier,
signoz.Instrumentation.ToProviderSettings(),
signoz.QueryParser,
)
if err != nil {
return nil, err
}
// initiate opamp
opAmpModel.Init(signoz.SQLStore, signoz.Instrumentation.Logger(), signoz.Modules.OrgGetter)
@@ -104,8 +136,6 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
logParsingPipelineController, err := logparsingpipeline.NewLogParsingPipelinesController(
signoz.SQLStore,
integrationsController.GetPipelinesForInstalledIntegrations,
reader,
signoz.Flagger,
)
if err != nil {
return nil, err
@@ -121,7 +151,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
}
// start the usagemanager
usageManager, err := usage.New(signoz.Licensing, signoz.TelemetryStore.ClickhouseDB(), signoz.Zeus, signoz.Modules.OrgGetter, signoz.Flagger)
usageManager, err := usage.New(signoz.Licensing, signoz.TelemetryStore.ClickhouseDB(), signoz.Zeus, signoz.Modules.OrgGetter)
if err != nil {
return nil, err
}
@@ -132,6 +162,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
apiOpts := api.APIHandlerOptions{
DataConnector: reader,
RulesManager: rm,
UsageManager: usageManager,
IntegrationsController: integrationsController,
CloudIntegrationsController: cloudIntegrationsController,
@@ -149,6 +180,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
s := &Server{
config: config,
signoz: signoz,
ruleManager: rm,
httpHostPort: baseconst.HTTPHostPort,
unavailableChannel: make(chan healthcheck.Status),
usageManager: usageManager,
@@ -194,7 +226,7 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
s.config.APIServer.Timeout.Default,
s.config.APIServer.Timeout.Max,
).Wrap)
r.Use(middleware.NewAudit(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes, s.signoz.Auditor).Wrap)
r.Use(middleware.NewLogging(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes).Wrap)
r.Use(middleware.NewComment().Wrap)
apiHandler.RegisterRoutes(r, am)
@@ -207,6 +239,7 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
apiHandler.RegisterWebSocketPaths(r, am)
apiHandler.RegisterMessagingQueuesRoutes(r, am)
apiHandler.RegisterThirdPartyApiRoutes(r, am)
apiHandler.MetricExplorerRoutes(r, am)
apiHandler.RegisterTraceFunnelsRoutes(r, am)
err := s.signoz.APIServer.AddToRouter(r)
@@ -229,20 +262,6 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
return nil, err
}
routePrefix := s.config.Global.ExternalPath()
if routePrefix != "" {
prefixed := http.StripPrefix(routePrefix, handler)
handler = http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/api/v1/health", "/api/v2/healthz", "/api/v2/readyz", "/api/v2/livez":
r.ServeHTTP(w, req)
return
}
prefixed.ServeHTTP(w, req)
})
}
return &http.Server{
Handler: handler,
}, nil
@@ -269,6 +288,8 @@ func (s *Server) initListeners() error {
// Start listening on http and private http port concurrently
func (s *Server) Start(ctx context.Context) error {
s.ruleManager.Start(ctx)
err := s.initListeners()
if err != nil {
return err
@@ -312,8 +333,47 @@ func (s *Server) Stop(ctx context.Context) error {
s.opampServer.Stop()
if s.ruleManager != nil {
s.ruleManager.Stop(ctx)
}
// stop usage manager
s.usageManager.Stop(ctx)
return nil
}
func makeRulesManager(ch baseint.Reader, cache cache.Cache, alertmanager alertmanager.Alertmanager, sqlstore sqlstore.SQLStore, telemetryStore telemetrystore.TelemetryStore, metadataStore telemetrytypes.MetadataStore, prometheus prometheus.Prometheus, orgGetter organization.Getter, querier querier.Querier, providerSettings factory.ProviderSettings, queryParser queryparser.QueryParser) (*baserules.Manager, error) {
ruleStore := sqlrulestore.NewRuleStore(sqlstore, queryParser, providerSettings)
maintenanceStore := sqlrulestore.NewMaintenanceStore(sqlstore)
// create manager opts
managerOpts := &baserules.ManagerOptions{
TelemetryStore: telemetryStore,
MetadataStore: metadataStore,
Prometheus: prometheus,
Context: context.Background(),
Reader: ch,
Querier: querier,
Logger: providerSettings.Logger,
Cache: cache,
EvalDelay: baseconst.GetEvalDelay(),
PrepareTaskFunc: rules.PrepareTaskFunc,
PrepareTestRuleFunc: rules.TestNotification,
Alertmanager: alertmanager,
OrgGetter: orgGetter,
RuleStore: ruleStore,
MaintenanceStore: maintenanceStore,
SqlStore: sqlstore,
QueryParser: queryParser,
}
// create Manager
manager, err := baserules.NewManager(managerOpts)
if err != nil {
return nil, fmt.Errorf("rule manager error: %v", err)
}
slog.Info("rules manager is ready")
return manager, nil
}

View File

@@ -5,33 +5,58 @@ import (
"encoding/json"
"fmt"
"log/slog"
"math"
"strings"
"sync"
"time"
"github.com/SigNoz/signoz/ee/query-service/anomaly"
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/types/rulestatehistorytypes"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/transition"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/valuer"
querierV2 "github.com/SigNoz/signoz/pkg/query-service/app/querier/v2"
"github.com/SigNoz/signoz/pkg/query-service/app/queryBuilder"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/utils/labels"
"github.com/SigNoz/signoz/pkg/query-service/utils/times"
"github.com/SigNoz/signoz/pkg/query-service/utils/timestamp"
"github.com/SigNoz/signoz/pkg/units"
baserules "github.com/SigNoz/signoz/pkg/query-service/rules"
"github.com/SigNoz/signoz/ee/anomaly"
querierV5 "github.com/SigNoz/signoz/pkg/querier"
anomalyV2 "github.com/SigNoz/signoz/ee/anomaly"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
)
const (
RuleTypeAnomaly = "anomaly_rule"
)
type AnomalyRule struct {
*baserules.BaseRule
mtx sync.Mutex
// querier is used for alerts migrated after the introduction of new query builder
querier querier.Querier
reader interfaces.Reader
provider anomaly.Provider
// querierV2 is used for alerts created after the introduction of new metrics query builder
querierV2 interfaces.Querier
// querierV5 is used for alerts migrated after the introduction of new query builder
querierV5 querierV5.Querier
provider anomaly.Provider
providerV2 anomalyV2.Provider
version string
logger *slog.Logger
@@ -45,70 +70,125 @@ func NewAnomalyRule(
id string,
orgID valuer.UUID,
p *ruletypes.PostableRule,
querier querier.Querier,
reader interfaces.Reader,
querierV5 querierV5.Querier,
logger *slog.Logger,
cache cache.Cache,
opts ...baserules.RuleOption,
) (*AnomalyRule, error) {
logger.Info("creating new AnomalyRule", slog.String("rule.id", id))
logger.Info("creating new AnomalyRule", "rule_id", id)
opts = append(opts, baserules.WithLogger(logger))
baseRule, err := baserules.NewBaseRule(id, orgID, p, opts...)
baseRule, err := baserules.NewBaseRule(id, orgID, p, reader, opts...)
if err != nil {
return nil, err
}
r := AnomalyRule{
t := AnomalyRule{
BaseRule: baseRule,
querier: querier,
version: p.Version,
logger: logger.With(slog.String("rule.id", id)),
}
switch p.RuleCondition.Seasonality {
case ruletypes.SeasonalityHourly:
r.seasonality = anomaly.SeasonalityHourly
case ruletypes.SeasonalityDaily:
r.seasonality = anomaly.SeasonalityDaily
case ruletypes.SeasonalityWeekly:
r.seasonality = anomaly.SeasonalityWeekly
switch strings.ToLower(p.RuleCondition.Seasonality) {
case "hourly":
t.seasonality = anomaly.SeasonalityHourly
case "daily":
t.seasonality = anomaly.SeasonalityDaily
case "weekly":
t.seasonality = anomaly.SeasonalityWeekly
default:
r.seasonality = anomaly.SeasonalityDaily
t.seasonality = anomaly.SeasonalityDaily
}
r.logger.Info("using seasonality", slog.String("rule.seasonality", r.seasonality.StringValue()))
logger.Info("using seasonality", "seasonality", t.seasonality.String())
if r.seasonality == anomaly.SeasonalityHourly {
r.provider = anomaly.NewHourlyProvider(
anomaly.WithQuerier[*anomaly.HourlyProvider](querier),
anomaly.WithLogger[*anomaly.HourlyProvider](r.logger),
querierOptsV2 := querierV2.QuerierOptions{
Reader: reader,
Cache: cache,
KeyGenerator: queryBuilder.NewKeyGenerator(),
}
t.querierV2 = querierV2.NewQuerier(querierOptsV2)
t.reader = reader
if t.seasonality == anomaly.SeasonalityHourly {
t.provider = anomaly.NewHourlyProvider(
anomaly.WithCache[*anomaly.HourlyProvider](cache),
anomaly.WithKeyGenerator[*anomaly.HourlyProvider](queryBuilder.NewKeyGenerator()),
anomaly.WithReader[*anomaly.HourlyProvider](reader),
)
} else if r.seasonality == anomaly.SeasonalityDaily {
r.provider = anomaly.NewDailyProvider(
anomaly.WithQuerier[*anomaly.DailyProvider](querier),
anomaly.WithLogger[*anomaly.DailyProvider](r.logger),
} else if t.seasonality == anomaly.SeasonalityDaily {
t.provider = anomaly.NewDailyProvider(
anomaly.WithCache[*anomaly.DailyProvider](cache),
anomaly.WithKeyGenerator[*anomaly.DailyProvider](queryBuilder.NewKeyGenerator()),
anomaly.WithReader[*anomaly.DailyProvider](reader),
)
} else if r.seasonality == anomaly.SeasonalityWeekly {
r.provider = anomaly.NewWeeklyProvider(
anomaly.WithQuerier[*anomaly.WeeklyProvider](querier),
anomaly.WithLogger[*anomaly.WeeklyProvider](r.logger),
} else if t.seasonality == anomaly.SeasonalityWeekly {
t.provider = anomaly.NewWeeklyProvider(
anomaly.WithCache[*anomaly.WeeklyProvider](cache),
anomaly.WithKeyGenerator[*anomaly.WeeklyProvider](queryBuilder.NewKeyGenerator()),
anomaly.WithReader[*anomaly.WeeklyProvider](reader),
)
}
return &r, nil
if t.seasonality == anomaly.SeasonalityHourly {
t.providerV2 = anomalyV2.NewHourlyProvider(
anomalyV2.WithQuerier[*anomalyV2.HourlyProvider](querierV5),
anomalyV2.WithLogger[*anomalyV2.HourlyProvider](logger),
)
} else if t.seasonality == anomaly.SeasonalityDaily {
t.providerV2 = anomalyV2.NewDailyProvider(
anomalyV2.WithQuerier[*anomalyV2.DailyProvider](querierV5),
anomalyV2.WithLogger[*anomalyV2.DailyProvider](logger),
)
} else if t.seasonality == anomaly.SeasonalityWeekly {
t.providerV2 = anomalyV2.NewWeeklyProvider(
anomalyV2.WithQuerier[*anomalyV2.WeeklyProvider](querierV5),
anomalyV2.WithLogger[*anomalyV2.WeeklyProvider](logger),
)
}
t.querierV5 = querierV5
t.version = p.Version
t.logger = logger
return &t, nil
}
func (r *AnomalyRule) Type() ruletypes.RuleType {
return ruletypes.RuleTypeAnomaly
return RuleTypeAnomaly
}
func (r *AnomalyRule) prepareQueryRange(ctx context.Context, ts time.Time) *qbtypes.QueryRangeRequest {
func (r *AnomalyRule) prepareQueryRange(ctx context.Context, ts time.Time) (*v3.QueryRangeParamsV3, error) {
r.logger.InfoContext(
ctx, "prepare query range request", slog.Int64("ts", ts.UnixMilli()),
slog.Int64("eval.window_ms", r.EvalWindow().Milliseconds()),
slog.Int64("eval.delay_ms", r.EvalDelay().Milliseconds()),
ctx, "prepare query range request v4", "ts", ts.UnixMilli(), "eval_window", r.EvalWindow().Milliseconds(), "eval_delay", r.EvalDelay().Milliseconds(),
)
st, en := r.Timestamps(ts)
start := st.UnixMilli()
end := en.UnixMilli()
compositeQuery := r.Condition().CompositeQuery
if compositeQuery.PanelType != v3.PanelTypeGraph {
compositeQuery.PanelType = v3.PanelTypeGraph
}
// default mode
return &v3.QueryRangeParamsV3{
Start: start,
End: end,
Step: int64(math.Max(float64(common.MinAllowedStepInterval(start, end)), 60)),
CompositeQuery: compositeQuery,
Variables: make(map[string]interface{}, 0),
NoCache: false,
}, nil
}
func (r *AnomalyRule) prepareQueryRangeV5(ctx context.Context, ts time.Time) (*qbtypes.QueryRangeRequest, error) {
r.logger.InfoContext(ctx, "prepare query range request v5", "ts", ts.UnixMilli(), "eval_window", r.EvalWindow().Milliseconds(), "eval_delay", r.EvalDelay().Milliseconds())
startTs, endTs := r.Timestamps(ts)
start, end := startTs.UnixMilli(), endTs.UnixMilli()
@@ -123,14 +203,25 @@ func (r *AnomalyRule) prepareQueryRange(ctx context.Context, ts time.Time) *qbty
}
req.CompositeQuery.Queries = make([]qbtypes.QueryEnvelope, len(r.Condition().CompositeQuery.Queries))
copy(req.CompositeQuery.Queries, r.Condition().CompositeQuery.Queries)
return req
return req, nil
}
func (r *AnomalyRule) GetSelectedQuery() string {
return r.Condition().GetSelectedQueryName()
}
func (r *AnomalyRule) buildAndRunQuery(ctx context.Context, orgID valuer.UUID, ts time.Time) (ruletypes.Vector, error) {
params := r.prepareQueryRange(ctx, ts)
params, err := r.prepareQueryRange(ctx, ts)
if err != nil {
return nil, err
}
err = r.PopulateTemporality(ctx, orgID, params)
if err != nil {
return nil, fmt.Errorf("internal error while setting temporality")
}
anomalies, err := r.provider.GetAnomalies(ctx, orgID, &anomaly.AnomaliesRequest{
anomalies, err := r.provider.GetAnomalies(ctx, orgID, &anomaly.GetAnomaliesRequest{
Params: params,
Seasonality: r.seasonality,
})
@@ -138,43 +229,87 @@ func (r *AnomalyRule) buildAndRunQuery(ctx context.Context, orgID valuer.UUID, t
return nil, err
}
var queryResult *qbtypes.TimeSeriesData
var queryResult *v3.Result
for _, result := range anomalies.Results {
if result.QueryName == r.SelectedQuery(ctx) {
if result.QueryName == r.GetSelectedQuery() {
queryResult = result
break
}
}
if queryResult == nil {
r.logger.WarnContext(ctx, "nil qb result", slog.Int64("ts", ts.UnixMilli()))
return ruletypes.Vector{}, nil
}
hasData := len(queryResult.Aggregations) > 0 &&
queryResult.Aggregations[0] != nil &&
len(queryResult.Aggregations[0].AnomalyScores) > 0
hasData := len(queryResult.AnomalyScores) > 0
if missingDataAlert := r.HandleMissingDataAlert(ctx, ts, hasData); missingDataAlert != nil {
return ruletypes.Vector{*missingDataAlert}, nil
} else if !hasData {
r.logger.WarnContext(ctx, "no anomaly result")
return ruletypes.Vector{}, nil
}
var resultVector ruletypes.Vector
scoresJSON, _ := json.Marshal(queryResult.Aggregations[0].AnomalyScores)
// TODO(srikanthccv): this could be noisy but we do this to answer false alert requests
r.logger.InfoContext(ctx, "anomaly scores", slog.String("anomaly.scores", string(scoresJSON)))
scoresJSON, _ := json.Marshal(queryResult.AnomalyScores)
r.logger.InfoContext(ctx, "anomaly scores", "scores", string(scoresJSON))
for _, series := range queryResult.AnomalyScores {
if !r.Condition().ShouldEval(series) {
r.logger.InfoContext(ctx, "not enough data points to evaluate series, skipping", "ruleid", r.ID(), "numPoints", len(series.Points), "requiredPoints", r.Condition().RequiredNumPoints)
continue
}
results, err := r.Threshold.Eval(*series, r.Unit(), ruletypes.EvalData{
ActiveAlerts: r.ActiveAlertsLabelFP(),
SendUnmatched: r.ShouldSendUnmatched(),
})
if err != nil {
return nil, err
}
resultVector = append(resultVector, results...)
}
return resultVector, nil
}
func (r *AnomalyRule) buildAndRunQueryV5(ctx context.Context, orgID valuer.UUID, ts time.Time) (ruletypes.Vector, error) {
params, err := r.prepareQueryRangeV5(ctx, ts)
if err != nil {
return nil, err
}
anomalies, err := r.providerV2.GetAnomalies(ctx, orgID, &anomalyV2.AnomaliesRequest{
Params: *params,
Seasonality: anomalyV2.Seasonality{String: valuer.NewString(r.seasonality.String())},
})
if err != nil {
return nil, err
}
var qbResult *qbtypes.TimeSeriesData
for _, result := range anomalies.Results {
if result.QueryName == r.GetSelectedQuery() {
qbResult = result
break
}
}
if qbResult == nil {
r.logger.WarnContext(ctx, "nil qb result", "ts", ts.UnixMilli())
}
queryResult := transition.ConvertV5TimeSeriesDataToV4Result(qbResult)
hasData := len(queryResult.AnomalyScores) > 0
if missingDataAlert := r.HandleMissingDataAlert(ctx, ts, hasData); missingDataAlert != nil {
return ruletypes.Vector{*missingDataAlert}, nil
}
var resultVector ruletypes.Vector
scoresJSON, _ := json.Marshal(queryResult.AnomalyScores)
r.logger.InfoContext(ctx, "anomaly scores", "scores", string(scoresJSON))
// Filter out new series if newGroupEvalDelay is configured
seriesToProcess := queryResult.Aggregations[0].AnomalyScores
seriesToProcess := queryResult.AnomalyScores
if r.ShouldSkipNewGroups() {
filteredSeries, filterErr := r.BaseRule.FilterNewSeries(ctx, ts, seriesToProcess)
// In case of error we log the error and continue with the original series
if filterErr != nil {
r.logger.ErrorContext(ctx, "error filtering new series", errors.Attr(filterErr))
r.logger.ErrorContext(ctx, "Error filtering new series, ", errors.Attr(filterErr), "rule_name", r.Name())
} else {
seriesToProcess = filteredSeries
}
@@ -182,14 +317,10 @@ func (r *AnomalyRule) buildAndRunQuery(ctx context.Context, orgID valuer.UUID, t
for _, series := range seriesToProcess {
if !r.Condition().ShouldEval(series) {
r.logger.InfoContext(
ctx, "not enough data points to evaluate series, skipping",
slog.Int("series.num_points", len(series.Values)),
slog.Int("series.required_points", r.Condition().RequiredNumPoints),
)
r.logger.InfoContext(ctx, "not enough data points to evaluate series, skipping", "ruleid", r.ID(), "numPoints", len(series.Points), "requiredPoints", r.Condition().RequiredNumPoints)
continue
}
results, err := r.Threshold.Eval(series, r.Unit(), ruletypes.EvalData{
results, err := r.Threshold.Eval(*series, r.Unit(), ruletypes.EvalData{
ActiveAlerts: r.ActiveAlertsLabelFP(),
SendUnmatched: r.ShouldSendUnmatched(),
})
@@ -210,9 +341,13 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
var res ruletypes.Vector
var err error
r.logger.InfoContext(ctx, "running query")
res, err = r.buildAndRunQuery(ctx, r.OrgID(), ts)
if r.version == "v5" {
r.logger.InfoContext(ctx, "running v5 query")
res, err = r.buildAndRunQueryV5(ctx, r.OrgID(), ts)
} else {
r.logger.InfoContext(ctx, "running v4 query")
res, err = r.buildAndRunQuery(ctx, r.OrgID(), ts)
}
if err != nil {
return 0, err
}
@@ -236,10 +371,7 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
}
value := valueFormatter.Format(smpl.V, r.Unit())
threshold := valueFormatter.Format(smpl.Target, smpl.TargetUnit)
r.logger.DebugContext(
ctx, "alert template data for rule", slog.String("formatter.name", valueFormatter.Name()),
slog.String("alert.value", value), slog.String("alert.threshold", threshold),
)
r.logger.DebugContext(ctx, "Alert template data for rule", "rule_name", r.Name(), "formatter", valueFormatter.Name(), "value", value, "threshold", threshold)
tmplData := ruletypes.AlertTemplateData(l, value, threshold)
// Inject some convenience variables that are easier to remember for users
@@ -254,34 +386,35 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
defs+text,
"__alert_"+r.Name(),
tmplData,
times.Time(timestamp.FromTime(ts)),
nil,
)
result, err := tmpl.Expand()
if err != nil {
result = fmt.Sprintf("<error expanding template: %s>", err)
r.logger.ErrorContext(ctx, "expanding alert template failed", errors.Attr(err), slog.Any("alert.template_data", tmplData))
r.logger.ErrorContext(ctx, "Expanding alert template failed", errors.Attr(err), "data", tmplData, "rule_name", r.Name())
}
return result
}
lb := ruletypes.NewBuilder(smpl.Metric...).Del(ruletypes.MetricNameLabel).Del(ruletypes.TemporalityLabel)
resultLabels := ruletypes.NewBuilder(smpl.Metric...).Del(ruletypes.MetricNameLabel).Del(ruletypes.TemporalityLabel).Labels()
lb := labels.NewBuilder(smpl.Metric).Del(labels.MetricNameLabel).Del(labels.TemporalityLabel)
resultLabels := labels.NewBuilder(smpl.Metric).Del(labels.MetricNameLabel).Del(labels.TemporalityLabel).Labels()
for name, value := range r.Labels().Map() {
lb.Set(name, expand(value))
}
lb.Set(ruletypes.AlertNameLabel, r.Name())
lb.Set(ruletypes.AlertRuleIDLabel, r.ID())
lb.Set(ruletypes.RuleSourceLabel, r.GeneratorURL())
lb.Set(labels.AlertNameLabel, r.Name())
lb.Set(labels.AlertRuleIdLabel, r.ID())
lb.Set(labels.RuleSourceLabel, r.GeneratorURL())
annotations := make(ruletypes.Labels, 0, len(r.Annotations().Map()))
annotations := make(labels.Labels, 0, len(r.Annotations().Map()))
for name, value := range r.Annotations().Map() {
annotations = append(annotations, ruletypes.Label{Name: name, Value: expand(value)})
annotations = append(annotations, labels.Label{Name: name, Value: expand(value)})
}
if smpl.IsMissing {
lb.Set(ruletypes.AlertNameLabel, "[No data] "+r.Name())
lb.Set(ruletypes.NoDataLabel, "true")
lb.Set(labels.AlertNameLabel, "[No data] "+r.Name())
lb.Set(labels.NoDataLabel, "true")
}
lbs := lb.Labels()
@@ -289,17 +422,17 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
resultFPs[h] = struct{}{}
if _, ok := alerts[h]; ok {
r.logger.ErrorContext(ctx, "the alert query returns duplicate records", slog.Any("alert", alerts[h]))
err = errors.NewInternalf(errors.CodeInternal, "duplicate alert found, vector contains metrics with the same labelset after applying alert labels")
r.logger.ErrorContext(ctx, "the alert query returns duplicate records", "rule_id", r.ID(), "alert", alerts[h])
err = fmt.Errorf("duplicate alert found, vector contains metrics with the same labelset after applying alert labels")
return 0, err
}
alerts[h] = &ruletypes.Alert{
Labels: lbs,
QueryResultLabels: resultLabels,
QueryResultLables: resultLabels,
Annotations: annotations,
ActiveAt: ts,
State: ruletypes.StatePending,
State: model.StatePending,
Value: smpl.V,
GeneratorURL: r.GeneratorURL(),
Receivers: ruleReceiverMap[lbs.Map()[ruletypes.LabelThresholdName]],
@@ -308,12 +441,12 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
}
}
r.logger.InfoContext(ctx, "number of alerts found", slog.Int("alert.count", len(alerts)))
r.logger.InfoContext(ctx, "number of alerts found", "rule_name", r.Name(), "alerts_count", len(alerts))
// alerts[h] is ready, add or update active list now
for h, a := range alerts {
// Check whether we already have alerting state for the identifying label set.
// Update the last value and annotations if so, create a new alert entry otherwise.
if alert, ok := r.Active[h]; ok && alert.State != ruletypes.StateInactive {
if alert, ok := r.Active[h]; ok && alert.State != model.StateInactive {
alert.Value = a.Value
alert.Annotations = a.Annotations
@@ -329,76 +462,76 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
r.Active[h] = a
}
itemsToAdd := []rulestatehistorytypes.RuleStateHistory{}
itemsToAdd := []model.RuleStateHistory{}
// Check if any pending alerts should be removed or fire now. Write out alert timeseries.
for fp, a := range r.Active {
labelsJSON, err := json.Marshal(a.QueryResultLabels)
labelsJSON, err := json.Marshal(a.QueryResultLables)
if err != nil {
r.logger.ErrorContext(ctx, "error marshaling labels", errors.Attr(err), slog.Any("alert.labels", a.Labels))
r.logger.ErrorContext(ctx, "error marshaling labels", errors.Attr(err), "labels", a.Labels)
}
if _, ok := resultFPs[fp]; !ok {
// If the alert was previously firing, keep it around for a given
// retention time so it is reported as resolved to the AlertManager.
if a.State == ruletypes.StatePending || (!a.ResolvedAt.IsZero() && ts.Sub(a.ResolvedAt) > ruletypes.ResolvedRetention) {
if a.State == model.StatePending || (!a.ResolvedAt.IsZero() && ts.Sub(a.ResolvedAt) > ruletypes.ResolvedRetention) {
delete(r.Active, fp)
}
if a.State != ruletypes.StateInactive {
a.State = ruletypes.StateInactive
if a.State != model.StateInactive {
a.State = model.StateInactive
a.ResolvedAt = ts
itemsToAdd = append(itemsToAdd, rulestatehistorytypes.RuleStateHistory{
itemsToAdd = append(itemsToAdd, model.RuleStateHistory{
RuleID: r.ID(),
RuleName: r.Name(),
State: ruletypes.StateInactive,
State: model.StateInactive,
StateChanged: true,
UnixMilli: ts.UnixMilli(),
Labels: rulestatehistorytypes.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLabels.Hash(),
Labels: model.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLables.Hash(),
Value: a.Value,
})
}
continue
}
if a.State == ruletypes.StatePending && ts.Sub(a.ActiveAt) >= r.HoldDuration().Duration() {
a.State = ruletypes.StateFiring
if a.State == model.StatePending && ts.Sub(a.ActiveAt) >= r.HoldDuration().Duration() {
a.State = model.StateFiring
a.FiredAt = ts
state := ruletypes.StateFiring
state := model.StateFiring
if a.Missing {
state = ruletypes.StateNoData
state = model.StateNoData
}
itemsToAdd = append(itemsToAdd, rulestatehistorytypes.RuleStateHistory{
itemsToAdd = append(itemsToAdd, model.RuleStateHistory{
RuleID: r.ID(),
RuleName: r.Name(),
State: state,
StateChanged: true,
UnixMilli: ts.UnixMilli(),
Labels: rulestatehistorytypes.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLabels.Hash(),
Labels: model.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLables.Hash(),
Value: a.Value,
})
}
// We need to change firing alert to recovering if the returned sample meets recovery threshold
changeFiringToRecovering := a.State == ruletypes.StateFiring && a.IsRecovering
changeFiringToRecovering := a.State == model.StateFiring && a.IsRecovering
// We need to change recovering alerts to firing if the returned sample meets target threshold
changeRecoveringToFiring := a.State == ruletypes.StateRecovering && !a.IsRecovering && !a.Missing
changeRecoveringToFiring := a.State == model.StateRecovering && !a.IsRecovering && !a.Missing
// in any of the above case we need to update the status of alert
if changeFiringToRecovering || changeRecoveringToFiring {
state := ruletypes.StateRecovering
state := model.StateRecovering
if changeRecoveringToFiring {
state = ruletypes.StateFiring
state = model.StateFiring
}
a.State = state
r.logger.DebugContext(ctx, "converting alert state", slog.Any("alert.state", state))
itemsToAdd = append(itemsToAdd, rulestatehistorytypes.RuleStateHistory{
r.logger.DebugContext(ctx, "converting alert state", "name", r.Name(), "state", state)
itemsToAdd = append(itemsToAdd, model.RuleStateHistory{
RuleID: r.ID(),
RuleName: r.Name(),
State: state,
StateChanged: true,
UnixMilli: ts.UnixMilli(),
Labels: rulestatehistorytypes.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLabels.Hash(),
Labels: model.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLables.Hash(),
Value: a.Value,
})
}
@@ -413,7 +546,7 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
itemsToAdd[idx] = item
}
_ = r.RecordRuleStateHistory(ctx, itemsToAdd)
r.RecordRuleStateHistory(ctx, prevState, currentState, itemsToAdd)
return len(r.Active), nil
}

View File

@@ -2,19 +2,21 @@ package rules
import (
"context"
"log/slog"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/SigNoz/signoz/ee/query-service/anomaly"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/query-service/app/clickhouseReader"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/SigNoz/signoz/ee/anomaly"
)
// mockAnomalyProvider is a mock implementation of anomaly.Provider for testing.
@@ -22,13 +24,13 @@ import (
// time periods (current, past period, current season, past season, past 2 seasons,
// past 3 seasons), making it cumbersome to create mock data.
type mockAnomalyProvider struct {
responses []*anomaly.AnomaliesResponse
responses []*anomaly.GetAnomaliesResponse
callCount int
}
func (m *mockAnomalyProvider) GetAnomalies(ctx context.Context, orgID valuer.UUID, req *anomaly.AnomaliesRequest) (*anomaly.AnomaliesResponse, error) {
func (m *mockAnomalyProvider) GetAnomalies(ctx context.Context, orgID valuer.UUID, req *anomaly.GetAnomaliesRequest) (*anomaly.GetAnomaliesResponse, error) {
if m.callCount >= len(m.responses) {
return &anomaly.AnomaliesResponse{Results: []*qbtypes.TimeSeriesData{}}, nil
return &anomaly.GetAnomaliesResponse{Results: []*v3.Result{}}, nil
}
resp := m.responses[m.callCount]
m.callCount++
@@ -47,46 +49,45 @@ func TestAnomalyRule_NoData_AlertOnAbsent(t *testing.T) {
postableRule := ruletypes.PostableRule{
AlertName: "Test anomaly no data",
AlertType: ruletypes.AlertTypeMetric,
RuleType: ruletypes.RuleTypeAnomaly,
RuleType: RuleTypeAnomaly,
Evaluation: &ruletypes.EvaluationEnvelope{Kind: ruletypes.RollingEvaluation, Spec: ruletypes.RollingWindow{
EvalWindow: evalWindow,
Frequency: valuer.MustParseTextDuration("1m"),
}},
RuleCondition: &ruletypes.RuleCondition{
CompareOperator: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
Target: &target,
CompositeQuery: &ruletypes.AlertCompositeQuery{
QueryType: ruletypes.QueryTypeBuilder,
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
CompareOp: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
Target: &target,
CompositeQuery: &v3.CompositeQuery{
QueryType: v3.QueryTypeBuilder,
BuilderQueries: map[string]*v3.BuilderQuery{
"A": {
QueryName: "A",
Expression: "A",
DataSource: v3.DataSourceMetrics,
Temporality: v3.Unspecified,
},
}},
},
},
SelectedQuery: "A",
Seasonality: ruletypes.SeasonalityDaily,
Seasonality: "daily",
Thresholds: &ruletypes.RuleThresholdData{
Kind: ruletypes.BasicThresholdKind,
Spec: ruletypes.BasicRuleThresholds{{
Name: "Test anomaly no data",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOperator: ruletypes.ValueIsAbove,
Name: "Test anomaly no data",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOp: ruletypes.ValueIsAbove,
}},
},
},
}
responseNoData := &anomaly.AnomaliesResponse{
Results: []*qbtypes.TimeSeriesData{
responseNoData := &anomaly.GetAnomaliesResponse{
Results: []*v3.Result{
{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
AnomalyScores: []*qbtypes.TimeSeries{},
}},
QueryName: "A",
AnomalyScores: []*v3.Series{},
},
},
}
@@ -114,17 +115,23 @@ func TestAnomalyRule_NoData_AlertOnAbsent(t *testing.T) {
t.Run(c.description, func(t *testing.T) {
postableRule.RuleCondition.AlertOnAbsent = c.alertOnAbsent
telemetryStore := telemetrystoretest.New(telemetrystore.Config{}, nil)
options := clickhouseReader.NewOptions("primaryNamespace")
reader := clickhouseReader.NewReader(slog.Default(), nil, telemetryStore, nil, "", time.Second, nil, nil, options)
rule, err := NewAnomalyRule(
"test-anomaly-rule",
valuer.GenerateUUID(),
&postableRule,
reader,
nil,
logger,
nil,
)
require.NoError(t, err)
rule.provider = &mockAnomalyProvider{
responses: []*anomaly.AnomaliesResponse{responseNoData},
responses: []*anomaly.GetAnomaliesResponse{responseNoData},
}
alertsFound, err := rule.Eval(context.Background(), evalTime)
@@ -149,47 +156,46 @@ func TestAnomalyRule_NoData_AbsentFor(t *testing.T) {
postableRule := ruletypes.PostableRule{
AlertName: "Test anomaly no data with AbsentFor",
AlertType: ruletypes.AlertTypeMetric,
RuleType: ruletypes.RuleTypeAnomaly,
RuleType: RuleTypeAnomaly,
Evaluation: &ruletypes.EvaluationEnvelope{Kind: ruletypes.RollingEvaluation, Spec: ruletypes.RollingWindow{
EvalWindow: evalWindow,
Frequency: valuer.MustParseTextDuration("1m"),
}},
RuleCondition: &ruletypes.RuleCondition{
CompareOperator: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
AlertOnAbsent: true,
Target: &target,
CompositeQuery: &ruletypes.AlertCompositeQuery{
QueryType: ruletypes.QueryTypeBuilder,
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
CompareOp: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
AlertOnAbsent: true,
Target: &target,
CompositeQuery: &v3.CompositeQuery{
QueryType: v3.QueryTypeBuilder,
BuilderQueries: map[string]*v3.BuilderQuery{
"A": {
QueryName: "A",
Expression: "A",
DataSource: v3.DataSourceMetrics,
Temporality: v3.Unspecified,
},
}},
},
},
SelectedQuery: "A",
Seasonality: ruletypes.SeasonalityDaily,
Seasonality: "daily",
Thresholds: &ruletypes.RuleThresholdData{
Kind: ruletypes.BasicThresholdKind,
Spec: ruletypes.BasicRuleThresholds{{
Name: "Test anomaly no data with AbsentFor",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOperator: ruletypes.ValueIsAbove,
Name: "Test anomaly no data with AbsentFor",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOp: ruletypes.ValueIsAbove,
}},
},
},
}
responseNoData := &anomaly.AnomaliesResponse{
Results: []*qbtypes.TimeSeriesData{
responseNoData := &anomaly.GetAnomaliesResponse{
Results: []*v3.Result{
{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
AnomalyScores: []*qbtypes.TimeSeries{},
}},
QueryName: "A",
AnomalyScores: []*v3.Series{},
},
},
}
@@ -223,35 +229,32 @@ func TestAnomalyRule_NoData_AbsentFor(t *testing.T) {
t1 := baseTime.Add(5 * time.Minute)
t2 := t1.Add(c.timeBetweenEvals)
responseWithData := &anomaly.AnomaliesResponse{
Results: []*qbtypes.TimeSeriesData{
responseWithData := &anomaly.GetAnomaliesResponse{
Results: []*v3.Result{
{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
AnomalyScores: []*qbtypes.TimeSeries{
{
Labels: []*qbtypes.Label{
{
Key: telemetrytypes.TelemetryFieldKey{Name: "Test"},
Value: "labels",
},
},
Values: []*qbtypes.TimeSeriesValue{
{Timestamp: baseTime.UnixMilli(), Value: 1.0},
{Timestamp: baseTime.Add(time.Minute).UnixMilli(), Value: 1.5},
},
AnomalyScores: []*v3.Series{
{
Labels: map[string]string{"test": "label"},
Points: []v3.Point{
{Timestamp: baseTime.UnixMilli(), Value: 1.0},
{Timestamp: baseTime.Add(time.Minute).UnixMilli(), Value: 1.5},
},
},
}},
},
},
},
}
rule, err := NewAnomalyRule("test-anomaly-rule", valuer.GenerateUUID(), &postableRule, nil, logger)
telemetryStore := telemetrystoretest.New(telemetrystore.Config{}, nil)
options := clickhouseReader.NewOptions("primaryNamespace")
reader := clickhouseReader.NewReader(slog.Default(), nil, telemetryStore, nil, "", time.Second, nil, nil, options)
rule, err := NewAnomalyRule("test-anomaly-rule", valuer.GenerateUUID(), &postableRule, reader, nil, logger, nil)
require.NoError(t, err)
rule.provider = &mockAnomalyProvider{
responses: []*anomaly.AnomaliesResponse{responseWithData, responseNoData},
responses: []*anomaly.GetAnomaliesResponse{responseWithData, responseNoData},
}
alertsFound1, err := rule.Eval(context.Background(), t1)

View File

@@ -11,7 +11,9 @@ import (
"github.com/google/uuid"
"github.com/SigNoz/signoz/pkg/errors"
basemodel "github.com/SigNoz/signoz/pkg/query-service/model"
baserules "github.com/SigNoz/signoz/pkg/query-service/rules"
"github.com/SigNoz/signoz/pkg/query-service/utils/labels"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -21,25 +23,24 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
rules := make([]baserules.Rule, 0)
var task baserules.Task
ruleID := baserules.RuleIDFromTaskName(opts.TaskName)
ruleId := baserules.RuleIdFromTaskName(opts.TaskName)
evaluation, err := opts.Rule.Evaluation.GetEvaluation()
if err != nil {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "evaluation is invalid: %v", err)
}
if opts.Rule.RuleType == ruletypes.RuleTypeThreshold {
// create a threshold rule
tr, err := baserules.NewThresholdRule(
ruleID,
ruleId,
opts.OrgID,
opts.Rule,
opts.Reader,
opts.Querier,
opts.Logger,
baserules.WithEvalDelay(opts.ManagerOpts.EvalDelay),
baserules.WithSQLStore(opts.SQLStore),
baserules.WithQueryParser(opts.ManagerOpts.QueryParser),
baserules.WithMetadataStore(opts.ManagerOpts.MetadataStore),
baserules.WithRuleStateHistoryModule(opts.ManagerOpts.RuleStateHistoryModule),
)
if err != nil {
@@ -55,15 +56,15 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
// create promql rule
pr, err := baserules.NewPromRule(
ruleID,
ruleId,
opts.OrgID,
opts.Rule,
opts.Logger,
opts.Reader,
opts.ManagerOpts.Prometheus,
baserules.WithSQLStore(opts.SQLStore),
baserules.WithQueryParser(opts.ManagerOpts.QueryParser),
baserules.WithMetadataStore(opts.ManagerOpts.MetadataStore),
baserules.WithRuleStateHistoryModule(opts.ManagerOpts.RuleStateHistoryModule),
)
if err != nil {
@@ -78,16 +79,17 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
} else if opts.Rule.RuleType == ruletypes.RuleTypeAnomaly {
// create anomaly rule
ar, err := NewAnomalyRule(
ruleID,
ruleId,
opts.OrgID,
opts.Rule,
opts.Reader,
opts.Querier,
opts.Logger,
opts.Cache,
baserules.WithEvalDelay(opts.ManagerOpts.EvalDelay),
baserules.WithSQLStore(opts.SQLStore),
baserules.WithQueryParser(opts.ManagerOpts.QueryParser),
baserules.WithMetadataStore(opts.ManagerOpts.MetadataStore),
baserules.WithRuleStateHistoryModule(opts.ManagerOpts.RuleStateHistoryModule),
)
if err != nil {
return task, err
@@ -99,7 +101,7 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
task = newTask(baserules.TaskTypeCh, opts.TaskName, evaluation.GetFrequency().Duration(), rules, opts.ManagerOpts, opts.NotifyFunc, opts.MaintenanceStore, opts.OrgID)
} else {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported rule type %s. Supported types: %s, %s", opts.Rule.RuleType, ruletypes.RuleTypeProm, ruletypes.RuleTypeThreshold)
return nil, fmt.Errorf("unsupported rule type %s. Supported types: %s, %s", opts.Rule.RuleType, ruletypes.RuleTypeProm, ruletypes.RuleTypeThreshold)
}
return task, nil
@@ -107,12 +109,12 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
// TestNotification prepares a dummy rule for given rule parameters and
// sends a test notification. returns alert count and error (if any)
func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
func TestNotification(opts baserules.PrepareTestRuleOptions) (int, *basemodel.ApiError) {
ctx := context.Background()
if opts.Rule == nil {
return 0, errors.NewInvalidInputf(errors.CodeInvalidInput, "rule is required")
return 0, basemodel.BadRequest(fmt.Errorf("rule is required"))
}
parsedRule := opts.Rule
@@ -132,14 +134,15 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
if parsedRule.RuleType == ruletypes.RuleTypeThreshold {
// add special labels for test alerts
parsedRule.Labels[ruletypes.RuleSourceLabel] = ""
parsedRule.Labels[ruletypes.AlertRuleIDLabel] = ""
parsedRule.Labels[labels.RuleSourceLabel] = ""
parsedRule.Labels[labels.AlertRuleIdLabel] = ""
// create a threshold rule
rule, err = baserules.NewThresholdRule(
alertname,
opts.OrgID,
parsedRule,
opts.Reader,
opts.Querier,
opts.Logger,
baserules.WithSendAlways(),
@@ -151,7 +154,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
if err != nil {
slog.Error("failed to prepare a new threshold rule for test", "name", alertname, errors.Attr(err))
return 0, err
return 0, basemodel.BadRequest(err)
}
} else if parsedRule.RuleType == ruletypes.RuleTypeProm {
@@ -162,6 +165,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
opts.OrgID,
parsedRule,
opts.Logger,
opts.Reader,
opts.ManagerOpts.Prometheus,
baserules.WithSendAlways(),
baserules.WithSendUnmatched(),
@@ -172,7 +176,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
if err != nil {
slog.Error("failed to prepare a new promql rule for test", "name", alertname, errors.Attr(err))
return 0, err
return 0, basemodel.BadRequest(err)
}
} else if parsedRule.RuleType == ruletypes.RuleTypeAnomaly {
// create anomaly rule
@@ -180,8 +184,10 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
alertname,
opts.OrgID,
parsedRule,
opts.Reader,
opts.Querier,
opts.Logger,
opts.Cache,
baserules.WithSendAlways(),
baserules.WithSendUnmatched(),
baserules.WithSQLStore(opts.SQLStore),
@@ -190,10 +196,10 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
)
if err != nil {
slog.Error("failed to prepare a new anomaly rule for test", "name", alertname, errors.Attr(err))
return 0, err
return 0, basemodel.BadRequest(err)
}
} else {
return 0, errors.NewInvalidInputf(errors.CodeInvalidInput, "failed to derive ruletype with given information")
return 0, basemodel.BadRequest(fmt.Errorf("failed to derive ruletype with given information"))
}
// set timestamp to current utc time
@@ -202,7 +208,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
alertsFound, err := rule.Eval(ctx, ts)
if err != nil {
slog.Error("evaluating rule failed", "rule", rule.Name(), errors.Attr(err))
return 0, err
return 0, basemodel.InternalError(fmt.Errorf("rule evaluation failed"))
}
rule.SendAlerts(ctx, ts, 0, time.Minute, opts.NotifyFunc)

View File

@@ -114,8 +114,11 @@ func TestManager_TestNotification_SendUnmatched_ThresholdRule(t *testing.T) {
},
})
count, err := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
require.Nil(t, err)
count, apiErr := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
if apiErr != nil {
t.Logf("TestNotification error: %v, type: %s", apiErr.Err, apiErr.Typ)
}
require.Nil(t, apiErr)
assert.Equal(t, tc.ExpectAlerts, count)
if tc.ExpectAlerts > 0 {
@@ -254,7 +257,7 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
WillReturnRows(samplesRows)
// Create Prometheus provider for this test
promProvider = prometheustest.New(context.Background(), instrumentationtest.New().ToProviderSettings(), prometheus.Config{Timeout: 2 * time.Minute}, store)
promProvider = prometheustest.New(context.Background(), instrumentationtest.New().ToProviderSettings(), prometheus.Config{}, store)
},
ManagerOptionsHook: func(opts *rules.ManagerOptions) {
// Set Prometheus provider for PromQL queries
@@ -265,8 +268,11 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
},
})
count, err := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
require.Nil(t, err)
count, apiErr := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
if apiErr != nil {
t.Logf("TestNotification error: %v, type: %s", apiErr.Err, apiErr.Typ)
}
require.Nil(t, apiErr)
assert.Equal(t, tc.ExpectAlerts, count)
if tc.ExpectAlerts > 0 {

View File

@@ -16,11 +16,9 @@ import (
"github.com/SigNoz/signoz/ee/query-service/model"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/query-service/utils/encryption"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/zeus"
)
@@ -45,18 +43,15 @@ type Manager struct {
zeus zeus.Zeus
orgGetter organization.Getter
flagger flagger.Flagger
}
func New(licenseService licensing.Licensing, clickhouseConn clickhouse.Conn, zeus zeus.Zeus, orgGetter organization.Getter, flagger flagger.Flagger) (*Manager, error) {
func New(licenseService licensing.Licensing, clickhouseConn clickhouse.Conn, zeus zeus.Zeus, orgGetter organization.Getter) (*Manager, error) {
m := &Manager{
clickhouseConn: clickhouseConn,
licenseService: licenseService,
scheduler: gocron.NewScheduler(time.UTC).Every(1).Day().At("00:00"), // send usage every at 00:00 UTC
zeus: zeus,
orgGetter: orgGetter,
flagger: flagger,
}
return m, nil
}
@@ -173,14 +168,7 @@ func (lm *Manager) UploadUsage(ctx context.Context) {
return
}
evalCtx := featuretypes.NewFlaggerEvaluationContext(organization.ID)
useZeus := lm.flagger.BooleanOrEmpty(ctx, flagger.FeaturePutMetersInZeus, evalCtx)
if useZeus {
errv2 = lm.zeus.PutMetersV2(ctx, payload.LicenseKey.String(), body)
} else {
errv2 = lm.zeus.PutMeters(ctx, payload.LicenseKey.String(), body)
}
errv2 = lm.zeus.PutMeters(ctx, payload.LicenseKey.String(), body)
if errv2 != nil {
slog.ErrorContext(ctx, "failed to upload usage", errors.Attr(errv2))
// not returning error here since it is captured in the failed count

View File

@@ -336,10 +336,9 @@ func (dialect *dialect) UpdatePrimaryKey(ctx context.Context, bun bun.IDB, oldMo
}
fkReference := ""
switch reference {
case Org:
if reference == Org {
fkReference = OrgReference
case User:
} else if reference == User {
fkReference = UserReference
}
@@ -393,10 +392,9 @@ func (dialect *dialect) AddPrimaryKey(ctx context.Context, bun bun.IDB, oldModel
}
fkReference := ""
switch reference {
case Org:
if reference == Org {
fkReference = OrgReference
case User:
} else if reference == User {
fkReference = UserReference
}

View File

@@ -14,21 +14,14 @@ import (
"github.com/uptrace/bun/dialect/pgdialect"
)
var _ Pooler = new(provider)
type provider struct {
settings factory.ScopedProviderSettings
sqldb *sql.DB
bundb *sqlstore.BunDB
pgxPool *pgxpool.Pool
dialect *dialect
formatter sqlstore.SQLFormatter
}
type Pooler interface {
Pool() *pgxpool.Pool
}
func NewFactory(hookFactories ...factory.ProviderFactory[sqlstore.SQLStoreHook, sqlstore.Config]) factory.ProviderFactory[sqlstore.SQLStore, sqlstore.Config] {
return factory.NewProviderFactory(factory.MustNewName("postgres"), func(ctx context.Context, providerSettings factory.ProviderSettings, config sqlstore.Config) (sqlstore.SQLStore, error) {
hooks := make([]sqlstore.SQLStoreHook, len(hookFactories))
@@ -54,7 +47,6 @@ func New(ctx context.Context, providerSettings factory.ProviderSettings, config
// Set the maximum number of open connections
pgConfig.MaxConns = int32(config.Connection.MaxOpenConns)
pgConfig.MaxConnLifetime = config.Connection.MaxConnLifetime
// Use pgxpool to create a connection pool
pool, err := pgxpool.NewWithConfig(ctx, pgConfig)
@@ -70,7 +62,6 @@ func New(ctx context.Context, providerSettings factory.ProviderSettings, config
settings: settings,
sqldb: sqldb,
bundb: bunDB,
pgxPool: pool,
dialect: new(dialect),
formatter: newFormatter(bunDB.Dialect()),
}, nil
@@ -84,10 +75,6 @@ func (provider *provider) SQLDB() *sql.DB {
return provider.sqldb
}
func (provider *provider) Pool() *pgxpool.Pool {
return provider.pgxPool
}
func (provider *provider) Dialect() sqlstore.SQLDialect {
return provider.dialect
}

View File

@@ -19,7 +19,7 @@ var (
once sync.Once
)
// initializes the Zeus configuration.
// initializes the Zeus configuration
func Config() zeus.Config {
once.Do(func() {
parsedURL, err := neturl.Parse(url)

View File

@@ -7,7 +7,6 @@ import (
"io"
"net/http"
"net/url"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
@@ -38,7 +37,6 @@ func New(ctx context.Context, providerSettings factory.ProviderSettings, config
providerSettings.MeterProvider,
client.WithRequestResponseLog(true),
client.WithRetryCount(3),
client.WithTimeout(30*time.Second),
)
if err != nil {
return nil, err
@@ -111,21 +109,6 @@ func (provider *Provider) GetDeployment(ctx context.Context, key string) ([]byte
return []byte(gjson.GetBytes(response, "data").String()), nil
}
func (provider *Provider) GetMeters(ctx context.Context, key string) ([]byte, error) {
response, err := provider.do(
ctx,
provider.config.URL.JoinPath("/v1/meters"),
http.MethodGet,
key,
nil,
)
if err != nil {
return nil, err
}
return []byte(gjson.GetBytes(response, "data").String()), nil
}
func (provider *Provider) PutMeters(ctx context.Context, key string, data []byte) error {
_, err := provider.do(
ctx,
@@ -138,18 +121,6 @@ func (provider *Provider) PutMeters(ctx context.Context, key string, data []byte
return err
}
func (provider *Provider) PutMetersV2(ctx context.Context, key string, data []byte) error {
_, err := provider.do(
ctx,
provider.config.URL.JoinPath("/v1/meters"),
http.MethodPost,
key,
data,
)
return err
}
func (provider *Provider) PutProfile(ctx context.Context, key string, profile *zeustypes.PostableProfile) error {
body, err := json.Marshal(profile)
if err != nil {
@@ -218,7 +189,7 @@ func (provider *Provider) do(ctx context.Context, url *url.URL, method string, k
return nil, provider.errFromStatusCode(response.StatusCode, errorMessage)
}
// This can be taken down to the client package.
// This can be taken down to the client package
func (provider *Provider) errFromStatusCode(statusCode int, errorMessage string) error {
switch statusCode {
case http.StatusBadRequest:

7
frontend/.eslintignore Normal file
View File

@@ -0,0 +1,7 @@
node_modules
build
*.typegen.ts
i18-generate-hash.js
src/parser/TraceOperatorParser/**
orval.config.ts

239
frontend/.eslintrc.cjs Normal file
View File

@@ -0,0 +1,239 @@
/**
* ESLint Configuration for SigNoz Frontend
*/
module.exports = {
ignorePatterns: [
'src/parser/*.ts',
'scripts/update-registry.js',
'scripts/generate-permissions-type.js',
],
env: {
browser: true,
es2021: true,
node: true,
},
extends: [
'eslint:recommended',
'plugin:react/recommended',
'plugin:@typescript-eslint/recommended',
'plugin:react-hooks/recommended',
'plugin:prettier/recommended',
'plugin:sonarjs/recommended',
'plugin:react/jsx-runtime',
],
parser: '@typescript-eslint/parser',
parserOptions: {
project: './tsconfig.json',
tsconfigRootDir: __dirname,
ecmaFeatures: {
jsx: true,
},
ecmaVersion: 2021,
sourceType: 'module',
},
plugins: [
'react', // React-specific rules
'@typescript-eslint', // TypeScript linting
'simple-import-sort', // Auto-sort imports
'react-hooks', // React Hooks rules
'prettier', // Code formatting
// 'jest', // TODO: Wait support on Biome to enable again
'jsx-a11y', // Accessibility rules
'import', // Import/export linting
'sonarjs', // Code quality/complexity
// TODO: Uncomment after running: yarn add -D eslint-plugin-spellcheck
// 'spellcheck', // Correct spellings
],
settings: {
react: {
version: 'detect',
},
'import/resolver': {
node: {
paths: ['src'],
extensions: ['.js', '.jsx', '.ts', '.tsx'],
},
},
},
rules: {
// Code quality rules
'prefer-const': 'error', // Enforces const for variables never reassigned
'no-var': 'error', // Disallows var, enforces let/const
'no-else-return': ['error', { allowElseIf: false }], // Reduces nesting by disallowing else after return
'no-cond-assign': 'error', // Prevents accidental assignment in conditions (if (x = 1) instead of if (x === 1))
'no-debugger': 'error', // Disallows debugger statements in production code
curly: 'error', // Requires curly braces for all control statements
eqeqeq: ['error', 'always', { null: 'ignore' }], // Enforces === and !== (allows == null for null/undefined check)
'no-console': ['error', { allow: ['warn', 'error'] }], // Warns on console.log, allows console.warn/error
// TODO: Change this to error in May 2026
'max-params': ['warn', 3], // a function can have max 3 params after which it should become an object
// TypeScript rules
'@typescript-eslint/explicit-function-return-type': 'error', // Requires explicit return types on functions
'@typescript-eslint/no-unused-vars': [
// Disallows unused variables/args
'error',
{
argsIgnorePattern: '^_', // Allows unused args prefixed with _ (e.g., _unusedParam)
varsIgnorePattern: '^_', // Allows unused vars prefixed with _ (e.g., _unusedVar)
},
],
'@typescript-eslint/no-explicit-any': 'warn', // Warns when using 'any' type (consider upgrading to error)
// TODO: Change to 'error' after fixing ~80 empty function placeholders in providers/contexts
'@typescript-eslint/no-empty-function': 'off', // Disallows empty function bodies
'@typescript-eslint/no-var-requires': 'error', // Disallows require() in TypeScript (use import instead)
'@typescript-eslint/ban-ts-comment': 'warn', // Allows @ts-ignore comments (sometimes needed for third-party libs)
'no-empty-function': 'off', // Disabled in favor of TypeScript version above
// React rules
'react/jsx-filename-extension': [
'error',
{
extensions: ['.tsx', '.jsx'], // Warns if JSX is used in non-.jsx/.tsx files
},
],
'react/prop-types': 'off', // Disabled - using TypeScript instead
'react/jsx-props-no-spreading': 'off', // Allows {...props} spreading (common in HOCs, forms, wrappers)
'react/no-array-index-key': 'error', // Prevents using array index as key (causes bugs when list changes)
// Accessibility rules
'jsx-a11y/label-has-associated-control': [
'error',
{
required: {
some: ['nesting', 'id'], // Labels must either wrap inputs or use htmlFor/id
},
},
],
// React Hooks rules
'react-hooks/rules-of-hooks': 'error', // Enforces Rules of Hooks (only call at top level)
'react-hooks/exhaustive-deps': 'warn', // Warns about missing dependencies in useEffect/useMemo/useCallback
// Import/export rules
'import/extensions': [
'error',
'ignorePackages',
{
js: 'never', // Disallows .js extension in imports
jsx: 'never', // Disallows .jsx extension in imports
ts: 'never', // Disallows .ts extension in imports
tsx: 'never', // Disallows .tsx extension in imports
},
],
'import/no-extraneous-dependencies': ['error', { devDependencies: true }], // Prevents importing packages not in package.json
'import/no-cycle': 'warn', // Warns about circular dependencies
// Import sorting rules
'simple-import-sort/imports': [
'error',
{
groups: [
['^react', '^@?\\w'], // React first, then external packages
['^@/'], // Absolute imports with @ alias
['^\\u0000'], // Side effect imports (import './file')
['^\\.'], // Relative imports
['^.+\\.s?css$'], // Style imports
],
},
],
'simple-import-sort/exports': 'error', // Auto-sorts exports
// Prettier - code formatting
'prettier/prettier': [
'error',
{},
{
usePrettierrc: true, // Uses .prettierrc.json for formatting rules
},
],
// SonarJS - code quality and complexity
'sonarjs/no-duplicate-string': 'off', // Disabled - can be noisy (enable periodically to check)
// State management governance
// Approved patterns: Zustand, nuqs (URL state), react-query (server state), useState/useRef/useReducer, localStorage/sessionStorage for simple cases
'no-restricted-imports': [
'error',
{
paths: [
{
name: 'redux',
message:
'[State mgmt] redux is deprecated. Migrate to Zustand, nuqs, or react-query.',
},
{
name: 'react-redux',
message:
'[State mgmt] react-redux is deprecated. Migrate to Zustand, nuqs, or react-query.',
},
{
name: 'xstate',
message:
'[State mgmt] xstate is deprecated. Migrate to Zustand or react-query.',
},
{
name: '@xstate/react',
message:
'[State mgmt] @xstate/react is deprecated. Migrate to Zustand or react-query.',
},
{
// Restrict React Context — useState/useRef/useReducer remain allowed
name: 'react',
importNames: ['createContext', 'useContext'],
message:
'[State mgmt] React Context is deprecated. Migrate shared state to Zustand.',
},
{
// immer used standalone as a store pattern is deprecated; Zustand bundles it internally
name: 'immer',
message:
'[State mgmt] Direct immer usage is deprecated. Use Zustand (which integrates immer via the immer middleware) instead.',
},
],
},
],
'no-restricted-syntax': [
'error',
{
selector:
// TODO: Make this generic on removal of redux
"CallExpression[callee.property.name='getState'][callee.object.name=/^use/]",
message:
'Avoid calling .getState() directly. Export a standalone action from the store instead.',
},
],
},
overrides: [
{
files: [
'**/*.test.{js,jsx,ts,tsx}',
'**/*.spec.{js,jsx,ts,tsx}',
'**/__tests__/**/*.{js,jsx,ts,tsx}',
],
rules: {
// Tests often have intentional duplication and complexity - disable SonarJS rules
'sonarjs/cognitive-complexity': 'off', // Tests can be complex
'sonarjs/no-identical-functions': 'off', // Similar test patterns are OK
'sonarjs/no-small-switch': 'off', // Small switches are OK in tests
},
},
{
files: ['src/api/generated/**/*.ts'],
rules: {
'@typescript-eslint/explicit-function-return-type': 'off',
'@typescript-eslint/explicit-module-boundary-types': 'off',
'no-nested-ternary': 'off',
'@typescript-eslint/no-unused-vars': 'warn',
},
},
{
// Store definition files are the only place .getState() is permitted —
// they are the canonical source for standalone action exports.
files: ['**/*Store.{ts,tsx}'],
rules: {
'no-restricted-syntax': 'off',
},
},
],
};

View File

@@ -1,28 +0,0 @@
{
"$schema": "./node_modules/oxfmt/configuration_schema.json",
"trailingComma": "all",
"useTabs": true,
"tabWidth": 1,
"singleQuote": true,
"jsxSingleQuote": false,
"semi": true,
"printWidth": 80,
"bracketSpacing": true,
"jsxBracketSameLine": false,
"arrowParens": "always",
"endOfLine": "lf",
"quoteProps": "as-needed",
"proseWrap": "preserve",
"htmlWhitespaceSensitivity": "css",
"embeddedLanguageFormatting": "auto",
"sortPackageJson": false,
"ignorePatterns": [
"build",
"coverage",
"public/",
"**/*.md",
"**/*.json",
"src/parser/**",
"src/TraceOperator/parser/**"
]
}

View File

@@ -1,615 +0,0 @@
{
"$schema": "./node_modules/oxlint/configuration_schema.json",
"jsPlugins": [
"./plugins/signoz.mjs",
"eslint-plugin-sonarjs"
],
"plugins": [
"eslint",
"react",
"react-perf",
"typescript",
"unicorn",
"jsx-a11y",
"import",
"jest",
"promise",
"jsdoc"
],
"categories": {
"correctness": "warn"
// TODO: Eventually turn this to error, and enable other categories
},
"env": {
"builtin": true,
"es2021": true,
"browser": true,
"jest": true,
"node": true
},
"options": {
"typeAware": true,
"typeCheck": false
},
"settings": {
"react": {
"version": "18.2.0"
}
},
"rules": {
"constructor-super": "error",
"for-direction": "error",
"getter-return": "error",
"no-async-promise-executor": "error",
"no-case-declarations": "error",
"no-class-assign": "error",
"no-compare-neg-zero": "error",
"no-cond-assign": "error",
// Prevents accidental assignment in conditions (if (x = 1) instead of if (x === 1))
"no-const-assign": "error",
"no-constant-binary-expression": "error",
"no-constant-condition": "error",
"no-control-regex": "error",
"no-debugger": "error",
// Disallows debugger statements in production code
"no-delete-var": "error",
"no-dupe-class-members": "error",
"no-dupe-else-if": "error",
"no-dupe-keys": "error",
"no-duplicate-case": "error",
"no-empty": "error",
"no-empty-character-class": "error",
"no-empty-pattern": "error",
"no-empty-static-block": "error",
"no-ex-assign": "error",
"no-extra-boolean-cast": "error",
"no-fallthrough": "error",
"no-func-assign": "error",
"no-global-assign": "error",
"no-import-assign": "error",
"no-invalid-regexp": "error",
"no-irregular-whitespace": "error",
"no-loss-of-precision": "error",
"no-misleading-character-class": "error",
"no-new-native-nonconstructor": "error",
"no-nonoctal-decimal-escape": "error",
"no-obj-calls": "error",
"no-prototype-builtins": "error",
"no-redeclare": "error",
"no-regex-spaces": "error",
"no-self-assign": "error",
"no-setter-return": "error",
"no-shadow-restricted-names": "warn",
// TODO: Change to error after migration to oxlint
"no-sparse-arrays": "error",
"no-this-before-super": "error",
"no-undef": "warn",
// TODO: Change to error after migration to oxlint
"no-unreachable": "warn",
// TODO: Change to error after the migration to oxlint
"no-unsafe-finally": "error",
"no-unsafe-negation": "error",
"no-unsafe-optional-chaining": "warn",
// TODO: Change to error after migration to oxlint
"no-unused-labels": "error",
"no-unused-private-class-members": "error",
"no-useless-backreference": "error",
"no-useless-catch": "error",
"no-useless-escape": "error",
"no-with": "error",
"require-yield": "error",
"use-isnan": "error",
"valid-typeof": "error",
"prefer-rest-params": "error",
"prefer-spread": "error",
"no-inner-declarations": "error",
// "no-octal": "error", // Not supported by oxlint
"react/display-name": "error",
"react/jsx-key": "warn",
// TODO: Change to error after migration to oxlint
"react/jsx-no-comment-textnodes": "error",
"react/jsx-no-duplicate-props": "error",
"react/jsx-no-target-blank": "warn",
// TODO: Change to error after migration to oxlint
"react/jsx-no-undef": "warn",
"react/no-children-prop": "error",
"react/no-danger-with-children": "error",
"react/no-direct-mutation-state": "error",
"react/no-find-dom-node": "error",
"react/no-is-mounted": "error",
"react/no-render-return-value": "error",
"react/no-string-refs": "error",
"react/no-unescaped-entities": "error",
"react/no-unknown-property": "error",
"react/require-render-return": "error",
"react/no-unsafe": "off",
"no-array-constructor": "error",
"@typescript-eslint/no-duplicate-enum-values": "warn",
// TODO: Change to error after migration to oxlint
"@typescript-eslint/no-empty-object-type": "error",
"@typescript-eslint/no-explicit-any": "warn",
// Warns when using 'any' type (consider upgrading to error)
"@typescript-eslint/no-empty-function": "off",
// TODO: Change to 'error' after fixing ~80 empty function placeholders in providers/contexts
"@typescript-eslint/ban-ts-comment": "warn",
// Warns when using @ts-ignore comments (sometimes needed for third-party libs)
"no-empty-function": "off",
// Disabled in favor of TypeScript version above
"@typescript-eslint/no-extra-non-null-assertion": "error",
"@typescript-eslint/no-misused-new": "error",
"@typescript-eslint/no-namespace": "error",
"@typescript-eslint/no-non-null-asserted-optional-chain": "error",
"@typescript-eslint/no-require-imports": "error",
"@typescript-eslint/no-this-alias": "error",
"@typescript-eslint/no-unnecessary-type-constraint": "warn",
// TODO: Change to error after migration to oxlint
"@typescript-eslint/no-unsafe-declaration-merging": "error",
"@typescript-eslint/no-unsafe-function-type": "error",
"no-unused-expressions": "warn",
// TODO: Change to error after migration to oxlint
"@typescript-eslint/no-wrapper-object-types": "error",
"@typescript-eslint/prefer-as-const": "error",
"@typescript-eslint/prefer-namespace-keyword": "error",
"@typescript-eslint/triple-slash-reference": "error",
"@typescript-eslint/adjacent-overload-signatures": "error",
"@typescript-eslint/ban-types": "error",
"@typescript-eslint/explicit-module-boundary-types": "warn",
"@typescript-eslint/no-empty-interface": "error",
"@typescript-eslint/no-inferrable-types": "error",
"@typescript-eslint/no-non-null-assertion": "warn",
"@typescript-eslint/no-unused-vars": [
// TypeScript-specific unused vars checking
"error",
{
"argsIgnorePattern": "^_",
"varsIgnorePattern": "^_",
"caughtErrors": "none"
}
],
"curly": "error",
// Requires curly braces for all control statements
// TODO: Change to error after migration to oxlint
"prefer-const": "warn",
// Enforces const for variables never reassigned
"no-var": "error",
// Disallows var, enforces let/const
"no-else-return": [
// Reduces nesting by disallowing else after return
"error",
{
"allowElseIf": false
}
],
"eqeqeq": [
// Enforces === and !== (allows == null for null/undefined check)
"error",
"always",
{
"null": "ignore"
}
],
"no-console": [
// Warns on console.log, allows console.warn/error
"error",
{
"allow": [
"warn",
"error"
]
}
],
"max-params": [
"warn",
3
],
// Warns when functions have more than 3 parameters
"@typescript-eslint/explicit-function-return-type": "error",
// Requires explicit return types on functions
"@typescript-eslint/no-var-requires": "error",
// Disallows require() in TypeScript (use import instead)
// Disabled - using TypeScript instead
"react/jsx-props-no-spreading": "off",
// Allows {...props} spreading (common in HOCs, forms, wrappers)
"react/jsx-filename-extension": [
// Warns if JSX is used in non-.jsx/.tsx files
"error",
{
"extensions": [
".tsx",
".jsx"
]
}
],
"react/no-array-index-key": "error",
// Prevents using array index as key (causes bugs when list changes)
"jsx-a11y/label-has-associated-control": [
// Accessibility rules - Labels must either wrap inputs or use htmlFor/id
"error",
{
"required": {
"some": [
"nesting",
"id"
]
}
}
],
"import/extensions": [
"error",
"ignorePackages",
{
// Import/export rules - Disallows .js/.jsx/.ts/.tsx extension in imports
"js": "never",
"jsx": "never",
"ts": "never",
"tsx": "never"
}
],
"import/no-cycle": "warn",
// Warns about circular dependencies
"import/first": "error",
"import/no-duplicates": "warn",
// TODO: Changed to warn during oxlint migration, should be changed to error
"arrow-body-style": "off",
"jest/no-disabled-tests": "warn",
// Jest test rules
"jest/no-focused-tests": "error",
"jest/no-identical-title": "warn",
"jest/prefer-to-have-length": "warn",
"jest/valid-expect": "warn",
// TODO: Change to error after migration to oxlint
// TODO: Change to error after migration to oxlint
"react-hooks/rules-of-hooks": "warn",
// React Hooks rules - Enforces Rules of Hooks (only call at top level)
"react-hooks/exhaustive-deps": "warn",
// Warns about missing dependencies in useEffect/useMemo/useCallback
// NOTE: The following react-hooks rules are not supported right know, follow the progress at https://github.com/oxc-project/oxc/issues/1022
// Most of them are for React Compiler, which we don't have enabled right know
// "react-hooks/config": "error",
// "react-hooks/error-boundaries": "error",
// "react-hooks/component-hook-factories": "error",
// "react-hooks/gating": "error",
// "react-hooks/globals": "error",
// "react-hooks/immutability": "error",
// "react-hooks/preserve-manual-memoization": "error",
// "react-hooks/purity": "error",
// "react-hooks/refs": "error",
// "react-hooks/set-state-in-effect": "error",
// "react-hooks/set-state-in-render": "error",
// "react-hooks/static-components": "error",
// "react-hooks/unsupported-syntax": "warn",
// "react-hooks/use-memo": "error",
// "react-hooks/incompatible-library": "warn",
"signoz/no-unsupported-asset-pattern": "error",
// Prevents the wrong usage of assets to break custom base path installations
"signoz/no-zustand-getstate-in-hooks": "error",
// Prevents useStore.getState() - export standalone actions instead
"signoz/no-navigator-clipboard": "error",
// Prevents navigator.clipboard - use useCopyToClipboard hook instead (disabled in tests via override)
"no-restricted-imports": [
"error",
{
"paths": [
{
"name": "redux",
"message": "[State mgmt] redux is deprecated. Migrate to Zustand, nuqs, or react-query."
},
{
"name": "react-redux",
"message": "[State mgmt] react-redux is deprecated. Migrate to Zustand, nuqs, or react-query."
},
{
"name": "xstate",
"message": "[State mgmt] xstate is deprecated. Migrate to Zustand or react-query."
},
{
"name": "@xstate/react",
"message": "[State mgmt] @xstate/react is deprecated. Migrate to Zustand or react-query."
},
{
"name": "react",
"importNames": [
"createContext",
"useContext"
],
"message": "[State mgmt] React Context is deprecated. Migrate shared state to Zustand."
},
{
"name": "immer",
"message": "[State mgmt] Direct immer usage is deprecated. Use Zustand (which integrates immer via the immer middleware) instead."
}
]
}
],
"react/no-array-index-key": "warn",
// TODO: Changed to warn during oxlint migration, should be changed to error,
"unicorn/error-message": "warn",
"unicorn/escape-case": "warn",
"unicorn/new-for-builtins": "warn",
"unicorn/no-abusive-eslint-disable": "warn",
"unicorn/no-console-spaces": "warn",
"unicorn/no-instanceof-array": "warn",
"unicorn/no-invalid-remove-event-listener": "warn",
"unicorn/no-new-array": "warn",
"unicorn/no-new-buffer": "warn",
"unicorn/no-thenable": "warn",
"unicorn/no-unreadable-array-destructuring": "warn",
"unicorn/no-useless-fallback-in-spread": "warn",
"unicorn/no-useless-length-check": "warn",
"unicorn/no-useless-promise-resolve-reject": "warn",
"unicorn/no-useless-spread": "warn",
"unicorn/no-zero-fractions": "warn",
"unicorn/number-literal-case": "warn",
"unicorn/prefer-array-find": "warn",
"unicorn/prefer-array-flat": "warn",
"unicorn/prefer-array-flat-map": "warn",
"unicorn/prefer-array-index-of": "warn",
"unicorn/prefer-array-some": "warn",
"unicorn/prefer-at": "warn",
"unicorn/prefer-code-point": "warn",
"unicorn/prefer-date-now": "warn",
"unicorn/prefer-default-parameters": "warn",
"unicorn/prefer-includes": "warn",
"unicorn/prefer-modern-math-apis": "warn",
"unicorn/prefer-native-coercion-functions": "warn",
"unicorn/prefer-node-protocol": "off",
"unicorn/prefer-number-properties": "warn",
"unicorn/prefer-optional-catch-binding": "warn",
"unicorn/prefer-regexp-test": "warn",
"unicorn/prefer-set-has": "warn",
"unicorn/prefer-string-replace-all": "warn",
"unicorn/prefer-string-slice": "warn",
"unicorn/prefer-string-starts-ends-with": "warn",
"unicorn/prefer-string-trim-start-end": "warn",
"unicorn/prefer-type-error": "warn",
"unicorn/require-array-join-separator": "warn",
"unicorn/require-number-to-fixed-digits-argument": "warn",
"unicorn/throw-new-error": "warn",
"unicorn/consistent-function-scoping": "warn",
"unicorn/explicit-length-check": "warn",
"unicorn/filename-case": [
"warn",
{
"case": "kebabCase"
}
],
"unicorn/no-array-for-each": "warn",
"unicorn/no-lonely-if": "warn",
"unicorn/no-negated-condition": "warn",
"unicorn/no-null": "warn",
"unicorn/no-object-as-default-parameter": "warn",
"unicorn/no-static-only-class": "warn",
"unicorn/no-this-assignment": "warn",
"unicorn/no-unreadable-iife": "warn",
"unicorn/no-useless-switch-case": "warn",
"unicorn/no-useless-undefined": "warn",
"unicorn/prefer-add-event-listener": "warn",
"unicorn/prefer-dom-node-append": "warn",
"unicorn/prefer-dom-node-dataset": "warn",
"unicorn/prefer-dom-node-remove": "warn",
"unicorn/prefer-dom-node-text-content": "warn",
"unicorn/prefer-keyboard-event-key": "warn",
"unicorn/prefer-math-trunc": "warn",
"unicorn/prefer-modern-dom-apis": "warn",
"unicorn/prefer-negative-index": "warn",
"unicorn/prefer-prototype-methods": "warn",
"unicorn/prefer-query-selector": "warn",
"unicorn/prefer-reflect-apply": "warn",
"unicorn/prefer-set-size": "warn",
"unicorn/prefer-spread": "warn",
"unicorn/prefer-ternary": "warn",
"unicorn/require-post-message-target-origin": "warn",
"oxc/bad-array-method-on-arguments": "error",
"oxc/bad-bitwise-operator": "error",
"oxc/bad-comparison-sequence": "error",
"oxc/bad-object-literal-comparison": "error",
"oxc/bad-replace-all-arg": "error",
"oxc/double-comparisons": "error",
"oxc/erasing-op": "error",
"oxc/misrefactored-assign-op": "error",
"oxc/missing-throw": "error",
"oxc/no-accumulating-spread": "error",
"oxc/no-async-endpoint-handlers": "error",
"oxc/no-const-enum": "error",
"oxc/number-arg-out-of-range": "error",
"oxc/only-used-in-recursion": "warn",
"oxc/uninvoked-array-callback": "error",
"jest/consistent-test-it": [
"warn",
{
"fn": "it"
}
],
"jest/expect-expect": "warn",
"jest/no-alias-methods": "warn",
"jest/no-commented-out-tests": "warn",
"jest/no-conditional-expect": "warn",
"jest/no-deprecated-functions": "warn",
"jest/no-done-callback": "warn",
"jest/no-duplicate-hooks": "warn",
"jest/no-export": "warn",
"jest/no-jasmine-globals": "warn",
"jest/no-mocks-import": "warn",
"jest/no-standalone-expect": "warn",
"jest/no-test-prefixes": "warn",
"jest/no-test-return-statement": "warn",
"jest/prefer-called-with": "off", // The auto-fix for this can break the tests when the function has args
"jest/prefer-comparison-matcher": "warn",
"jest/prefer-equality-matcher": "warn",
"jest/prefer-expect-resolves": "warn",
"jest/prefer-hooks-on-top": "warn",
"jest/prefer-spy-on": "warn",
"jest/prefer-strict-equal": "warn",
"jest/prefer-to-be": "warn",
"jest/prefer-to-contain": "warn",
"jest/prefer-todo": "warn",
"jest/valid-describe-callback": "warn",
"jest/valid-title": "warn",
"promise/catch-or-return": "warn",
"promise/no-return-wrap": "error",
"promise/param-names": "warn",
"promise/always-return": "warn",
"promise/no-nesting": "warn",
"promise/no-promise-in-callback": "warn",
"promise/no-callback-in-promise": "warn",
"promise/avoid-new": "off",
"promise/no-new-statics": "error",
"promise/no-return-in-finally": "error",
"promise/valid-params": "error",
"import/no-default-export": "off",
"import/no-named-as-default": "warn",
"import/no-named-as-default-member": "warn",
"import/no-self-import": "error",
"import/no-webpack-loader-syntax": "error",
"jsx-a11y/alt-text": "error",
"jsx-a11y/anchor-has-content": "warn",
"jsx-a11y/anchor-is-valid": "warn",
"jsx-a11y/aria-activedescendant-has-tabindex": "error",
"jsx-a11y/aria-props": "error",
"jsx-a11y/aria-role": "error",
"jsx-a11y/aria-unsupported-elements": "error",
"jsx-a11y/autocomplete-valid": "error",
"jsx-a11y/click-events-have-key-events": "warn",
"jsx-a11y/heading-has-content": "error",
"jsx-a11y/html-has-lang": "error",
"jsx-a11y/iframe-has-title": "error",
"jsx-a11y/img-redundant-alt": "warn",
"jsx-a11y/media-has-caption": "warn",
"jsx-a11y/mouse-events-have-key-events": "warn",
"jsx-a11y/no-access-key": "error",
"jsx-a11y/no-autofocus": "warn",
"jsx-a11y/no-distracting-elements": "error",
"jsx-a11y/no-redundant-roles": "warn",
"jsx-a11y/role-has-required-aria-props": "warn",
"jsx-a11y/role-supports-aria-props": "error",
"jsx-a11y/scope": "error",
"jsx-a11y/tabindex-no-positive": "warn",
// SonarJS rules - migrated from ESLint
"sonarjs/cognitive-complexity": "warn", // TODO: Change to error after migration
// Prevents overly complex functions (use SonarQube/SonarCloud for detailed analysis)
"sonarjs/max-switch-cases": "error",
// Limits switch statement cases
"sonarjs/no-all-duplicated-branches": "error",
// Prevents identical if/else branches
"sonarjs/no-collapsible-if": "error",
// Suggests merging nested ifs
"sonarjs/no-collection-size-mischeck": "error",
// Validates collection size checks
"sonarjs/no-duplicated-branches": "error",
// Detects duplicate conditional branches
"sonarjs/no-duplicate-string": "off",
// Warns on repeated string literals (was disabled)
"sonarjs/no-element-overwrite": "error",
// Prevents array element overwrites
"sonarjs/no-empty-collection": "error",
// Detects empty collections
"sonarjs/no-extra-arguments": "off",
// Detects extra function arguments (TypeScript handles this)
"sonarjs/no-gratuitous-expressions": "error",
// Removes unnecessary expressions
"sonarjs/no-identical-conditions": "error",
// Prevents duplicate conditions
"sonarjs/no-identical-expressions": "error",
// Detects duplicate expressions
"sonarjs/no-identical-functions": "error",
// Finds duplicated function implementations
"sonarjs/no-ignored-return": "error",
// Ensures return values are used
"sonarjs/no-inverted-boolean-check": "off",
// Simplifies boolean checks (was disabled)
"sonarjs/no-nested-switch": "error",
// Prevents nested switch statements
"sonarjs/no-nested-template-literals": "error",
// Avoids nested template literals
"sonarjs/no-redundant-boolean": "warn", // TODO: Change to error after migration
// Removes redundant boolean literals
"sonarjs/no-redundant-jump": "error",
// Removes unnecessary returns/continues
"sonarjs/no-same-line-conditional": "error",
// Prevents same-line conditionals
"sonarjs/no-small-switch": "error",
// Discourages tiny switch statements
"sonarjs/no-unused-collection": "error",
// Finds unused collections
"sonarjs/no-use-of-empty-return-value": "error",
// Prevents using void returns
"sonarjs/non-existent-operator": "error",
// Catches typos like =+ instead of +=
"sonarjs/prefer-immediate-return": "error",
// Returns directly instead of assigning
"sonarjs/prefer-object-literal": "error",
// Prefers object literals
"sonarjs/prefer-single-boolean-return": "error",
// Simplifies boolean returns
"sonarjs/prefer-while": "error",
// Suggests while loops over for loops
"sonarjs/elseif-without-else": "off"
// Requires final else in if-else-if chains (was disabled)
},
"ignorePatterns": [
"src/parser/*.ts",
"scripts/update-registry.cjs",
"scripts/generate-permissions-type.cjs",
"**/node_modules",
"**/build",
"**/*.typegen.ts",
"**/i18-generate-hash.cjs",
"src/parser/TraceOperatorParser/**/*",
"**/orval.config.ts"
],
"overrides": [
{
"files": [
"src/api/generated/**/*.ts"
],
"rules": {
"@typescript-eslint/explicit-function-return-type": "off",
"@typescript-eslint/explicit-module-boundary-types": "off",
"no-nested-ternary": "off",
"no-unused-vars": [
"warn",
{
"argsIgnorePattern": "^_",
"varsIgnorePattern": "^_",
"caughtErrors": "none"
}
]
}
},
{
// Test files: disable clipboard rule and import/first
"files": [
"**/*.test.ts",
"**/*.test.tsx",
"**/*.test.js",
"**/*.test.jsx",
"**/*.spec.ts",
"**/*.spec.tsx",
"**/*.spec.js",
"**/*.spec.jsx",
"**/__tests__/**/*.ts",
"**/__tests__/**/*.tsx",
"**/__tests__/**/*.js",
"**/__tests__/**/*.jsx"
],
"rules": {
"import/first": "off",
// Should ignore due to mocks
"signoz/no-navigator-clipboard": "off"
// Tests can use navigator.clipboard directly
}
},
{
// Store files are allowed to use .getState() as they export standalone actions
"files": [
"**/*Store.ts",
"**/*Store.tsx"
],
"rules": {
"signoz/no-zustand-getstate-in-hooks": "off"
}
}
]
}

View File

@@ -1,9 +0,0 @@
const path = require('path');
module.exports = {
plugins: [path.join(__dirname, 'stylelint-rules/no-unsupported-asset-url.js')],
customSyntax: 'postcss-scss',
rules: {
'local/no-unsupported-asset-url': true,
},
};

View File

@@ -1,7 +1,8 @@
{
"editor.formatOnSave": true,
"editor.defaultFormatter": "oxc.oxc-vscode",
"editor.codeActionsOnSave": {
"source.fixAll.oxc": "explicit"
}
"editor.formatOnSave": true,
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.codeActionsOnSave": {
"source.fixAll.eslint": "explicit"
},
"prettier.requireConfig": true
}

View File

@@ -12,7 +12,7 @@
or
`docker build . -t tagname`
**Tag to remote url- Introduce versioning later on**
**Tag to remote url- Introduce versinoing later on**
```
docker tag signoz/frontend:latest 7296823551/signoz:latest

View File

@@ -1 +0,0 @@
export default 'test-file-stub';

View File

@@ -2,7 +2,6 @@
interface SafeNavigateOptions {
replace?: boolean;
state?: unknown;
newTab?: boolean;
}
interface SafeNavigateTo {
@@ -21,7 +20,9 @@ interface UseSafeNavigateReturn {
export const useSafeNavigate = (): UseSafeNavigateReturn => ({
safeNavigate: jest.fn(
(_to: SafeNavigateToType, _options?: SafeNavigateOptions) => {},
(to: SafeNavigateToType, options?: SafeNavigateOptions) => {
console.log(`Mock safeNavigate called with:`, to, options);
},
) as jest.MockedFunction<
(to: SafeNavigateToType, options?: SafeNavigateOptions) => void
>,

View File

@@ -0,0 +1,29 @@
# SigNoz E2E Test Plan
This directory contains the structured test plan for the SigNoz application. Each subfolder corresponds to a main module or feature area, and contains scenario files for all user journeys, edge cases, and cross-module flows. These documents serve as the basis for generating Playwright MCP-driven E2E tests.
## Structure
- Each main module (e.g., logs, traces, dashboards, alerts, settings, etc.) has its own folder or markdown file.
- Each file contains detailed scenario templates, including preconditions, step-by-step actions, and expected outcomes.
- Use these documents to write, review, and update test cases as the application evolves.
## Folders & Files
- `logs/` — Logs module scenarios
- `traces/` — Traces module scenarios
- `metrics/` — Metrics module scenarios
- `dashboards/` — Dashboards module scenarios
- `alerts/` — Alerts module scenarios
- `services/` — Services module scenarios
- `settings/` — Settings and all sub-settings scenarios
- `onboarding/` — Onboarding and signup flows
- `navigation/` — Navigation, sidebar, and cross-module flows
- `exceptions/` — Exception and error handling scenarios
- `external-apis/` — External API monitoring scenarios
- `messaging-queues/` — Messaging queue scenarios
- `infrastructure/` — Infrastructure monitoring scenarios
- `help-support/` — Help & support scenarios
- `user-preferences/` — User preferences and personalization scenarios
- `service-map/` — Service map scenarios
- `saved-views/` — Saved views scenarios

View File

@@ -0,0 +1,16 @@
# Settings Module Test Plan
This folder contains E2E test scenarios for the Settings module and all sub-settings.
## Scenario Categories
- General settings (org/workspace, branding, version info)
- Billing settings
- Members & SSO
- Custom domain
- Integrations
- Notification channels
- API keys
- Ingestion
- Account settings (profile, password, preferences)
- Keyboard shortcuts

View File

@@ -0,0 +1,43 @@
# Account Settings E2E Scenarios (Updated)
## 1. Update Name
- **Precondition:** User is logged in
- **Steps:**
1. Click 'Update name' button
2. Edit name field in the modal/dialog
3. Save changes
- **Expected:** Name is updated in the UI
## 2. Update Email
- **Note:** The email field is not editable in the current UI.
## 3. Reset Password
- **Precondition:** User is logged in
- **Steps:**
1. Click 'Reset password' button
2. Complete reset flow (modal/dialog or external flow)
- **Expected:** Password is reset
## 4. Toggle 'Adapt to my timezone'
- **Precondition:** User is logged in
- **Steps:**
1. Toggle 'Adapt to my timezone' switch
- **Expected:** Timezone adapts accordingly (UI feedback/confirmation should be checked)
## 5. Toggle Theme (Dark/Light)
- **Precondition:** User is logged in
- **Steps:**
1. Toggle theme radio buttons ('Dark', 'Light Beta')
- **Expected:** Theme changes
## 6. Toggle Sidebar Always Open
- **Precondition:** User is logged in
- **Steps:**
1. Toggle 'Keep the primary sidebar always open' switch
- **Expected:** Sidebar remains open/closed as per toggle

View File

@@ -0,0 +1,26 @@
# API Keys E2E Scenarios (Updated)
## 1. Create a New API Key
- **Precondition:** User is admin
- **Steps:**
1. Click 'New Key' button
2. Enter details in the modal/dialog
3. Click 'Save'
- **Expected:** API key is created and listed in the table
## 2. Revoke an API Key
- **Precondition:** API key exists
- **Steps:**
1. In the table, locate the API key row
2. Click the revoke/delete button (icon button in the Action column)
3. Confirm if prompted
- **Expected:** API key is revoked/removed from the table
## 3. View API Key Usage
- **Precondition:** API key exists
- **Steps:**
1. View the 'Last used' and 'Expired' columns in the table
- **Expected:** Usage data is displayed for each API key

View File

@@ -0,0 +1,17 @@
# Billing Settings E2E Scenarios (Updated)
## 1. View Billing Information
- **Precondition:** User is admin
- **Steps:**
1. Navigate to Billing Settings
2. Wait for the billing chart/data to finish loading
- **Expected:**
- Billing heading and subheading are displayed
- Usage/cost table is visible with columns: Unit, Data Ingested, Price per Unit, Cost (Billing period to date)
- "Download CSV" and "Manage Billing" buttons are present and enabled after loading
- Test clicking "Download CSV" and "Manage Billing" for expected behavior (e.g., file download, navigation, or modal)
> Note: If these features are expected to trigger specific flows, document the observed behavior for each button.

View File

@@ -0,0 +1,18 @@
# Custom Domain E2E Scenarios (Updated)
## 1. Add or Update Custom Domain
- **Precondition:** User is admin
- **Steps:**
1. Click 'Customize teams URL' button
2. In the 'Customize your teams URL' dialog, enter the preferred subdomain
3. Click 'Apply Changes'
- **Expected:** Domain is set/updated for the team (UI feedback/confirmation should be checked)
## 2. Verify Domain Ownership
- **Note:** No explicit 'Verify' button or flow is present in the current UI. If verification is required, it may be handled automatically or via support.
## 3. Remove a Custom Domain
- **Note:** No explicit 'Remove' button or flow is present in the current UI. The only available action is to update the subdomain.

View File

@@ -0,0 +1,31 @@
# General Settings E2E Scenarios
## 1. View General Settings
- **Precondition:** User is logged in
- **Steps:**
1. Navigate to General Settings
- **Expected:** General settings are displayed
## 2. Update Organization/Workspace Name
- **Precondition:** User is admin
- **Steps:**
1. Edit organization/workspace name
2. Save changes
- **Expected:** Name is updated and visible
## 3. Update Logo or Branding
- **Precondition:** User is admin
- **Steps:**
1. Upload new logo/branding
2. Save changes
- **Expected:** Branding is updated
## 4. View Version/Build Info
- **Precondition:** User is logged in
- **Steps:**
1. View version/build info section
- **Expected:** Version/build info is displayed

View File

@@ -0,0 +1,20 @@
# Ingestion E2E Scenarios (Updated)
## 1. View Ingestion Sources
- **Precondition:** User is admin
- **Steps:**
1. Navigate to the Integrations page
- **Expected:** List of available data sources/integrations is displayed
## 2. Configure Ingestion Sources
- **Precondition:** User is admin
- **Steps:**
1. Click 'Configure' for a data source/integration
2. Complete the configuration flow (modal or page, as available)
- **Expected:** Source is configured (UI feedback/confirmation should be checked)
## 3. Disable/Enable Ingestion
- **Note:** No visible enable/disable toggle for ingestion sources in the current UI. Ingestion is managed via the Integrations configuration flows.

View File

@@ -0,0 +1,51 @@
# Integrations E2E Scenarios (Updated)
## 1. View List of Available Integrations
- **Precondition:** User is logged in
- **Steps:**
1. Navigate to Integrations
- **Expected:** List of integrations is displayed, each with a name, description, and 'Configure' button
## 2. Search Integrations by Name/Type
- **Precondition:** Integrations exist
- **Steps:**
1. Enter search/filter criteria in the 'Search for an integration...' box
- **Expected:** Only matching integrations are shown
## 3. Connect a New Integration
- **Precondition:** User is admin
- **Steps:**
1. Click 'Configure' for an integration
2. Complete the configuration flow (modal or page, as available)
- **Expected:** Integration is connected/configured (UI feedback/confirmation should be checked)
## 4. Disconnect an Integration
- **Note:** No visible 'Disconnect' button in the main list. This may be available in the configuration flow for a connected integration.
## 5. Configure Integration Settings
- **Note:** Configuration is handled in the flow after clicking 'Configure' for an integration.
## 6. Test Integration Connection
- **Note:** No visible 'Test Connection' button in the main list. This may be available in the configuration flow.
## 7. View Integration Status/Logs
- **Note:** No visible status/logs section in the main list. This may be available in the configuration flow.
## 8. Filter Integrations by Category
- **Note:** No explicit category filter in the current UI, only a search box.
## 9. View Integration Documentation/Help
- **Note:** No visible 'Help/Docs' button in the main list. This may be available in the configuration flow.
## 10. Update Integration Configuration
- **Note:** Configuration is handled in the flow after clicking 'Configure' for an integration.

View File

@@ -0,0 +1,19 @@
# Keyboard Shortcuts E2E Scenarios (Updated)
## 1. View Keyboard Shortcuts
- **Precondition:** User is logged in
- **Steps:**
1. Navigate to Keyboard Shortcuts
- **Expected:** Shortcuts are displayed in categorized tables (Global, Logs Explorer, Query Builder, Dashboard)
## 2. Customize Keyboard Shortcuts (if supported)
- **Note:** Customization is not available in the current UI. Shortcuts are view-only.
## 3. Use Keyboard Shortcuts for Navigation/Actions
- **Precondition:** User is logged in
- **Steps:**
1. Use shortcut for navigation/action (e.g., shift+s for Services, cmd+enter for running query)
- **Expected:** Navigation/action is performed as per shortcut

View File

@@ -0,0 +1,49 @@
# Members & SSO E2E Scenarios (Updated)
## 1. Invite a New Member
- **Precondition:** User is admin
- **Steps:**
1. Click 'Invite Members' button
2. In the 'Invite team members' dialog, enter email address, name (optional), and select role
3. (Optional) Click 'Add another team member' to invite more
4. Click 'Invite team members' to send invite(s)
- **Expected:** Pending invite appears in the 'Pending Invites' table
## 2. Remove a Member
- **Precondition:** User is admin, member exists
- **Steps:**
1. In the 'Members' table, locate the member row
2. Click 'Delete' in the Action column
3. Confirm removal if prompted
- **Expected:** Member is removed from the table
## 3. Update Member Roles
- **Precondition:** User is admin, member exists
- **Steps:**
1. In the 'Members' table, locate the member row
2. Click 'Edit' in the Action column
3. Change role in the edit dialog/modal
4. Save changes
- **Expected:** Member role is updated in the table
## 4. Configure SSO
- **Precondition:** User is admin
- **Steps:**
1. In the 'Authenticated Domains' section, locate the domain row
2. Click 'Configure SSO' or 'Edit Google Auth' as available
3. Complete SSO provider configuration in the modal/dialog
4. Save settings
- **Expected:** SSO is configured for the domain
## 5. Login via SSO
- **Precondition:** SSO is configured
- **Steps:**
1. Log out from the app
2. On the login page, click 'Login with SSO'
3. Complete SSO login flow
- **Expected:** User is logged in via SSO

View File

@@ -0,0 +1,39 @@
# Notification Channels E2E Scenarios (Updated)
## 1. Add a New Notification Channel
- **Precondition:** User is admin
- **Steps:**
1. Click 'New Alert Channel' button
2. In the 'New Notification Channel' form, fill in required fields (Name, Type, Webhook URL, etc.)
3. (Optional) Toggle 'Send resolved alerts'
4. (Optional) Click 'Test' to send a test notification
5. Click 'Save' to add the channel
- **Expected:** Channel is added and listed in the table
## 2. Test Notification Channel
- **Precondition:** Channel is being created or edited
- **Steps:**
1. In the 'New Notification Channel' or 'Edit Notification Channel' form, click 'Test'
- **Expected:** Test notification is sent (UI feedback/confirmation should be checked)
## 3. Remove a Notification Channel
- **Precondition:** Channel is added
- **Steps:**
1. In the table, locate the channel row
2. Click 'Delete' in the Action column
3. Confirm removal if prompted
- **Expected:** Channel is removed from the table
## 4. Update Notification Channel Settings
- **Precondition:** Channel is added
- **Steps:**
1. In the table, locate the channel row
2. Click 'Edit' in the Action column
3. In the 'Edit Notification Channel' form, update fields as needed
4. (Optional) Click 'Test' to send a test notification
5. Click 'Save' to update the channel
- **Expected:** Settings are updated

View File

@@ -0,0 +1,199 @@
# SigNoz Test Plan Validation Report
This report documents the validation of the E2E test plan against the current live application using Playwright MCP. Each module is reviewed for coverage, gaps, and required updates.
---
## Home Module
- **Coverage:**
- Widgets for logs, traces, metrics, dashboards, alerts, services, saved views, onboarding checklist
- Quick access buttons: Explore Logs, Create dashboard, Create an alert
- **Gaps/Updates:**
- Add scenarios for checklist interactions (e.g., “Ill do this later”, progress tracking)
- Add scenarios for Saved Views and cross-module links
- Add scenario for onboarding checklist completion
---
## Logs Module
- **Coverage:**
- Explorer, Pipelines, Views tabs
- Filtering by service, environment, severity, host, k8s, etc.
- Search, save view, create alert, add to dashboard, export, view mode switching
- **Gaps/Updates:**
- Add scenario for quick filter customization
- Add scenario for “Old Explorer” button
- Add scenario for frequency chart toggle
- Add scenario for “Stage & Run Query” workflow
---
## Traces Module
- **Coverage:**
- Tabs: Explorer, Funnels, Views
- Filtering by name, error status, duration, environment, function, service, RPC, status code, HTTP, trace ID, etc.
- Search, save view, create alert, add to dashboard, export, view mode switching (List, Traces, Time Series, Table)
- Pagination, quick filter customization, group by, aggregation
- **Gaps/Updates:**
- Add scenario for quick filter customization
- Add scenario for “Stage & Run Query” workflow
- Add scenario for all view modes (List, Traces, Time Series, Table)
- Add scenario for group by/aggregation
- Add scenario for trace detail navigation (clicking on trace row)
- Add scenario for Funnels tab (create/edit/delete funnel)
- Add scenario for Views tab (manage saved views)
---
## Metrics Module
- **Coverage:**
- Tabs: Summary, Explorer, Views
- Filtering by metric, type, unit, etc.
- Search, save view, add to dashboard, export, view mode switching (chart, table, proportion view)
- Pagination, group by, aggregation, custom queries
- **Gaps/Updates:**
- Add scenario for Proportion View in Summary
- Add scenario for all view modes (chart, table, proportion)
- Add scenario for group by/aggregation
- Add scenario for custom queries in Explorer
- Add scenario for Views tab (manage saved views)
---
## Dashboards Module
- **Coverage:**
- List, search, and filter dashboards
- Create new dashboard (button and template link)
- Edit, delete, and view dashboard details
- Add/edit/delete widgets (implied by dashboard detail)
- Pagination through dashboards
- **Gaps/Updates:**
- Add scenario for browsing dashboard templates (external link)
- Add scenario for requesting new template
- Add scenario for dashboard owner and creation info
- Add scenario for dashboard tags and filtering by tags
- Add scenario for dashboard sharing (if available)
- Add scenario for dashboard image/preview
---
## Messaging Queues Module
- **Coverage:**
- Overview tab: queue metrics, filters (Service Name, Span Name, Msg System, Destination, Kind)
- Search across all columns
- Pagination of queue data
- Sync and Share buttons
- Tabs for Kafka and Celery
- **Gaps/Updates:**
- Add scenario for Kafka tab (detailed metrics, actions)
- Add scenario for Celery tab (detailed metrics, actions)
- Add scenario for filter combinations and edge cases
- Add scenario for sharing queue data
- Add scenario for time range selection
---
## External APIs Module
- **Coverage:**
- Accessed via side navigation under MORE
- Explorer tab: domain, endpoints, last used, rate, error %, avg. latency
- Filters: Deployment Environment, Service Name, Rpc Method, Show IP addresses
- Table pagination
- Share and Stage & Run Query buttons
- **Gaps/Updates:**
- Add scenario for customizing quick filters
- Add scenario for running and staging queries
- Add scenario for sharing API data
- Add scenario for edge cases in filters and table data
---
## Alerts Module
- **Coverage:**
- Alert Rules tab: list, search, create (New Alert), edit, delete, enable/disable, severity, labels, actions
- Triggered Alerts tab (visible in tablist)
- Configuration tab (visible in tablist)
- Table pagination
- **Gaps/Updates:**
- Add scenario for triggered alerts (view, acknowledge, resolve)
- Add scenario for alert configuration (settings, integrations)
- Add scenario for edge cases in alert creation and management
- Add scenario for searching and filtering alerts
---
## Integrations Module
- **Coverage:**
- Integrations tab: list, search, configure (e.g., AWS), request new integration
- One-click setup for AWS monitoring
- Request more integrations (form)
- **Gaps/Updates:**
- Add scenario for configuring integrations (step-by-step)
- Add scenario for searching and filtering integrations
- Add scenario for requesting new integrations
- Add scenario for edge cases (e.g., failed configuration)
---
## Exceptions Module
- **Coverage:**
- All Exceptions: list, search, filter (Deployment Environment, Service Name, Host Name, K8s Cluster/Deployment/Namespace, Net Peer Name)
- Table: Exception Type, Error Message, Count, Last Seen, First Seen, Application
- Pagination
- Exception detail links
- Share and Stage & Run Query buttons
- **Gaps/Updates:**
- Add scenario for exception detail view
- Add scenario for advanced filtering and edge cases
- Add scenario for sharing and running queries
- Add scenario for error grouping and navigation
---
## Service Map Module
- **Coverage:**
- Service Map visualization (main graph)
- Filters: environment, resource attributes
- Time range selection
- Sync and Share buttons
- **Gaps/Updates:**
- Add scenario for interacting with the map (zoom, pan, select service)
- Add scenario for filtering and edge cases
- Add scenario for sharing the map
- Add scenario for time range and environment combinations
---
## Billing Module
- **Coverage:**
- Billing overview: cost monitoring, invoices, CSV download (disabled), manage billing (disabled)
- Teams Cloud section
- Billing table: Unit, Data Ingested, Price per Unit, Cost (Billing period to date)
- **Gaps/Updates:**
- Add scenario for invoice download and management (when enabled)
- Add scenario for cost monitoring and edge cases
- Add scenario for billing table data validation
- Add scenario for permissions and access control
---
## Usage Explorer Module
- **Status:**
- Not accessible in the current environment. Removing from test plan flows.
---
## [Next modules will be filled as validation proceeds]

View File

@@ -0,0 +1,42 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('Account Settings - View and Assert Static Controls', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Assert General section and controls (confirmed by DOM)
await expect(
page.getByLabel('My Settings').getByText('General'),
).toBeVisible();
await expect(page.getByText('Manage your account settings.')).toBeVisible();
await expect(page.getByRole('button', { name: 'Update name' })).toBeVisible();
await expect(
page.getByRole('button', { name: 'Reset password' }),
).toBeVisible();
// Assert User Preferences section and controls (confirmed by DOM)
await expect(page.getByText('User Preferences')).toBeVisible();
await expect(
page.getByText('Tailor the SigNoz console to work according to your needs.'),
).toBeVisible();
await expect(page.getByText('Select your theme')).toBeVisible();
const themeSelector = page.getByTestId('theme-selector');
await expect(themeSelector.getByText('Dark')).toBeVisible();
await expect(themeSelector.getByText('Light')).toBeVisible();
await expect(themeSelector.getByText('System')).toBeVisible();
await expect(page.getByTestId('timezone-adaptation-switch')).toBeVisible();
await expect(page.getByTestId('side-nav-pinned-switch')).toBeVisible();
});

View File

@@ -0,0 +1,42 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('API Keys Settings - View and Interact', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click API Keys tab in the settings sidebar (by data-testid)
await page.getByTestId('api-keys').click();
// Assert heading and subheading
await expect(page.getByRole('heading', { name: 'API Keys' })).toBeVisible();
await expect(
page.getByText('Create and manage API keys for the SigNoz API'),
).toBeVisible();
// Assert presence of New Key button
const newKeyBtn = page.getByRole('button', { name: 'New Key' });
await expect(newKeyBtn).toBeVisible();
// Assert table columns
await expect(page.getByText('Last used').first()).toBeVisible();
await expect(page.getByText('Expired').first()).toBeVisible();
// Assert at least one API key row with action buttons
// Select the first action cell's first button (icon button)
const firstActionCell = page.locator('table tr').nth(1).locator('td').last();
const deleteBtn = firstActionCell.locator('button').first();
await expect(deleteBtn).toBeVisible();
});

View File

@@ -0,0 +1,71 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
// E2E: Billing Settings - View Billing Information and Button Actions
test('View Billing Information and Button Actions', async ({
page,
context,
}) => {
// Ensure user is logged in
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click Billing tab in the settings sidebar (by data-testid)
await page.getByTestId('billing').click();
// Wait for billing chart/data to finish loading
await page.getByText('loading').first().waitFor({ state: 'hidden' });
// Assert visibility of subheading (unique)
await expect(
page.getByText(
'Manage your billing information, invoices, and monitor costs.',
),
).toBeVisible();
// Assert visibility of Teams Cloud heading
await expect(page.getByRole('heading', { name: 'Teams Cloud' })).toBeVisible();
// Assert presence of summary and detailed tables
await expect(page.getByText('TOTAL SPENT')).toBeVisible();
await expect(page.getByText('Data Ingested')).toBeVisible();
await expect(page.getByText('Price per Unit')).toBeVisible();
await expect(page.getByText('Cost (Billing period to date)')).toBeVisible();
// Assert presence of alert and note
await expect(
page.getByText('Your current billing period is from', { exact: false }),
).toBeVisible();
await expect(
page.getByText('Billing metrics are updated once every 24 hours.'),
).toBeVisible();
// Test Download CSV button
const [download] = await Promise.all([
page.waitForEvent('download'),
page.getByRole('button', { name: 'cloud-download Download CSV' }).click(),
]);
// Optionally, check download file name
expect(download.suggestedFilename()).toContain('billing_usage');
// Test Manage Billing button (opens Stripe in new tab)
const [newPage] = await Promise.all([
context.waitForEvent('page'),
page.getByTestId('header-billing-button').click(),
]);
await newPage.waitForLoadState();
expect(newPage.url()).toContain('stripe.com');
await newPage.close();
});

View File

@@ -0,0 +1,52 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('Custom Domain Settings - View and Interact', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click Custom Domain tab in the settings sidebar (by data-testid)
await page.getByTestId('custom-domain').click();
// Wait for custom domain chart/data to finish loading
await page.getByText('loading').first().waitFor({ state: 'hidden' });
// Assert heading and subheading
await expect(
page.getByRole('heading', { name: 'Custom Domain Settings' }),
).toBeVisible();
await expect(
page.getByText('Personalize your workspace domain effortlessly.'),
).toBeVisible();
// Assert presence of Customize teams URL button
const customizeBtn = page.getByRole('button', {
name: 'Customize teams URL',
});
await expect(customizeBtn).toBeVisible();
await customizeBtn.click();
// Assert modal/dialog fields and buttons
await expect(
page.getByRole('dialog', { name: 'Customize your teams URL' }),
).toBeVisible();
await expect(page.getByLabel('Teams URL subdomain')).toBeVisible();
await expect(
page.getByRole('button', { name: 'Apply Changes' }),
).toBeVisible();
await expect(page.getByRole('button', { name: 'Close' })).toBeVisible();
// Close the modal
await page.getByRole('button', { name: 'Close' }).click();
});

View File

@@ -0,0 +1,32 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('View General Settings', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click General tab in the settings sidebar (by data-testid)
await page.getByTestId('general').click();
// Wait for General tab to be visible
await page.getByRole('tabpanel', { name: 'General' }).waitFor();
// Assert visibility of definitive/static elements
await expect(page.getByRole('heading', { name: 'Metrics' })).toBeVisible();
await expect(page.getByRole('heading', { name: 'Traces' })).toBeVisible();
await expect(page.getByRole('heading', { name: 'Logs' })).toBeVisible();
await expect(page.getByText('Please')).toBeVisible();
await expect(page.getByRole('link', { name: 'email us' })).toBeVisible();
});

View File

@@ -0,0 +1,48 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('Ingestion Settings - View and Interact', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click Ingestion tab in the settings sidebar (by data-testid)
await page.getByTestId('ingestion').click();
// Assert heading and subheading (Integrations page)
await expect(
page.getByRole('heading', { name: 'Integrations' }),
).toBeVisible();
await expect(
page.getByText('Manage Integrations for this workspace'),
).toBeVisible();
// Assert presence of search box
await expect(
page.getByPlaceholder('Search for an integration...'),
).toBeVisible();
// Assert at least one data source with Configure button
const configureBtn = page.getByRole('button', { name: 'Configure' }).first();
await expect(configureBtn).toBeVisible();
// Assert Request more integrations section
await expect(
page.getByText(
"Can't find what youre looking for? Request more integrations",
),
).toBeVisible();
await expect(page.getByPlaceholder('Enter integration name...')).toBeVisible();
await expect(page.getByRole('button', { name: 'Submit' })).toBeVisible();
});

View File

@@ -0,0 +1,48 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('Integrations Settings - View and Interact', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click Integrations tab in the settings sidebar (by data-testid)
await page.getByTestId('integrations').click();
// Assert heading and subheading
await expect(
page.getByRole('heading', { name: 'Integrations' }),
).toBeVisible();
await expect(
page.getByText('Manage Integrations for this workspace'),
).toBeVisible();
// Assert presence of search box
await expect(
page.getByPlaceholder('Search for an integration...'),
).toBeVisible();
// Assert at least one integration with Configure button
const configureBtn = page.getByRole('button', { name: 'Configure' }).first();
await expect(configureBtn).toBeVisible();
// Assert Request more integrations section
await expect(
page.getByText(
"Can't find what youre looking for? Request more integrations",
),
).toBeVisible();
await expect(page.getByPlaceholder('Enter integration name...')).toBeVisible();
await expect(page.getByRole('button', { name: 'Submit' })).toBeVisible();
});

View File

@@ -0,0 +1,56 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('Members & SSO Settings - View and Interact', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click Members & SSO tab in the settings sidebar (by data-testid)
await page.getByTestId('members-sso').click();
// Assert headings and tables
await expect(
page.getByRole('heading', { name: /Members \(\d+\)/ }),
).toBeVisible();
await expect(
page.getByRole('heading', { name: /Pending Invites \(\d+\)/ }),
).toBeVisible();
await expect(
page.getByRole('heading', { name: 'Authenticated Domains' }),
).toBeVisible();
// Assert Invite Members button is visible and clickable
const inviteBtn = page.getByRole('button', { name: /Invite Members/ });
await expect(inviteBtn).toBeVisible();
await inviteBtn.click();
// Assert Invite Members modal/dialog appears (modal title is unique)
await expect(page.getByText('Invite team members').first()).toBeVisible();
// Close the modal (use unique 'Close' button)
await page.getByRole('button', { name: 'Close' }).click();
// Assert Edit and Delete buttons are present for at least one member
const editBtn = page.getByRole('button', { name: /Edit/ }).first();
const deleteBtn = page.getByRole('button', { name: /Delete/ }).first();
await expect(editBtn).toBeVisible();
await expect(deleteBtn).toBeVisible();
// Assert Add Domains button is visible
await expect(page.getByRole('button', { name: /Add Domains/ })).toBeVisible();
// Assert Configure SSO or Edit Google Auth button is visible for at least one domain
const ssoBtn = page
.getByRole('button', { name: /Configure SSO|Edit Google Auth/ })
.first();
await expect(ssoBtn).toBeVisible();
});

View File

@@ -0,0 +1,57 @@
import { expect, test } from '@playwright/test';
import { ensureLoggedIn } from '../../../utils/login.util';
test('Notification Channels Settings - View and Interact', async ({ page }) => {
await ensureLoggedIn(page);
// 1. Open the sidebar settings menu using data-testid
await page.getByTestId('settings-nav-item').click();
// 2. Click Account Settings in the dropdown (by role/name or data-testid if available)
await page.getByRole('menuitem', { name: 'Account Settings' }).click();
// Assert the main tabpanel/heading (confirmed by DOM)
await expect(page.getByTestId('settings-page-title')).toBeVisible();
// Focus on the settings page sidenav
await page.getByTestId('settings-page-sidenav').focus();
// Click Notification Channels tab in the settings sidebar (by data-testid)
await page.getByTestId('notification-channels').click();
// Wait for loading to finish
await page.getByText('loading').first().waitFor({ state: 'hidden' });
// Assert presence of New Alert Channel button
const newChannelBtn = page.getByRole('button', { name: /New Alert Channel/ });
await expect(newChannelBtn).toBeVisible();
// Assert table columns
await expect(page.getByText('Name')).toBeVisible();
await expect(page.getByText('Type')).toBeVisible();
await expect(page.getByText('Action')).toBeVisible();
// Click New Alert Channel and assert modal fields/buttons
await newChannelBtn.click();
await expect(
page.getByRole('heading', { name: 'New Notification Channel' }),
).toBeVisible();
await expect(page.getByLabel('Name')).toBeVisible();
await expect(page.getByLabel('Type')).toBeVisible();
await expect(page.getByLabel('Webhook URL')).toBeVisible();
await expect(
page.getByRole('switch', { name: 'Send resolved alerts' }),
).toBeVisible();
await expect(page.getByRole('button', { name: 'Save' })).toBeVisible();
await expect(page.getByRole('button', { name: 'Test' })).toBeVisible();
await expect(page.getByRole('button', { name: 'Back' })).toBeVisible();
// Close modal
await page.getByRole('button', { name: 'Back' }).click();
// Assert Edit and Delete buttons for at least one channel
const editBtn = page.getByRole('button', { name: 'Edit' }).first();
const deleteBtn = page.getByRole('button', { name: 'Delete' }).first();
await expect(editBtn).toBeVisible();
await expect(deleteBtn).toBeVisible();
});

View File

@@ -0,0 +1,35 @@
import { Page } from '@playwright/test';
// Read credentials from environment variables
const username = process.env.LOGIN_USERNAME;
const password = process.env.LOGIN_PASSWORD;
const baseURL = process.env.BASE_URL;
/**
* Ensures the user is logged in. If not, performs the login steps.
* Follows the MCP process step-by-step.
*/
export async function ensureLoggedIn(page: Page): Promise<void> {
// if already in home page, return
if (await page.url().includes('/home')) {
return;
}
if (!username || !password) {
throw new Error(
'E2E_EMAIL and E2E_PASSWORD environment variables must be set.',
);
}
await page.goto(`${baseURL}/login`);
await page.getByTestId('email').click();
await page.getByTestId('email').fill(username);
await page.getByTestId('initiate_login').click();
await page.getByTestId('password').click();
await page.getByTestId('password').fill(password);
await page.getByRole('button', { name: 'Login' }).click();
await page
.getByText('Hello there, Welcome to your')
.waitFor({ state: 'visible' });
}

View File

@@ -62,29 +62,6 @@
<link data-react-helmet="true" rel="shortcut icon" href="/favicon.ico" />
</head>
<body data-theme="default">
<script>
// Apply theme class synchronously before React renders to prevent flash.
// Mirrors the logic in ThemeProvider (hooks/useDarkMode/index.tsx).
(function () {
try {
var theme = localStorage.getItem('THEME');
var autoSwitch = localStorage.getItem('THEME_AUTO_SWITCH') === 'true';
if (autoSwitch) {
theme = window.matchMedia('(prefers-color-scheme: dark)').matches
? 'dark'
: 'light';
}
if (theme === 'light') {
document.body.classList.add('lightMode');
} else {
// Default to dark when no preference is stored
document.body.classList.add('dark', 'darkMode');
}
} catch (e) {
document.body.classList.add('dark', 'darkMode');
}
})();
</script>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>

View File

@@ -11,11 +11,7 @@ const config: Config.InitialOptions = {
moduleFileExtensions: ['ts', 'tsx', 'js', 'json'],
modulePathIgnorePatterns: ['dist'],
moduleNameMapper: {
'\\.(png|jpg|jpeg|gif|svg|webp|avif|ico|bmp|tiff)$':
'<rootDir>/__mocks__/fileMock.ts',
'^@/(.*)$': '<rootDir>/src/$1',
'\\.(css|less|scss)$': '<rootDir>/__mocks__/cssMock.ts',
'\\.module\\.mjs$': '<rootDir>/__mocks__/cssMock.ts',
'\\.md$': '<rootDir>/__mocks__/cssMock.ts',
'^uplot$': '<rootDir>/__mocks__/uplotMock.ts',
'^@signozhq/resizable$': '<rootDir>/__mocks__/resizableMock.tsx',
@@ -45,8 +41,7 @@ const config: Config.InitialOptions = {
'^.+\\.(js|jsx)$': 'babel-jest',
},
transformIgnorePatterns: [
// @chenglou/pretext is ESM-only; @signozhq/ui pulls it in via text-ellipsis.
'node_modules/(?!(lodash-es|react-dnd|core-dnd|@react-dnd|dnd-core|react-dnd-html5-backend|axios|@chenglou/pretext|@signozhq/design-tokens|@signozhq/table|@signozhq/calendar|@signozhq/input|@signozhq/popover|@signozhq/*|date-fns|d3-interpolate|d3-color|api|@codemirror|@lezer|@marijn|@grafana|nuqs)/)',
'node_modules/(?!(lodash-es|react-dnd|core-dnd|@react-dnd|dnd-core|react-dnd-html5-backend|axios|@signozhq/design-tokens|@signozhq/table|@signozhq/calendar|@signozhq/input|@signozhq/popover|@signozhq/button|@signozhq/sonner|@signozhq/*|date-fns|d3-interpolate|d3-color|api|@codemirror|@lezer|@marijn|@grafana|nuqs)/)',
],
setupFilesAfterEnv: ['<rootDir>/jest.setup.ts'],
testPathIgnorePatterns: ['/node_modules/', '/public/'],

View File

@@ -24,34 +24,6 @@ window.matchMedia =
};
};
if (!HTMLElement.prototype.scrollIntoView) {
HTMLElement.prototype.scrollIntoView = function (): void {};
}
// Patch getComputedStyle to handle CSS parsing errors from @signozhq/* packages.
// These packages inject CSS at import time via style-inject / vite-plugin-css-injected-by-js.
// jsdom's nwsapi cannot parse some of the injected selectors (e.g. Tailwind's :animate-in),
// causing SyntaxErrors during getComputedStyle / getByRole calls.
const _origGetComputedStyle = window.getComputedStyle;
window.getComputedStyle = function (
elt: Element,
pseudoElt?: string | null,
): CSSStyleDeclaration {
try {
return _origGetComputedStyle.call(window, elt, pseudoElt);
} catch {
// Return a minimal CSSStyleDeclaration so callers (testing-library, Radix UI)
// see the element as visible and without animations.
return ({
display: '',
visibility: '',
opacity: '1',
animationName: 'none',
getPropertyValue: () => '',
} as unknown) as CSSStyleDeclaration;
}
};
beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());

View File

@@ -8,13 +8,11 @@
"dev": "vite",
"build": "vite build",
"preview": "vite preview",
"prettify": "oxfmt",
"fmt": "echo 'Disabled due to migration' || oxfmt --check",
"lint": "oxlint ./src && stylelint \"src/**/*.scss\"",
"lint:js": "oxlint ./src",
"lint:generated": "oxlint ./src/api/generated --fix",
"lint:fix": "oxlint ./src --fix",
"lint:styles": "stylelint \"src/**/*.scss\"",
"prettify": "prettier --write .",
"fmt": "prettier --check .",
"lint": "eslint ./src",
"lint:generated": "eslint ./src/api/generated --fix",
"lint:fix": "eslint ./src --fix",
"jest": "jest",
"jest:coverage": "jest --coverage",
"jest:watch": "jest --watch",
@@ -44,16 +42,34 @@
"@mdx-js/loader": "2.3.0",
"@mdx-js/react": "2.3.0",
"@monaco-editor/react": "^4.3.1",
"@playwright/test": "1.55.1",
"@radix-ui/react-tabs": "1.0.4",
"@radix-ui/react-tooltip": "1.0.7",
"@sentry/react": "8.41.0",
"@sentry/vite-plugin": "2.22.6",
"@signozhq/design-tokens": "2.1.4",
"@signozhq/badge": "0.0.2",
"@signozhq/button": "0.0.2",
"@signozhq/calendar": "0.0.0",
"@signozhq/callout": "0.0.2",
"@signozhq/checkbox": "0.0.2",
"@signozhq/combobox": "0.0.2",
"@signozhq/command": "0.0.0",
"@signozhq/design-tokens": "2.1.1",
"@signozhq/dialog": "^0.0.2",
"@signozhq/drawer": "0.0.4",
"@signozhq/icons": "0.1.0",
"@signozhq/resizable": "0.0.2",
"@signozhq/ui": "0.0.10",
"@tanstack/react-table": "8.21.3",
"@tanstack/react-virtual": "3.13.22",
"@signozhq/input": "0.0.2",
"@signozhq/popover": "0.0.0",
"@signozhq/radio-group": "0.0.2",
"@signozhq/resizable": "0.0.0",
"@signozhq/sonner": "0.1.0",
"@signozhq/switch": "0.0.2",
"@signozhq/table": "0.3.7",
"@signozhq/toggle-group": "0.0.1",
"@signozhq/tooltip": "0.0.2",
"@signozhq/ui": "0.0.5",
"@tanstack/react-table": "8.20.6",
"@tanstack/react-virtual": "3.11.2",
"@uiw/codemirror-theme-copilot": "4.23.11",
"@uiw/codemirror-theme-github": "4.24.1",
"@uiw/react-codemirror": "4.23.10",
@@ -69,6 +85,7 @@
"antd-table-saveas-excel": "2.2.1",
"antlr4": "4.13.2",
"axios": "1.12.0",
"babel-eslint": "^10.1.0",
"babel-jest": "^29.6.4",
"babel-loader": "9.1.3",
"babel-plugin-named-asset-import": "^0.3.7",
@@ -147,7 +164,6 @@
"vite-plugin-html": "3.2.2",
"web-vitals": "^0.2.4",
"xstate": "^4.31.0",
"zod": "4.3.6",
"zustand": "5.0.11"
},
"browserslist": {
@@ -200,9 +216,20 @@
"@types/redux-mock-store": "1.0.4",
"@types/styled-components": "^5.1.4",
"@types/uuid": "^8.3.1",
"@typescript-eslint/eslint-plugin": "^4.33.0",
"@typescript-eslint/parser": "^4.33.0",
"autoprefixer": "10.4.19",
"babel-plugin-styled-components": "^1.12.0",
"eslint-plugin-sonarjs": "4.0.2",
"eslint": "^7.32.0",
"eslint-config-prettier": "^8.3.0",
"eslint-plugin-import": "^2.28.1",
"eslint-plugin-jest": "^29.15.0",
"eslint-plugin-jsx-a11y": "^6.5.1",
"eslint-plugin-prettier": "^4.0.0",
"eslint-plugin-react": "^7.24.0",
"eslint-plugin-react-hooks": "^4.3.0",
"eslint-plugin-simple-import-sort": "^7.0.0",
"eslint-plugin-sonarjs": "^0.12.0",
"husky": "^7.0.4",
"imagemin": "^8.0.1",
"imagemin-svgo": "^10.0.1",
@@ -214,20 +241,15 @@
"msw": "1.3.2",
"npm-run-all": "latest",
"orval": "7.18.0",
"oxfmt": "0.41.0",
"oxlint": "1.59.0",
"oxlint-tsgolint": "0.20.0",
"portfinder-sync": "^0.0.2",
"postcss": "8.5.6",
"postcss-scss": "4.0.9",
"prettier": "2.2.1",
"prop-types": "15.8.1",
"react-hooks-testing-library": "0.6.0",
"react-resizable": "3.0.4",
"redux-mock-store": "1.5.4",
"sass": "1.97.3",
"sharp": "0.34.5",
"stylelint": "17.7.0",
"stylelint-scss": "7.0.0",
"svgo": "4.0.0",
"ts-api-utils": "2.4.0",
"ts-jest": "29.4.6",
@@ -240,8 +262,7 @@
},
"lint-staged": {
"*.(js|jsx|ts|tsx)": [
"echo 'Disabled due to migration' || oxfmt --check",
"oxlint --fix",
"eslint --fix",
"sh scripts/typecheck-staged.sh"
]
},

View File

@@ -0,0 +1,95 @@
import { defineConfig, devices } from '@playwright/test';
import dotenv from 'dotenv';
import path from 'path';
// Read from ".env" file.
dotenv.config({ path: path.resolve(__dirname, '.env') });
/**
* Read environment variables from file.
* https://github.com/motdotla/dotenv
*/
// import dotenv from 'dotenv';
// import path from 'path';
// dotenv.config({ path: path.resolve(__dirname, '.env') });
/**
* See https://playwright.dev/docs/test-configuration.
*/
export default defineConfig({
testDir: './e2e/tests',
/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Run tests in parallel even in CI - optimized for GitHub Actions free tier */
workers: process.env.CI ? 2 : undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: 'html',
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
baseURL:
process.env.SIGNOZ_E2E_BASE_URL || 'https://app.us.staging.signoz.cloud',
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: 'on-first-retry',
colorScheme: 'dark',
locale: 'en-US',
viewport: { width: 1280, height: 720 },
},
/* Configure projects for major browsers */
projects: [
{
name: 'chromium',
use: {
launchOptions: { args: ['--start-maximized'] },
viewport: null,
colorScheme: 'dark',
locale: 'en-US',
baseURL: 'https://app.us.staging.signoz.cloud',
trace: 'on-first-retry',
},
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
/* Test against mobile viewports. */
// {
// name: 'Mobile Chrome',
// use: { ...devices['Pixel 5'] },
// },
// {
// name: 'Mobile Safari',
// use: { ...devices['iPhone 12'] },
// },
/* Test against branded browsers. */
// {
// name: 'Microsoft Edge',
// use: { ...devices['Desktop Edge'], channel: 'msedge' },
// },
// {
// name: 'Google Chrome',
// use: { ...devices['Desktop Chrome'], channel: 'chrome' },
// },
],
/* Run your local dev server before starting the tests */
// webServer: {
// command: 'npm run start',
// url: 'http://localhost:3000',
// reuseExistingServer: !process.env.CI,
// },
});

View File

@@ -1,47 +0,0 @@
/**
* Rule: no-navigator-clipboard
*
* Prevents direct usage of navigator.clipboard.
*
* This rule catches patterns like:
* navigator.clipboard.writeText(...)
* navigator.clipboard.readText()
* const cb = navigator.clipboard
*
* Instead, use the useCopyToClipboard hook from react-use library.
*
* ESLint equivalent:
* "no-restricted-properties": [
* "error",
* {
* "object": "navigator",
* "property": "clipboard",
* "message": "Do not use navigator.clipboard directly..."
* }
* ]
*/
export default {
create(context) {
return {
MemberExpression(node) {
const object = node.object;
const property = node.property;
// Check if it's navigator.clipboard
if (
object.type === 'Identifier' &&
object.name === 'navigator' &&
property.type === 'Identifier' &&
property.name === 'clipboard'
) {
context.report({
node,
message:
'Do not use navigator.clipboard directly since it does not work well with specific browsers. Use hook useCopyToClipboard from react-use library. https://streamich.github.io/react-use/?path=/story/side-effects-usecopytoclipboard--docs',
});
}
},
};
},
};

View File

@@ -1,333 +0,0 @@
/**
* Rule: no-unsupported-asset-pattern
*
* Enforces that all asset references (SVG, PNG, etc.) go through Vite's module
* pipeline via ES imports (`import fooUrl from '@/assets/...'`) rather than
* hard-coded strings or public/ paths.
*
* Why this matters: when the app is served from a sub-path (base-path deployment),
* Vite rewrites ES import URLs automatically. String literals and public/ references
* bypass that rewrite and break at runtime.
*
* Covers four AST patterns:
* 1. Literal — plain string: "/Icons/logo.svg"
* 2. TemplateLiteral — template string: `/Icons/${name}.svg`
* 3. BinaryExpression — concatenation: "/Icons/" + name + ".svg"
* 4. ImportDeclaration / ImportExpression — static & dynamic imports
*/
import {
hasAssetExtension,
containsAssetExtension,
extractUrlPath,
isAbsolutePath,
isPublicRelative,
isRelativePublicDir,
isValidAssetImport,
isExternalUrl,
} from './shared/asset-patterns.mjs';
const PUBLIC_DIR_SEGMENTS = ['/Icons/', '/Images/', '/Logos/', '/svgs/'];
function collectBinaryStringParts(node) {
if (node.type === 'Literal' && typeof node.value === 'string')
return [node.value];
if (node.type === 'BinaryExpression' && node.operator === '+') {
return [
...collectBinaryStringParts(node.left),
...collectBinaryStringParts(node.right),
];
}
if (node.type === 'TemplateLiteral') {
return node.quasis.map((q) => q.value.raw);
}
return [null];
}
export default {
meta: {
type: 'problem',
docs: {
description:
'Disallow Vite-unsafe asset reference patterns that break runtime base-path deployments',
category: 'Asset Migration',
recommended: true,
},
schema: [],
messages: {
absoluteString:
'Absolute asset path "{{ value }}" is not base-path-safe. ' +
"Use an ES import instead: import fooUrl from '@/assets/...' and reference the variable.",
templateLiteral:
'Dynamic asset path with absolute prefix is not base-path-safe. ' +
"Use new URL('./asset.svg', import.meta.url).href for dynamic asset paths.",
absoluteImport:
'Asset imported via absolute path is not supported. ' +
"Use import fooUrl from '@/assets/...' instead.",
publicImport:
"Assets in public/ bypass Vite's module pipeline — their URLs are not base-path-aware and will break when the app is served from a sub-path (e.g. /app/). " +
"Use an ES import instead: import fooUrl from '@/assets/...' so Vite injects the correct base path.",
relativePublicString:
'Relative public-dir path "{{ value }}" is not base-path-safe. ' +
"Use an ES import instead: import fooUrl from '@/assets/...' and reference the variable.",
invalidAssetImport:
"Asset '{{ src }}' must be imported from src/assets/ using either '@/assets/...' " +
'or a relative path into src/assets/.',
},
},
create(context) {
return {
Literal(node) {
if (node.parent && node.parent.type === 'ImportDeclaration') {
return;
}
const value = node.value;
if (typeof value !== 'string') return;
if (isExternalUrl(value)) return;
if (isAbsolutePath(value) && containsAssetExtension(value)) {
context.report({
node,
messageId: 'absoluteString',
data: { value },
});
return;
}
if (isRelativePublicDir(value) && containsAssetExtension(value)) {
context.report({
node,
messageId: 'relativePublicString',
data: { value },
});
return;
}
if (isPublicRelative(value) && containsAssetExtension(value)) {
context.report({
node,
messageId: 'relativePublicString',
data: { value },
});
return;
}
const urlPath = extractUrlPath(value);
if (urlPath && isExternalUrl(urlPath)) return;
if (urlPath && isAbsolutePath(urlPath) && containsAssetExtension(urlPath)) {
context.report({
node,
messageId: 'absoluteString',
data: { value: urlPath },
});
return;
}
if (
urlPath &&
isRelativePublicDir(urlPath) &&
containsAssetExtension(urlPath)
) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: urlPath },
});
return;
}
if (
urlPath &&
isPublicRelative(urlPath) &&
containsAssetExtension(urlPath)
) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: urlPath },
});
}
},
TemplateLiteral(node) {
const quasis = node.quasis;
if (!quasis || quasis.length === 0) return;
const firstQuasi = quasis[0].value.raw;
if (isExternalUrl(firstQuasi)) return;
const hasAssetExt = quasis.some((q) => containsAssetExtension(q.value.raw));
if (isAbsolutePath(firstQuasi) && hasAssetExt) {
context.report({
node,
messageId: 'templateLiteral',
});
return;
}
if (isRelativePublicDir(firstQuasi) && hasAssetExt) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: firstQuasi },
});
return;
}
const hasPublicSegment = quasis.some((q) =>
PUBLIC_DIR_SEGMENTS.some((seg) => q.value.raw.includes(seg)),
);
if (hasPublicSegment && hasAssetExt) {
context.report({
node,
messageId: 'templateLiteral',
});
return;
}
if (quasis.length === 1) {
if (isPublicRelative(firstQuasi) && hasAssetExt) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: firstQuasi },
});
return;
}
const urlPath = extractUrlPath(firstQuasi);
if (urlPath && isExternalUrl(urlPath)) return;
if (urlPath && isAbsolutePath(urlPath) && hasAssetExtension(urlPath)) {
context.report({
node,
messageId: 'templateLiteral',
});
return;
}
if (
urlPath &&
isRelativePublicDir(urlPath) &&
hasAssetExtension(urlPath)
) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: urlPath },
});
return;
}
if (urlPath && isPublicRelative(urlPath) && hasAssetExtension(urlPath)) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: urlPath },
});
}
return;
}
if (firstQuasi.includes('url(') && hasAssetExt) {
const urlMatch = firstQuasi.match(/^url\(\s*['"]?\//);
if (urlMatch) {
context.report({
node,
messageId: 'templateLiteral',
});
}
}
},
BinaryExpression(node) {
if (node.operator !== '+') return;
const parts = collectBinaryStringParts(node);
const prefixParts = [];
for (const part of parts) {
if (part === null) break;
prefixParts.push(part);
}
const staticPrefix = prefixParts.join('');
if (isExternalUrl(staticPrefix)) return;
const hasExt = parts.some(
(part) => part !== null && containsAssetExtension(part),
);
if (isAbsolutePath(staticPrefix) && hasExt) {
context.report({
node,
messageId: 'templateLiteral',
});
return;
}
if (isPublicRelative(staticPrefix) && hasExt) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: staticPrefix },
});
return;
}
if (isRelativePublicDir(staticPrefix) && hasExt) {
context.report({
node,
messageId: 'relativePublicString',
data: { value: staticPrefix },
});
}
},
ImportDeclaration(node) {
const src = node.source.value;
if (typeof src !== 'string') return;
if (!hasAssetExtension(src)) return;
if (isAbsolutePath(src)) {
context.report({ node, messageId: 'absoluteImport' });
return;
}
if (isPublicRelative(src)) {
context.report({ node, messageId: 'publicImport' });
return;
}
if (!isValidAssetImport(src)) {
context.report({
node,
messageId: 'invalidAssetImport',
data: { src },
});
}
},
ImportExpression(node) {
const src = node.source;
if (!src || src.type !== 'Literal' || typeof src.value !== 'string') return;
const value = src.value;
if (!hasAssetExtension(value)) return;
if (isAbsolutePath(value)) {
context.report({ node, messageId: 'absoluteImport' });
return;
}
if (isPublicRelative(value)) {
context.report({ node, messageId: 'publicImport' });
return;
}
if (!isValidAssetImport(value)) {
context.report({
node,
messageId: 'invalidAssetImport',
data: { src: value },
});
}
},
};
},
};

View File

@@ -1,53 +0,0 @@
/**
* Rule: no-zustand-getstate-in-hooks
*
* Prevents calling .getState() on Zustand hooks.
*
* This rule catches patterns like:
* useStore.getState()
* useAppStore.getState()
*
* Instead, export a standalone action from the store.
*
* ESLint equivalent:
* "no-restricted-syntax": [
* "error",
* {
* "selector": "CallExpression[callee.property.name='getState'][callee.object.name=/^use/]",
* "message": "Avoid calling .getState() directly. Export a standalone action from the store instead."
* }
* ]
*/
export default {
create(context) {
return {
CallExpression(node) {
const callee = node.callee;
// Check if it's a member expression (e.g., useStore.getState())
if (callee.type !== 'MemberExpression') {
return;
}
// Check if the property is 'getState'
const property = callee.property;
if (property.type !== 'Identifier' || property.name !== 'getState') {
return;
}
// Check if the object name starts with 'use'
const object = callee.object;
if (object.type !== 'Identifier' || !object.name.startsWith('use')) {
return;
}
context.report({
node,
message:
'Avoid calling .getState() directly. Export a standalone action from the store instead.',
});
},
};
},
};

View File

@@ -1,106 +0,0 @@
export const ALLOWED_ASSET_EXTENSIONS = [
'.svg',
'.png',
'.webp',
'.jpg',
'.jpeg',
'.gif',
];
/**
* Returns true if the string ends with an asset extension.
* e.g. "/Icons/foo.svg" → true, "/Icons/foo.svg.bak" → false
*/
export function hasAssetExtension(str) {
if (typeof str !== 'string') return false;
return ALLOWED_ASSET_EXTENSIONS.some((ext) => str.endsWith(ext));
}
// Like hasAssetExtension but also matches mid-string with boundary check,
// e.g. "/foo.svg?v=1" → true, "/icons.svg-dir/" → true (- is non-alphanumeric boundary)
export function containsAssetExtension(str) {
if (typeof str !== 'string') return false;
return ALLOWED_ASSET_EXTENSIONS.some((ext) => {
const idx = str.indexOf(ext);
if (idx === -1) return false;
const afterIdx = idx + ext.length;
// Broad boundary (any non-alphanumeric) is intentional — the real guard against
// false positives is the upstream conditions (isAbsolutePath, isRelativePublicDir, etc.)
// that must pass before this is reached. "/icons.svg-dir/" → true (- is a boundary).
return afterIdx >= str.length || /[^a-zA-Z0-9]/.test(str[afterIdx]);
});
}
/**
* Extracts the asset path from a CSS url() wrapper.
* Handles single quotes, double quotes, unquoted, and whitespace variations.
* e.g.
* "url('/Icons/foo.svg')" → "/Icons/foo.svg"
* "url( '../assets/bg.png' )" → "../assets/bg.png"
* "url(/Icons/foo.svg)" → "/Icons/foo.svg"
* Returns null if the string is not a url() wrapper.
*/
export function extractUrlPath(str) {
if (typeof str !== 'string') return null;
// Match url( [whitespace] [quote?] path [quote?] [whitespace] )
// Capture group: [^'")\s]+ matches path until quote, closing paren, or whitespace
const match = str.match(/^url\(\s*['"]?([^'")\s]+)['"]?\s*\)$/);
return match ? match[1] : null;
}
/**
* Returns true if the string is an absolute path (starts with /).
* Absolute paths in url() bypass <base href> and fail under any URL prefix.
*/
export function isAbsolutePath(str) {
if (typeof str !== 'string') return false;
return str.startsWith('/') && !str.startsWith('//');
}
/**
* Returns true if the path imports from the public/ directory.
* Relative imports into public/ cause asset duplication in dist/.
*/
export function isPublicRelative(str) {
if (typeof str !== 'string') return false;
return str.includes('/public/') || str.startsWith('public/');
}
/**
* Returns true if the string is a relative reference into a known public-dir folder.
* e.g. "Icons/foo.svg", `Logos/aws-dark.svg`, "Images/bg.png"
* These bypass Vite's module pipeline even without a leading slash.
*/
export const PUBLIC_DIR_SEGMENTS = ['Icons/', 'Images/', 'Logos/', 'svgs/'];
export function isRelativePublicDir(str) {
if (typeof str !== 'string') return false;
return PUBLIC_DIR_SEGMENTS.some((seg) => str.startsWith(seg));
}
/**
* Returns true if an asset import path is valid (goes through Vite's module pipeline).
* Valid: @/assets/..., any relative path containing /assets/, or node_modules packages.
* Invalid: absolute paths, public/ dir, or relative paths outside src/assets/.
*/
export function isValidAssetImport(str) {
if (typeof str !== 'string') return false;
if (str.startsWith('@/assets/')) return true;
if (str.includes('/assets/')) return true;
// Not starting with . or / means it's a node_modules package — always valid
if (!str.startsWith('.') && !str.startsWith('/')) return true;
return false;
}
/**
* Returns true if the string is an external URL.
* Used to avoid false positives on CDN/API URLs with asset extensions.
*/
export function isExternalUrl(str) {
if (typeof str !== 'string') return false;
return (
str.startsWith('http://') ||
str.startsWith('https://') ||
str.startsWith('//')
);
}

View File

@@ -1,21 +0,0 @@
/**
* Oxlint custom rules plugin for SigNoz.
*
* This plugin aggregates all custom SigNoz linting rules.
* Individual rules are defined in the ./rules directory.
*/
import noZustandGetStateInHooks from './rules/no-zustand-getstate-in-hooks.mjs';
import noNavigatorClipboard from './rules/no-navigator-clipboard.mjs';
import noUnsupportedAssetPattern from './rules/no-unsupported-asset-pattern.mjs';
export default {
meta: {
name: 'signoz',
},
rules: {
'no-zustand-getstate-in-hooks': noZustandGetStateInHooks,
'no-navigator-clipboard': noNavigatorClipboard,
'no-unsupported-asset-pattern': noUnsupportedAssetPattern,
},
};

View File

@@ -0,0 +1,16 @@
RULE: All test code for this repo must be generated by following the step-by-step Playwright MCP process as described below.
- You are a playwright test generator.
- You are given a scenario and you need to generate a playwright test for it.
- Use login util if not logged in.
- DO NOT generate test code based on the scenario alone.
- DO run steps one by one using the tools provided by the Playwright MCP.
- Only after all steps are completed, emit a Playwright TypeScript test that uses @playwright/test based on message history
- Gather correct selectors before writing the test
- DO NOT valiate for dynamic content in the tests, only validate for the correctness with meta data
- Always inspect the DOM at each navigation or interaction step to determine the correct selector for the next action. Do not assume selectors, confirm via inspection before proceeding.
- Assert visibility of definitive/static elements in the UI (such as labels, headings, or section titles) rather than dynamic values or content that may change between runs.
- Save generated test file in the tests directory
- Execute the test file and iterate until the test passes

View File

Before

Width:  |  Height:  |  Size: 9.1 KiB

After

Width:  |  Height:  |  Size: 9.1 KiB

View File

Before

Width:  |  Height:  |  Size: 2.3 KiB

After

Width:  |  Height:  |  Size: 2.3 KiB

Some files were not shown because too many files have changed in this diff Show More