Compare commits

..

51 Commits

Author SHA1 Message Date
swapnil-signoz
94f432a3db Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-23 21:03:58 +05:30
swapnil-signoz
0276496669 refactor: method chaining formatting 2026-03-23 19:17:48 +05:30
swapnil-signoz
6f3a2e6a83 refactor: returning error only for create methods 2026-03-23 18:02:41 +05:30
swapnil-signoz
d9f8a4ae5a Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-22 17:13:40 +05:30
swapnil-signoz
2d8a00bf18 fix: update error code for service not found 2026-03-20 20:53:33 +05:30
swapnil-signoz
f1b26b310f Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-20 20:51:44 +05:30
swapnil-signoz
d6caa4f2c7 Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-18 14:08:14 +05:30
swapnil-signoz
f86371566d refactor: clean up 2026-03-18 13:45:31 +05:30
swapnil-signoz
9115803084 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-18 13:42:43 +05:30
swapnil-signoz
0c14d8f966 refactor: review comments 2026-03-18 13:40:17 +05:30
swapnil-signoz
7afb461af8 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-18 11:14:33 +05:30
swapnil-signoz
a21fbb4ee0 refactor: clean up 2026-03-18 11:14:05 +05:30
swapnil-signoz
0369842f3d refactor: clean up 2026-03-17 23:40:14 +05:30
swapnil-signoz
59cd96562a Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 23:10:54 +05:30
swapnil-signoz
cc4475cab7 refactor: updating store methods 2026-03-17 23:10:15 +05:30
swapnil-signoz
ac8c648420 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 21:09:47 +05:30
swapnil-signoz
bede6be4b8 feat: adding method for service id creation 2026-03-17 21:09:26 +05:30
swapnil-signoz
dd3d60e6df Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 20:49:31 +05:30
swapnil-signoz
538ab686d2 refactor: using serviceID type 2026-03-17 20:49:17 +05:30
swapnil-signoz
936a325cb9 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-17 17:25:58 +05:30
swapnil-signoz
c6cdcd0143 refactor: renaming service type to service id 2026-03-17 17:25:29 +05:30
swapnil-signoz
cd9211d718 refactor: clean up types 2026-03-17 17:04:27 +05:30
swapnil-signoz
0601c28782 feat: adding integration test 2026-03-17 11:02:46 +05:30
swapnil-signoz
580610dbfa Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-16 23:02:19 +05:30
swapnil-signoz
2d2aa02a81 refactor: split upsert store method 2026-03-16 18:27:42 +05:30
swapnil-signoz
dd9723ad13 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-16 17:42:03 +05:30
swapnil-signoz
3651469416 Merge branch 'main' of https://github.com/SigNoz/signoz into refactor/cloud-integration-types 2026-03-16 17:41:52 +05:30
swapnil-signoz
febce75734 refactor: update Dashboard struct comments and remove unused fields 2026-03-16 17:41:28 +05:30
swapnil-signoz
e1616f3487 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-16 17:36:15 +05:30
swapnil-signoz
4b94287ac7 refactor: add comments for backward compatibility in PostableAgentCheckInRequest 2026-03-16 15:48:20 +05:30
swapnil-signoz
1575c7c54c refactor: streamlining types 2026-03-16 15:39:32 +05:30
swapnil-signoz
8def3f835b refactor: adding comments and removed wrong code 2026-03-16 11:10:53 +05:30
swapnil-signoz
11ed15f4c5 feat: implement cloud integration store 2026-03-14 17:05:02 +05:30
swapnil-signoz
f47877cca9 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-14 17:01:51 +05:30
swapnil-signoz
bb2b9215ba fix: correct GetService signature and remove shadowed Data field 2026-03-14 16:59:07 +05:30
swapnil-signoz
3111904223 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-14 16:36:35 +05:30
swapnil-signoz
003e2c30d8 Merge branch 'main' into refactor/cloud-integration-types 2026-03-14 16:25:35 +05:30
swapnil-signoz
00fe516d10 refactor: update cloud integration types and module interface 2026-03-14 16:25:16 +05:30
swapnil-signoz
0305f4f7db refactor: using struct for map 2026-03-13 16:09:26 +05:30
swapnil-signoz
c60019a6dc Merge branch 'main' into refactor/cloud-integration-types 2026-03-12 23:41:22 +05:30
swapnil-signoz
acde2a37fa feat: adding updated types for cloud integration 2026-03-12 23:40:44 +05:30
swapnil-signoz
945241a52a Merge branch 'main' into refactor/cloud-integration-types 2026-03-12 19:45:50 +05:30
swapnil-signoz
e967f80c86 Merge branch 'main' into refactor/cloud-integration-types 2026-03-02 16:39:42 +05:30
swapnil-signoz
a09dc325de Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-02 16:39:20 +05:30
swapnil-signoz
379b4f7fc4 refactor: removing interface check 2026-03-02 14:50:37 +05:30
swapnil-signoz
5e536ae077 Merge branch 'refactor/cloud-integration-types' into refactor/cloud-integration-impl-store 2026-03-02 14:49:35 +05:30
swapnil-signoz
234585e642 Merge branch 'main' into refactor/cloud-integration-types 2026-03-02 14:49:19 +05:30
swapnil-signoz
2cc14f1ad4 Merge branch 'main' into refactor/cloud-integration-impl-store 2026-03-02 14:49:00 +05:30
swapnil-signoz
dc4ed4d239 feat: adding sql store implementation 2026-03-02 14:44:56 +05:30
swapnil-signoz
7281c36873 refactor: store interfaces to use local types and error 2026-03-02 13:27:46 +05:30
swapnil-signoz
40288776e8 feat: adding cloud integration type for refactor 2026-02-28 16:59:14 +05:30
1326 changed files with 30791 additions and 145190 deletions

View File

@@ -27,8 +27,8 @@ services:
- ${PWD}/fs/tmp/var/lib/clickhouse/user_scripts/:/var/lib/clickhouse/user_scripts/
- ${PWD}/../../../deploy/common/clickhouse/custom-function.xml:/etc/clickhouse-server/custom-function.xml
ports:
- "127.0.0.1:8123:8123"
- "127.0.0.1:9000:9000"
- '127.0.0.1:8123:8123'
- '127.0.0.1:9000:9000'
tty: true
healthcheck:
test:
@@ -47,16 +47,13 @@ services:
condition: service_healthy
environment:
- CLICKHOUSE_SKIP_USER_SETUP=1
networks:
- default
- signoz-devenv
zookeeper:
image: signoz/zookeeper:3.7.1
container_name: zookeeper
volumes:
- ${PWD}/fs/tmp/zookeeper:/bitnami/zookeeper
ports:
- "127.0.0.1:2181:2181"
- '127.0.0.1:2181:2181'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
healthcheck:
@@ -77,19 +74,12 @@ services:
entrypoint:
- /bin/sh
command:
- -c
- |
/signoz-otel-collector migrate bootstrap &&
/signoz-otel-collector migrate sync up &&
/signoz-otel-collector migrate async up
- -c
- |
/signoz-otel-collector migrate bootstrap &&
/signoz-otel-collector migrate sync up &&
/signoz-otel-collector migrate async up
depends_on:
clickhouse:
condition: service_healthy
restart: on-failure
networks:
- default
- signoz-devenv
networks:
signoz-devenv:
name: signoz-devenv

View File

@@ -3,7 +3,7 @@ services:
image: signoz/signoz-otel-collector:v0.142.0
container_name: signoz-otel-collector-dev
entrypoint:
- /bin/sh
- /bin/sh
command:
- -c
- |
@@ -34,11 +34,4 @@ services:
retries: 3
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- default
- signoz-devenv
networks:
signoz-devenv:
name: signoz-devenv
- "host.docker.internal:host-gateway"

View File

@@ -12,10 +12,10 @@ receivers:
scrape_configs:
- job_name: otel-collector
static_configs:
- targets:
- localhost:8888
labels:
job_name: otel-collector
- targets:
- localhost:8888
labels:
job_name: otel-collector
processors:
batch:
@@ -29,26 +29,7 @@ processors:
signozspanmetrics/delta:
metrics_exporter: signozclickhousemetrics
metrics_flush_interval: 60s
latency_histogram_buckets:
[
100us,
1ms,
2ms,
6ms,
10ms,
50ms,
100ms,
250ms,
500ms,
1000ms,
1400ms,
2000ms,
5s,
10s,
20s,
40s,
60s,
]
latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 50ms, 100ms, 250ms, 500ms, 1000ms, 1400ms, 2000ms, 5s, 10s, 20s, 40s, 60s ]
dimensions_cache_size: 100000
aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA
enable_exp_histogram: true
@@ -79,13 +60,13 @@ extensions:
exporters:
clickhousetraces:
datasource: tcp://clickhouse:9000/signoz_traces
datasource: tcp://host.docker.internal:9000/signoz_traces
low_cardinal_exception_grouping: ${env:LOW_CARDINAL_EXCEPTION_GROUPING}
use_new_schema: true
signozclickhousemetrics:
dsn: tcp://clickhouse:9000/signoz_metrics
dsn: tcp://host.docker.internal:9000/signoz_metrics
clickhouselogsexporter:
dsn: tcp://clickhouse:9000/signoz_logs
dsn: tcp://host.docker.internal:9000/signoz_logs
timeout: 10s
use_new_schema: true
@@ -112,4 +93,4 @@ service:
logs:
receivers: [otlp]
processors: [batch]
exporters: [clickhouselogsexporter]
exporters: [clickhouselogsexporter]

4
.github/CODEOWNERS vendored
View File

@@ -86,8 +86,6 @@ go.mod @therealpandey
/pkg/types/alertmanagertypes @srikanthccv
/pkg/alertmanager/ @srikanthccv
/pkg/ruler/ @srikanthccv
/pkg/modules/rulestatehistory/ @srikanthccv
/pkg/types/rulestatehistorytypes/ @srikanthccv
# Correlation-adjacent
@@ -107,7 +105,7 @@ go.mod @therealpandey
/pkg/modules/authdomain/ @vikrantgupta25
/pkg/modules/role/ @vikrantgupta25
# IdentN Owners
# IdentN Owners
/pkg/identn/ @vikrantgupta25
/pkg/http/middleware/identn.go @vikrantgupta25

View File

@@ -51,12 +51,9 @@ jobs:
- alerts
- ingestionkeys
- rootuser
- serviceaccount
sqlstore-provider:
- postgres
- sqlite
sqlite-mode:
- wal
clickhouse-version:
- 25.5.6
- 25.12.5
@@ -104,7 +101,6 @@ jobs:
--basetemp=./tmp/ \
src/${{matrix.src}} \
--sqlstore-provider ${{matrix.sqlstore-provider}} \
--sqlite-mode ${{matrix.sqlite-mode}} \
--postgres-version ${{matrix.postgres-version}} \
--clickhouse-version ${{matrix.clickhouse-version}} \
--schema-migrator-version ${{matrix.schema-migrator-version}}

2
.gitignore vendored
View File

@@ -51,8 +51,6 @@ ee/query-service/tests/test-deploy/data/
# local data
*.backup
*.db
*.db-shm
*.db-wal
**/db
/deploy/docker/clickhouse-setup/data/
/deploy/docker-swarm/clickhouse-setup/data/

View File

@@ -6,14 +6,12 @@ linters:
- depguard
- errcheck
- forbidigo
- godot
- govet
- iface
- ineffassign
- misspell
- nilnil
- sloglint
- staticcheck
- wastedassign
- unparam
- unused

View File

@@ -17,7 +17,5 @@
},
"[html]": {
"editor.defaultFormatter": "vscode.html-language-features"
},
"python-envs.defaultEnvManager": "ms-python.python:system",
"python-envs.pythonProjects": []
}
}

View File

@@ -8,25 +8,19 @@ import (
"github.com/SigNoz/signoz/cmd"
"github.com/SigNoz/signoz/pkg/analytics"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authn"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/authz/openfgaauthz"
"github.com/SigNoz/signoz/pkg/authz/openfgaschema"
"github.com/SigNoz/signoz/pkg/authz/openfgaserver"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/gateway/noopgateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/licensing/nooplicensing"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration/implcloudintegration"
"github.com/SigNoz/signoz/pkg/modules/dashboard"
"github.com/SigNoz/signoz/pkg/modules/dashboard/impldashboard"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/query-service/app"
"github.com/SigNoz/signoz/pkg/queryparser"
@@ -84,13 +78,8 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(ctx context.Context, providerSettings factory.ProviderSettings, store authtypes.AuthNStore, licensing licensing.Licensing) (map[authtypes.AuthNProvider]authn.AuthN, error) {
return signoz.NewAuthNs(ctx, providerSettings, store, licensing)
},
func(ctx context.Context, sqlstore sqlstore.SQLStore, _ licensing.Licensing, _ dashboard.Module) (factory.ProviderFactory[authz.AuthZ, authz.Config], error) {
openfgaDataStore, err := openfgaserver.NewSQLStore(sqlstore)
if err != nil {
return nil, err
}
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), openfgaDataStore), nil
func(ctx context.Context, sqlstore sqlstore.SQLStore, _ licensing.Licensing, _ dashboard.Module) factory.ProviderFactory[authz.AuthZ, authz.Config] {
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx))
},
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, _ querier.Querier, _ licensing.Licensing) dashboard.Module {
return impldashboard.NewModule(impldashboard.NewStore(store), settings, analytics, orgGetter, queryParser)
@@ -98,15 +87,9 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(_ licensing.Licensing) factory.ProviderFactory[gateway.Gateway, gateway.Config] {
return noopgateway.NewProviderFactory()
},
func(_ licensing.Licensing) factory.NamedMap[factory.ProviderFactory[auditor.Auditor, auditor.Config]] {
return signoz.NewAuditorProviderFactories()
},
func(ps factory.ProviderSettings, q querier.Querier, a analytics.Analytics) querier.Handler {
return querier.NewHandler(ps, q, a)
},
func(_ sqlstore.SQLStore, _ global.Global, _ zeus.Zeus, _ gateway.Gateway, _ licensing.Licensing, _ serviceaccount.Module, _ cloudintegration.Config) (cloudintegration.Module, error) {
return implcloudintegration.NewModule(), nil
},
)
if err != nil {
logger.ErrorContext(ctx, "failed to create signoz", errors.Attr(err))

View File

@@ -8,17 +8,13 @@ import (
"github.com/spf13/cobra"
"github.com/SigNoz/signoz/cmd"
"github.com/SigNoz/signoz/ee/auditor/otlphttpauditor"
"github.com/SigNoz/signoz/ee/authn/callbackauthn/oidccallbackauthn"
"github.com/SigNoz/signoz/ee/authn/callbackauthn/samlcallbackauthn"
"github.com/SigNoz/signoz/ee/authz/openfgaauthz"
"github.com/SigNoz/signoz/ee/authz/openfgaschema"
"github.com/SigNoz/signoz/ee/authz/openfgaserver"
"github.com/SigNoz/signoz/ee/gateway/httpgateway"
enterpriselicensing "github.com/SigNoz/signoz/ee/licensing"
"github.com/SigNoz/signoz/ee/licensing/httplicensing"
"github.com/SigNoz/signoz/ee/modules/cloudintegration/implcloudintegration"
"github.com/SigNoz/signoz/ee/modules/cloudintegration/implcloudintegration/implcloudprovider"
"github.com/SigNoz/signoz/ee/modules/dashboard/impldashboard"
eequerier "github.com/SigNoz/signoz/ee/querier"
enterpriseapp "github.com/SigNoz/signoz/ee/query-service/app"
@@ -27,20 +23,15 @@ import (
enterprisezeus "github.com/SigNoz/signoz/ee/zeus"
"github.com/SigNoz/signoz/ee/zeus/httpzeus"
"github.com/SigNoz/signoz/pkg/analytics"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/authn"
"github.com/SigNoz/signoz/pkg/authz"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
pkgcloudintegration "github.com/SigNoz/signoz/pkg/modules/cloudintegration/implcloudintegration"
"github.com/SigNoz/signoz/pkg/modules/dashboard"
pkgimpldashboard "github.com/SigNoz/signoz/pkg/modules/dashboard/impldashboard"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/queryparser"
"github.com/SigNoz/signoz/pkg/signoz"
@@ -48,7 +39,6 @@ import (
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/sqlstore/sqlstorehook"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
"github.com/SigNoz/signoz/pkg/version"
"github.com/SigNoz/signoz/pkg/zeus"
)
@@ -128,12 +118,8 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
return authNs, nil
},
func(ctx context.Context, sqlstore sqlstore.SQLStore, licensing licensing.Licensing, dashboardModule dashboard.Module) (factory.ProviderFactory[authz.AuthZ, authz.Config], error) {
openfgaDataStore, err := openfgaserver.NewSQLStore(sqlstore)
if err != nil {
return nil, err
}
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), openfgaDataStore, licensing, dashboardModule), nil
func(ctx context.Context, sqlstore sqlstore.SQLStore, licensing licensing.Licensing, dashboardModule dashboard.Module) factory.ProviderFactory[authz.AuthZ, authz.Config] {
return openfgaauthz.NewProviderFactory(sqlstore, openfgaschema.NewSchema().Get(ctx), licensing, dashboardModule)
},
func(store sqlstore.SQLStore, settings factory.ProviderSettings, analytics analytics.Analytics, orgGetter organization.Getter, queryParser queryparser.QueryParser, querier querier.Querier, licensing licensing.Licensing) dashboard.Module {
return impldashboard.NewModule(pkgimpldashboard.NewStore(store), settings, analytics, orgGetter, queryParser, querier, licensing)
@@ -141,32 +127,12 @@ func runServer(ctx context.Context, config signoz.Config, logger *slog.Logger) e
func(licensing licensing.Licensing) factory.ProviderFactory[gateway.Gateway, gateway.Config] {
return httpgateway.NewProviderFactory(licensing)
},
func(licensing licensing.Licensing) factory.NamedMap[factory.ProviderFactory[auditor.Auditor, auditor.Config]] {
factories := signoz.NewAuditorProviderFactories()
if err := factories.Add(otlphttpauditor.NewFactory(licensing, version.Info)); err != nil {
panic(err)
}
return factories
},
func(ps factory.ProviderSettings, q querier.Querier, a analytics.Analytics) querier.Handler {
communityHandler := querier.NewHandler(ps, q, a)
return eequerier.NewHandler(ps, q, communityHandler)
},
func(sqlStore sqlstore.SQLStore, global global.Global, zeus zeus.Zeus, gateway gateway.Gateway, licensing licensing.Licensing, serviceAccount serviceaccount.Module, config cloudintegration.Config) (cloudintegration.Module, error) {
defStore := pkgcloudintegration.NewServiceDefinitionStore()
awsCloudProviderModule, err := implcloudprovider.NewAWSCloudProvider(defStore)
if err != nil {
return nil, err
}
azureCloudProviderModule := implcloudprovider.NewAzureCloudProvider()
cloudProvidersMap := map[cloudintegrationtypes.CloudProviderType]cloudintegration.CloudProviderModule{
cloudintegrationtypes.CloudProviderTypeAWS: awsCloudProviderModule,
cloudintegrationtypes.CloudProviderTypeAzure: azureCloudProviderModule,
}
return implcloudintegration.NewModule(pkgcloudintegration.NewStore(sqlStore), global, zeus, gateway, licensing, serviceAccount, cloudProvidersMap, config)
},
)
if err != nil {
logger.ErrorContext(ctx, "failed to create signoz", errors.Attr(err))
return err

View File

@@ -82,18 +82,13 @@ sqlstore:
provider: sqlite
# The maximum number of open connections to the database.
max_open_conns: 100
# The maximum amount of time a connection may be reused.
# If max_conn_lifetime == 0, connections are not closed due to a connection's age.
max_conn_lifetime: 0
sqlite:
# The path to the SQLite database file.
path: /var/lib/signoz/signoz.db
# The journal mode for the sqlite database. Supported values: delete, wal.
mode: wal
# The timeout for the sqlite database to wait for a lock.
# Mode is the mode to use for the sqlite database.
mode: delete
# BusyTimeout is the timeout for the sqlite database to wait for a lock.
busy_timeout: 10s
# The default transaction locking behavior. Supported values: deferred, immediate, exclusive.
transaction_mode: deferred
##################### APIServer #####################
apiserver:
@@ -149,8 +144,6 @@ telemetrystore:
##################### Prometheus #####################
prometheus:
# The maximum time a PromQL query is allowed to run before being aborted.
timeout: 2m
active_query_tracker:
# Whether to enable the active query tracker.
enabled: true
@@ -357,51 +350,3 @@ identn:
impersonation:
# toggle impersonation identN, when enabled, all requests will impersonate the root user
enabled: false
##################### Service Account #####################
serviceaccount:
email:
# email domain for the service account principal
domain: signozserviceaccount.com
analytics:
# toggle service account analytics
enabled: true
##################### Auditor #####################
auditor:
# Specifies the auditor provider to use.
# noop: discards all audit events (community default).
# otlphttp: exports audit events via OTLP HTTP (enterprise).
provider: noop
# The async channel capacity for audit events. Events are dropped when full (fail-open).
buffer_size: 1000
# The maximum number of events per export batch.
batch_size: 100
# The maximum time between export flushes.
flush_interval: 1s
otlphttp:
# The target scheme://host:port/path of the OTLP HTTP endpoint.
endpoint: http://localhost:4318/v1/logs
# Whether to use HTTP instead of HTTPS.
insecure: false
# The maximum duration for an export attempt.
timeout: 10s
# Additional HTTP headers sent with every export request.
headers: {}
retry:
# Whether to retry on transient failures.
enabled: true
# The initial wait time before the first retry.
initial_interval: 5s
# The upper bound on backoff interval.
max_interval: 30s
# The total maximum time spent retrying.
max_elapsed_time: 60s
##################### Cloud Integration #####################
cloudintegration:
# cloud integration agent configuration
agent:
# The version of the cloud integration agent.
version: v0.0.8

View File

@@ -190,7 +190,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:v0.118.0
image: signoz/signoz:v0.116.1
ports:
- "8080:8080" # signoz port
# - "6060:6060" # pprof port

View File

@@ -117,7 +117,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:v0.118.0
image: signoz/signoz:v0.116.1
ports:
- "8080:8080" # signoz port
volumes:

View File

@@ -181,7 +181,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:${VERSION:-v0.118.0}
image: signoz/signoz:${VERSION:-v0.116.1}
container_name: signoz
ports:
- "8080:8080" # signoz port

View File

@@ -109,7 +109,7 @@ services:
# - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
signoz:
!!merge <<: *db-depend
image: signoz/signoz:${VERSION:-v0.118.0}
image: signoz/signoz:${VERSION:-v0.116.1}
container_name: signoz
ports:
- "8080:8080" # signoz port

File diff suppressed because it is too large Load Diff

View File

@@ -123,7 +123,6 @@ if err := router.Handle("/api/v1/things", handler.New(
Description: "This endpoint creates a thing",
Request: new(types.PostableThing),
RequestContentType: "application/json",
RequestQuery: new(types.QueryableThing),
Response: new(types.GettableThing),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
@@ -156,8 +155,6 @@ The `handler.New` function ties the HTTP handler to OpenAPI metadata via `OpenAP
- **Request / RequestContentType**:
- `Request` is a Go type that describes the request body or form.
- `RequestContentType` is usually `"application/json"` or `"application/x-www-form-urlencoded"` (for callbacks like SAML).
- **RequestQuery**:
- `RequestQuery` is a Go type that descirbes query url params.
- **RequestExamples**: An array of `handler.OpenAPIExample` that provide concrete request payloads in the generated spec. See [Adding request examples](#adding-request-examples) below.
- **Response / ResponseContentType**:
- `Response` is the Go type for the successful response payload.

View File

@@ -193,7 +193,6 @@ uv run pytest --basetemp=./tmp/ -vv --reuse src/passwordauthn/01_register.py::te
Tests can be configured using pytest options:
- `--sqlstore-provider` - Choose database provider (default: postgres)
- `--sqlite-mode` - SQLite journal mode: `delete` or `wal` (default: delete). Only relevant when `--sqlstore-provider=sqlite`.
- `--postgres-version` - PostgreSQL version (default: 15)
- `--clickhouse-version` - ClickHouse version (default: 25.5.6)
- `--zookeeper-version` - Zookeeper version (default: 3.7.1)
@@ -203,6 +202,7 @@ Example:
uv run pytest --basetemp=./tmp/ -vv --reuse --sqlstore-provider=postgres --postgres-version=14 src/auth/
```
## What should I remember?
- **Always use the `--reuse` flag** when setting up the environment to keep containers running
@@ -213,4 +213,3 @@ uv run pytest --basetemp=./tmp/ -vv --reuse --sqlstore-provider=postgres --postg
- **Use descriptive test names** that clearly indicate what is being tested
- **Leverage fixtures** for common setup and authentication
- **Test both success and failure scenarios** to ensure robust functionality
- **`--sqlite-mode=wal` does not work on macOS.** The integration test environment runs SigNoz inside a Linux container with the SQLite database file mounted from the macOS host. WAL mode requires shared memory between connections, and connections crossing the VM boundary (macOS host ↔ Linux container) cannot share the WAL index, resulting in `SQLITE_IOERR_SHORT_READ`. WAL mode is tested in CI on Linux only.

View File

@@ -273,7 +273,6 @@ Options can be simple (direct link) or nested (with another question):
- Place logo files in `public/Logos/`
- Use SVG format
- Reference as `"/Logos/your-logo.svg"`
- **Fetching Icons**: New icons can be easily fetched from [OpenBrand](https://openbrand.sh/). Use the pattern `https://openbrand.sh/?url=<TARGET_URL>`, where `<TARGET_URL>` is the URL-encoded link to the service's website. For example, to get Render's logo, use [https://openbrand.sh/?url=https%3A%2F%2Frender.com](https://openbrand.sh/?url=https%3A%2F%2Frender.com).
- **Optimize new SVGs**: Run any newly downloaded SVGs through an optimizer like [SVGOMG (svgo)](https://svgomg.net/) or use `npx svgo public/Logos/your-logo.svg` to minimise their size before committing.
### 4. Links

View File

@@ -16,7 +16,7 @@ func (hp *HourlyProvider) GetBaseSeasonalProvider() *BaseSeasonalProvider {
return &hp.BaseSeasonalProvider
}
// NewHourlyProvider now uses the generic option type.
// NewHourlyProvider now uses the generic option type
func NewHourlyProvider(opts ...GenericProviderOption[*HourlyProvider]) *HourlyProvider {
hp := &HourlyProvider{
BaseSeasonalProvider: BaseSeasonalProvider{},

View File

@@ -32,7 +32,7 @@ func (s Seasonality) IsValid() bool {
}
type AnomaliesRequest struct {
Params *qbtypes.QueryRangeRequest
Params qbtypes.QueryRangeRequest
Seasonality Seasonality
}
@@ -47,7 +47,7 @@ type AnomaliesResponse struct {
// | |
// (rounded value for past peiod) + (seasonal growth)
//
// score = abs(value - prediction) / stddev (current_season_query).
// score = abs(value - prediction) / stddev (current_season_query)
type anomalyQueryParams struct {
// CurrentPeriodQuery is the query range params for period user is looking at or eval window
// Example: (now-5m, now), (now-30m, now), (now-1h, now)
@@ -81,7 +81,7 @@ type anomalyQueryParams struct {
Past3SeasonQuery qbtypes.QueryRangeRequest
}
func prepareAnomalyQueryParams(req *qbtypes.QueryRangeRequest, seasonality Seasonality) *anomalyQueryParams {
func prepareAnomalyQueryParams(req qbtypes.QueryRangeRequest, seasonality Seasonality) *anomalyQueryParams {
start := req.Start
end := req.End

View File

@@ -18,12 +18,12 @@ var (
movingAvgWindowSize = 7
)
// BaseProvider is an interface that includes common methods for all provider types.
// BaseProvider is an interface that includes common methods for all provider types
type BaseProvider interface {
GetBaseSeasonalProvider() *BaseSeasonalProvider
}
// GenericProviderOption is a generic type for provider options.
// GenericProviderOption is a generic type for provider options
type GenericProviderOption[T BaseProvider] func(T)
func WithQuerier[T BaseProvider](querier querier.Querier) GenericProviderOption[T] {
@@ -121,7 +121,7 @@ func (p *BaseSeasonalProvider) getResults(ctx context.Context, orgID valuer.UUID
}
// getMatchingSeries gets the matching series from the query result
// for the given series.
// for the given series
func (p *BaseSeasonalProvider) getMatchingSeries(_ context.Context, queryResult *qbtypes.TimeSeriesData, series *qbtypes.TimeSeries) *qbtypes.TimeSeries {
if queryResult == nil || len(queryResult.Aggregations) == 0 || len(queryResult.Aggregations[0].Series) == 0 {
return nil
@@ -155,14 +155,13 @@ func (p *BaseSeasonalProvider) getStdDev(series *qbtypes.TimeSeries) float64 {
avg := p.getAvg(series)
var sum float64
for _, smpl := range series.Values {
d := smpl.Value - avg
sum += d * d
sum += math.Pow(smpl.Value-avg, 2)
}
return math.Sqrt(sum / float64(len(series.Values)))
}
// getMovingAvg gets the moving average for the given series
// for the given window size and start index.
// for the given window size and start index
func (p *BaseSeasonalProvider) getMovingAvg(series *qbtypes.TimeSeries, movingAvgWindowSize, startIdx int) float64 {
if series == nil || len(series.Values) == 0 {
return 0
@@ -237,7 +236,7 @@ func (p *BaseSeasonalProvider) getPredictedSeries(
// getBounds gets the upper and lower bounds for the given series
// for the given z score threshold
// moving avg of the previous period series + z score threshold * std dev of the series
// moving avg of the previous period series - z score threshold * std dev of the series.
// moving avg of the previous period series - z score threshold * std dev of the series
func (p *BaseSeasonalProvider) getBounds(
series, predictedSeries, weekSeries *qbtypes.TimeSeries,
zScoreThreshold float64,
@@ -270,7 +269,7 @@ func (p *BaseSeasonalProvider) getBounds(
// getExpectedValue gets the expected value for the given series
// for the given index
// prevSeriesAvg + currentSeasonSeriesAvg - mean of past season series, past2 season series and past3 season series.
// prevSeriesAvg + currentSeasonSeriesAvg - mean of past season series, past2 season series and past3 season series
func (p *BaseSeasonalProvider) getExpectedValue(
_, prevSeries, currentSeasonSeries, pastSeasonSeries, past2SeasonSeries, past3SeasonSeries *qbtypes.TimeSeries, idx int,
) float64 {
@@ -284,7 +283,7 @@ func (p *BaseSeasonalProvider) getExpectedValue(
// getScore gets the anomaly score for the given series
// for the given index
// (value - expectedValue) / std dev of the series.
// (value - expectedValue) / std dev of the series
func (p *BaseSeasonalProvider) getScore(
series, prevSeries, weekSeries, weekPrevSeries, past2SeasonSeries, past3SeasonSeries *qbtypes.TimeSeries, value float64, idx int,
) float64 {
@@ -297,7 +296,7 @@ func (p *BaseSeasonalProvider) getScore(
// getAnomalyScores gets the anomaly scores for the given series
// for the given index
// (value - expectedValue) / std dev of the series.
// (value - expectedValue) / std dev of the series
func (p *BaseSeasonalProvider) getAnomalyScores(
series, prevSeries, currentSeasonSeries, pastSeasonSeries, past2SeasonSeries, past3SeasonSeries *qbtypes.TimeSeries,
) *qbtypes.TimeSeries {

View File

@@ -1,143 +0,0 @@
package otlphttpauditor
import (
"bytes"
"context"
"io"
"log/slog"
"net/http"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/audittypes"
collogspb "go.opentelemetry.io/proto/otlp/collector/logs/v1"
"google.golang.org/protobuf/proto"
spb "google.golang.org/genproto/googleapis/rpc/status"
)
const (
maxHTTPResponseReadBytes int64 = 64 * 1024
protobufContentType string = "application/x-protobuf"
)
func (provider *provider) export(ctx context.Context, events []audittypes.AuditEvent) error {
logs := audittypes.NewPLogsFromAuditEvents(events, "signoz", provider.build.Version(), "signoz.audit")
request, err := provider.marshaler.MarshalLogs(logs)
if err != nil {
return errors.Wrapf(err, errors.TypeInternal, auditor.ErrCodeAuditExportFailed, "failed to marshal audit logs")
}
if err := provider.send(ctx, request); err != nil {
provider.settings.Logger().ErrorContext(ctx, "audit export failed", errors.Attr(err), slog.Int("dropped_log_records", len(events)))
return err
}
return nil
}
// Posts a protobuf-encoded OTLP request to the configured endpoint.
// Retries are handled by the underlying heimdall HTTP client.
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlphttpexporter/otlp.go
func (provider *provider) send(ctx context.Context, body []byte) error {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.config.OTLPHTTP.Endpoint.String(), bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", protobufContentType)
res, err := provider.httpClient.Do(req)
if err != nil {
return err
}
defer func() {
_, _ = io.CopyN(io.Discard, res.Body, maxHTTPResponseReadBytes)
res.Body.Close()
}()
if res.StatusCode >= 200 && res.StatusCode <= 299 {
provider.onSuccess(ctx, res)
return nil
}
return provider.onErr(res)
}
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/01b07fcbb7a253bd996c290dcae6166e71d13732/exporter/otlphttpexporter/otlp.go#L403.
func (provider *provider) onSuccess(ctx context.Context, res *http.Response) {
resBytes, err := readResponseBody(res)
if err != nil || resBytes == nil {
return
}
exportResponse := &collogspb.ExportLogsServiceResponse{}
if err := proto.Unmarshal(resBytes, exportResponse); err != nil {
return
}
ps := exportResponse.GetPartialSuccess()
if ps == nil {
return
}
if ps.GetErrorMessage() != "" || ps.GetRejectedLogRecords() != 0 {
provider.settings.Logger().WarnContext(ctx, "partial success response", slog.String("message", ps.GetErrorMessage()), slog.Int64("dropped_log_records", ps.GetRejectedLogRecords()))
}
}
func (provider *provider) onErr(res *http.Response) error {
status := readResponseStatus(res)
if status != nil {
return errors.Newf(errors.TypeInternal, auditor.ErrCodeAuditExportFailed, "request to %s responded with status code %d, Message=%s, Details=%v", provider.config.OTLPHTTP.Endpoint.String(), res.StatusCode, status.Message, status.Details)
}
return errors.Newf(errors.TypeInternal, auditor.ErrCodeAuditExportFailed, "request to %s responded with status code %d", provider.config.OTLPHTTP.Endpoint.String(), res.StatusCode)
}
// Reads at most maxHTTPResponseReadBytes from the response body.
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/01b07fcbb7a253bd996c290dcae6166e71d13732/exporter/otlphttpexporter/otlp.go#L275.
func readResponseBody(resp *http.Response) ([]byte, error) {
if resp.ContentLength == 0 {
return nil, nil
}
maxRead := resp.ContentLength
if maxRead == -1 || maxRead > maxHTTPResponseReadBytes {
maxRead = maxHTTPResponseReadBytes
}
protoBytes := make([]byte, maxRead)
n, err := io.ReadFull(resp.Body, protoBytes)
if n == 0 && (err == nil || errors.Is(err, io.EOF)) {
return nil, nil
}
if err != nil && !errors.Is(err, io.ErrUnexpectedEOF) {
return nil, err
}
return protoBytes[:n], nil
}
// Decodes a protobuf-encoded Status from 4xx/5xx response bodies. Returns nil if the response is empty or cannot be decoded.
// Ref: https://github.com/open-telemetry/opentelemetry-collector/blob/01b07fcbb7a253bd996c290dcae6166e71d13732/exporter/otlphttpexporter/otlp.go#L310.
func readResponseStatus(resp *http.Response) *spb.Status {
if resp.StatusCode < 400 || resp.StatusCode > 599 {
return nil
}
respBytes, err := readResponseBody(resp)
if err != nil || respBytes == nil {
return nil
}
respStatus := &spb.Status{}
if err := proto.Unmarshal(respBytes, respStatus); err != nil {
return nil
}
return respStatus
}

View File

@@ -1,97 +0,0 @@
package otlphttpauditor
import (
"context"
"github.com/SigNoz/signoz/pkg/auditor"
"github.com/SigNoz/signoz/pkg/auditor/auditorserver"
"github.com/SigNoz/signoz/pkg/factory"
client "github.com/SigNoz/signoz/pkg/http/client"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/types/audittypes"
"github.com/SigNoz/signoz/pkg/version"
"go.opentelemetry.io/collector/pdata/plog"
)
var _ auditor.Auditor = (*provider)(nil)
type provider struct {
settings factory.ScopedProviderSettings
config auditor.Config
licensing licensing.Licensing
build version.Build
server *auditorserver.Server
marshaler plog.ProtoMarshaler
httpClient *client.Client
}
func NewFactory(licensing licensing.Licensing, build version.Build) factory.ProviderFactory[auditor.Auditor, auditor.Config] {
return factory.NewProviderFactory(factory.MustNewName("otlphttp"), func(ctx context.Context, providerSettings factory.ProviderSettings, config auditor.Config) (auditor.Auditor, error) {
return newProvider(ctx, providerSettings, config, licensing, build)
})
}
func newProvider(_ context.Context, providerSettings factory.ProviderSettings, config auditor.Config, licensing licensing.Licensing, build version.Build) (auditor.Auditor, error) {
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/ee/auditor/otlphttpauditor")
httpClient, err := client.New(
settings.Logger(),
providerSettings.TracerProvider,
providerSettings.MeterProvider,
client.WithTimeout(config.OTLPHTTP.Timeout),
client.WithRetryCount(retryCountFromConfig(config.OTLPHTTP.Retry)),
retrierOption(config.OTLPHTTP.Retry),
)
if err != nil {
return nil, err
}
provider := &provider{
settings: settings,
config: config,
licensing: licensing,
build: build,
marshaler: plog.ProtoMarshaler{},
httpClient: httpClient,
}
server, err := auditorserver.New(settings,
auditorserver.Config{
BufferSize: config.BufferSize,
BatchSize: config.BatchSize,
FlushInterval: config.FlushInterval,
},
provider.export,
)
if err != nil {
return nil, err
}
provider.server = server
return provider, nil
}
func (provider *provider) Start(ctx context.Context) error {
return provider.server.Start(ctx)
}
func (provider *provider) Audit(ctx context.Context, event audittypes.AuditEvent) {
if event.PrincipalAttributes.PrincipalOrgID.IsZero() {
provider.settings.Logger().WarnContext(ctx, "audit event dropped as org_id is zero")
return
}
if _, err := provider.licensing.GetActive(ctx, event.PrincipalAttributes.PrincipalOrgID); err != nil {
return
}
provider.server.Add(ctx, event)
}
func (provider *provider) Healthy() <-chan struct{} {
return provider.server.Healthy()
}
func (provider *provider) Stop(ctx context.Context) error {
return provider.server.Stop(ctx)
}

View File

@@ -1,52 +0,0 @@
package otlphttpauditor
import (
"time"
"github.com/SigNoz/signoz/pkg/auditor"
client "github.com/SigNoz/signoz/pkg/http/client"
)
// retrier implements client.Retriable with exponential backoff
// derived from auditor.RetryConfig.
type retrier struct {
initialInterval time.Duration
maxInterval time.Duration
}
func newRetrier(cfg auditor.RetryConfig) *retrier {
return &retrier{
initialInterval: cfg.InitialInterval,
maxInterval: cfg.MaxInterval,
}
}
// NextInterval returns the backoff duration for the given retry attempt.
// Uses exponential backoff: initialInterval * 2^retry, capped at maxInterval.
func (r *retrier) NextInterval(retry int) time.Duration {
interval := r.initialInterval
for range retry {
interval *= 2
}
return min(interval, r.maxInterval)
}
func retrierOption(cfg auditor.RetryConfig) client.Option {
return client.WithRetriable(newRetrier(cfg))
}
func retryCountFromConfig(cfg auditor.RetryConfig) int {
if !cfg.Enabled || cfg.MaxElapsedTime <= 0 {
return 0
}
count := 0
elapsed := time.Duration(0)
interval := cfg.InitialInterval
for elapsed < cfg.MaxElapsedTime {
elapsed += interval
interval = min(interval*2, cfg.MaxInterval)
count++
}
return count
}

View File

@@ -16,7 +16,6 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
openfgav1 "github.com/openfga/api/proto/openfga/v1"
openfgapkgtransformer "github.com/openfga/language/pkg/go/transformer"
"github.com/openfga/openfga/pkg/storage"
)
type provider struct {
@@ -27,14 +26,14 @@ type provider struct {
registry []authz.RegisterTypeable
}
func NewProviderFactory(sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, openfgaDataStore storage.OpenFGADatastore, licensing licensing.Licensing, registry ...authz.RegisterTypeable) factory.ProviderFactory[authz.AuthZ, authz.Config] {
func NewProviderFactory(sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, licensing licensing.Licensing, registry ...authz.RegisterTypeable) factory.ProviderFactory[authz.AuthZ, authz.Config] {
return factory.NewProviderFactory(factory.MustNewName("openfga"), func(ctx context.Context, ps factory.ProviderSettings, config authz.Config) (authz.AuthZ, error) {
return newOpenfgaProvider(ctx, ps, config, sqlstore, openfgaSchema, openfgaDataStore, licensing, registry)
return newOpenfgaProvider(ctx, ps, config, sqlstore, openfgaSchema, licensing, registry)
})
}
func newOpenfgaProvider(ctx context.Context, settings factory.ProviderSettings, config authz.Config, sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, openfgaDataStore storage.OpenFGADatastore, licensing licensing.Licensing, registry []authz.RegisterTypeable) (authz.AuthZ, error) {
pkgOpenfgaAuthzProvider := pkgopenfgaauthz.NewProviderFactory(sqlstore, openfgaSchema, openfgaDataStore)
func newOpenfgaProvider(ctx context.Context, settings factory.ProviderSettings, config authz.Config, sqlstore sqlstore.SQLStore, openfgaSchema []openfgapkgtransformer.ModuleFile, licensing licensing.Licensing, registry []authz.RegisterTypeable) (authz.AuthZ, error) {
pkgOpenfgaAuthzProvider := pkgopenfgaauthz.NewProviderFactory(sqlstore, openfgaSchema)
pkgAuthzService, err := pkgOpenfgaAuthzProvider.New(ctx, settings, config)
if err != nil {
return nil, err
@@ -58,10 +57,6 @@ func (provider *provider) Start(ctx context.Context) error {
return provider.openfgaServer.Start(ctx)
}
func (provider *provider) Healthy() <-chan struct{} {
return provider.openfgaServer.Healthy()
}
func (provider *provider) Stop(ctx context.Context) error {
return provider.openfgaServer.Stop(ctx)
}

View File

@@ -16,6 +16,7 @@ type Server struct {
}
func NewOpenfgaServer(ctx context.Context, pkgAuthzService authz.AuthZ) (*Server, error) {
return &Server{
pkgAuthzService: pkgAuthzService,
}, nil
@@ -25,31 +26,14 @@ func (server *Server) Start(ctx context.Context) error {
return server.pkgAuthzService.Start(ctx)
}
func (server *Server) Healthy() <-chan struct{} {
return server.pkgAuthzService.Healthy()
}
func (server *Server) Stop(ctx context.Context) error {
return server.pkgAuthzService.Stop(ctx)
}
func (server *Server) CheckWithTupleCreation(ctx context.Context, claims authtypes.Claims, orgID valuer.UUID, relation authtypes.Relation, typeable authtypes.Typeable, selectors []authtypes.Selector, _ []authtypes.Selector) error {
subject := ""
switch claims.Principal {
case authtypes.PrincipalUser:
user, err := authtypes.NewSubject(authtypes.TypeableUser, claims.UserID, orgID, nil)
if err != nil {
return err
}
subject = user
case authtypes.PrincipalServiceAccount:
serviceAccount, err := authtypes.NewSubject(authtypes.TypeableServiceAccount, claims.ServiceAccountID, orgID, nil)
if err != nil {
return err
}
subject = serviceAccount
subject, err := authtypes.NewSubject(authtypes.TypeableUser, claims.UserID, orgID, nil)
if err != nil {
return err
}
tupleSlice, err := typeable.Tuples(subject, relation, selectors, orgID)

View File

@@ -1,32 +0,0 @@
package openfgaserver
import (
"github.com/SigNoz/signoz/ee/sqlstore/postgressqlstore"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/openfga/openfga/pkg/storage"
"github.com/openfga/openfga/pkg/storage/postgres"
"github.com/openfga/openfga/pkg/storage/sqlcommon"
"github.com/openfga/openfga/pkg/storage/sqlite"
)
func NewSQLStore(store sqlstore.SQLStore) (storage.OpenFGADatastore, error) {
switch store.BunDB().Dialect().Name().String() {
case "sqlite":
return sqlite.NewWithDB(store.SQLDB(), &sqlcommon.Config{
MaxTuplesPerWriteField: 100,
MaxTypesPerModelField: 100,
})
case "pg":
pgStore, ok := store.(postgressqlstore.Pooler)
if !ok {
panic(errors.New(errors.TypeInternal, errors.CodeInternal, "postgressqlstore should implement Pooler"))
}
return postgres.NewWithDB(pgStore.Pool(), nil, &sqlcommon.Config{
MaxTuplesPerWriteField: 100,
MaxTypesPerModelField: 100,
})
}
return nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid store type: %s", store.BunDB().Dialect().Name().String())
}

View File

@@ -13,7 +13,7 @@ var (
once sync.Once
)
// Config initializes the licensing configuration.
// initializes the licensing configuration
func Config(pollInterval time.Duration, failureThreshold int) licensing.Config {
once.Do(func() {
config = licensing.Config{PollInterval: pollInterval, FailureThreshold: failureThreshold}

View File

@@ -1,132 +0,0 @@
package implcloudprovider
import (
"context"
"fmt"
"net/url"
"sort"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
)
type awscloudprovider struct {
serviceDefinitions cloudintegrationtypes.ServiceDefinitionStore
}
func NewAWSCloudProvider(defStore cloudintegrationtypes.ServiceDefinitionStore) (cloudintegration.CloudProviderModule, error) {
return &awscloudprovider{serviceDefinitions: defStore}, nil
}
func (provider *awscloudprovider) GetConnectionArtifact(ctx context.Context, account *cloudintegrationtypes.Account, req *cloudintegrationtypes.GetConnectionArtifactRequest) (*cloudintegrationtypes.ConnectionArtifact, error) {
baseURL := fmt.Sprintf(cloudintegrationtypes.CloudFormationQuickCreateBaseURL.StringValue(), req.Config.Aws.DeploymentRegion)
u, _ := url.Parse(baseURL)
q := u.Query()
q.Set("region", req.Config.Aws.DeploymentRegion)
u.Fragment = "/stacks/quickcreate"
u.RawQuery = q.Encode()
q = u.Query()
q.Set("stackName", cloudintegrationtypes.AgentCloudFormationBaseStackName.StringValue())
q.Set("templateURL", fmt.Sprintf(cloudintegrationtypes.AgentCloudFormationTemplateS3Path.StringValue(), req.Config.AgentVersion))
q.Set("param_SigNozIntegrationAgentVersion", req.Config.AgentVersion)
q.Set("param_SigNozApiUrl", req.Credentials.SigNozAPIURL)
q.Set("param_SigNozApiKey", req.Credentials.SigNozAPIKey)
q.Set("param_SigNozAccountId", account.ID.StringValue())
q.Set("param_IngestionUrl", req.Credentials.IngestionURL)
q.Set("param_IngestionKey", req.Credentials.IngestionKey)
return &cloudintegrationtypes.ConnectionArtifact{
Aws: &cloudintegrationtypes.AWSConnectionArtifact{
ConnectionURL: u.String() + "?&" + q.Encode(), // this format is required by AWS
},
}, nil
}
func (provider *awscloudprovider) ListServiceDefinitions(ctx context.Context) ([]*cloudintegrationtypes.ServiceDefinition, error) {
return provider.serviceDefinitions.List(ctx, cloudintegrationtypes.CloudProviderTypeAWS)
}
func (provider *awscloudprovider) GetServiceDefinition(ctx context.Context, serviceID cloudintegrationtypes.ServiceID) (*cloudintegrationtypes.ServiceDefinition, error) {
serviceDef, err := provider.serviceDefinitions.Get(ctx, cloudintegrationtypes.CloudProviderTypeAWS, serviceID)
if err != nil {
return nil, err
}
// override cloud integration dashboard id
for index, dashboard := range serviceDef.Assets.Dashboards {
serviceDef.Assets.Dashboards[index].ID = cloudintegrationtypes.GetCloudIntegrationDashboardID(cloudintegrationtypes.CloudProviderTypeAWS, serviceID.StringValue(), dashboard.ID)
}
return serviceDef, nil
}
func (provider *awscloudprovider) BuildIntegrationConfig(
ctx context.Context,
account *cloudintegrationtypes.Account,
services []*cloudintegrationtypes.StorableCloudIntegrationService,
) (*cloudintegrationtypes.ProviderIntegrationConfig, error) {
// Sort services for deterministic output
sort.Slice(services, func(i, j int) bool {
return services[i].Type.StringValue() < services[j].Type.StringValue()
})
compiledMetrics := new(cloudintegrationtypes.AWSMetricsCollectionStrategy)
compiledLogs := new(cloudintegrationtypes.AWSLogsCollectionStrategy)
var compiledS3Buckets map[string][]string
for _, storedSvc := range services {
svcCfg, err := cloudintegrationtypes.NewServiceConfigFromJSON(cloudintegrationtypes.CloudProviderTypeAWS, storedSvc.Config)
if err != nil {
return nil, err
}
svcDef, err := provider.GetServiceDefinition(ctx, storedSvc.Type)
if err != nil {
return nil, err
}
strategy := svcDef.TelemetryCollectionStrategy.AWS
logsEnabled := svcCfg.IsLogsEnabled(cloudintegrationtypes.CloudProviderTypeAWS)
// S3Sync: logs come directly from configured S3 buckets, not CloudWatch subscriptions
if storedSvc.Type == cloudintegrationtypes.AWSServiceS3Sync {
if logsEnabled && svcCfg.AWS.Logs.S3Buckets != nil {
compiledS3Buckets = svcCfg.AWS.Logs.S3Buckets
}
// no need to go ahead as the code block specifically checks for the S3Sync service
continue
}
if logsEnabled && strategy.Logs != nil {
compiledLogs.Subscriptions = append(compiledLogs.Subscriptions, strategy.Logs.Subscriptions...)
}
metricsEnabled := svcCfg.IsMetricsEnabled(cloudintegrationtypes.CloudProviderTypeAWS)
if metricsEnabled && strategy.Metrics != nil {
compiledMetrics.StreamFilters = append(compiledMetrics.StreamFilters, strategy.Metrics.StreamFilters...)
}
}
collectionStrategy := new(cloudintegrationtypes.AWSTelemetryCollectionStrategy)
if len(compiledMetrics.StreamFilters) > 0 {
collectionStrategy.Metrics = compiledMetrics
}
if len(compiledLogs.Subscriptions) > 0 {
collectionStrategy.Logs = compiledLogs
}
if compiledS3Buckets != nil {
collectionStrategy.S3Buckets = compiledS3Buckets
}
return &cloudintegrationtypes.ProviderIntegrationConfig{
AWS: &cloudintegrationtypes.AWSIntegrationConfig{
EnabledRegions: account.Config.AWS.Regions,
TelemetryCollectionStrategy: collectionStrategy,
},
}, nil
}

View File

@@ -1,34 +0,0 @@
package implcloudprovider
import (
"context"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
)
type azurecloudprovider struct{}
func NewAzureCloudProvider() cloudintegration.CloudProviderModule {
return &azurecloudprovider{}
}
func (provider *azurecloudprovider) GetConnectionArtifact(ctx context.Context, account *cloudintegrationtypes.Account, req *cloudintegrationtypes.GetConnectionArtifactRequest) (*cloudintegrationtypes.ConnectionArtifact, error) {
panic("implement me")
}
func (provider *azurecloudprovider) ListServiceDefinitions(ctx context.Context) ([]*cloudintegrationtypes.ServiceDefinition, error) {
panic("implement me")
}
func (provider *azurecloudprovider) GetServiceDefinition(ctx context.Context, serviceID cloudintegrationtypes.ServiceID) (*cloudintegrationtypes.ServiceDefinition, error) {
panic("implement me")
}
func (provider *azurecloudprovider) BuildIntegrationConfig(
ctx context.Context,
account *cloudintegrationtypes.Account,
services []*cloudintegrationtypes.StorableCloudIntegrationService,
) (*cloudintegrationtypes.ProviderIntegrationConfig, error) {
panic("implement me")
}

View File

@@ -1,540 +0,0 @@
package implcloudintegration
import (
"context"
"fmt"
"sort"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/gateway"
"github.com/SigNoz/signoz/pkg/global"
"github.com/SigNoz/signoz/pkg/licensing"
"github.com/SigNoz/signoz/pkg/modules/cloudintegration"
"github.com/SigNoz/signoz/pkg/modules/serviceaccount"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/cloudintegrationtypes"
"github.com/SigNoz/signoz/pkg/types/dashboardtypes"
"github.com/SigNoz/signoz/pkg/types/serviceaccounttypes"
"github.com/SigNoz/signoz/pkg/types/zeustypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/SigNoz/signoz/pkg/zeus"
)
type module struct {
store cloudintegrationtypes.Store
gateway gateway.Gateway
zeus zeus.Zeus
licensing licensing.Licensing
global global.Global
serviceAccount serviceaccount.Module
cloudProvidersMap map[cloudintegrationtypes.CloudProviderType]cloudintegration.CloudProviderModule
config cloudintegration.Config
}
func NewModule(
store cloudintegrationtypes.Store,
global global.Global,
zeus zeus.Zeus,
gateway gateway.Gateway,
licensing licensing.Licensing,
serviceAccount serviceaccount.Module,
cloudProvidersMap map[cloudintegrationtypes.CloudProviderType]cloudintegration.CloudProviderModule,
config cloudintegration.Config,
) (cloudintegration.Module, error) {
return &module{
store: store,
global: global,
zeus: zeus,
gateway: gateway,
licensing: licensing,
serviceAccount: serviceAccount,
cloudProvidersMap: cloudProvidersMap,
config: config,
}, nil
}
// GetConnectionCredentials returns credentials required to generate connection artifact. eg. apiKey, ingestionKey etc.
// It will return creds it can deduce and return empty value for others.
func (module *module) GetConnectionCredentials(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (*cloudintegrationtypes.Credentials, error) {
// get license to get the deployment details
license, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
// get deployment details from zeus, ignore error
respBytes, _ := module.zeus.GetDeployment(ctx, license.Key)
var signozAPIURL string
if len(respBytes) > 0 {
// parse deployment details, ignore error, if client is asking api url every time check for possible error
deployment, _ := zeustypes.NewGettableDeployment(respBytes)
if deployment != nil {
signozAPIURL, _ = cloudintegrationtypes.GetSigNozAPIURLFromDeployment(deployment)
}
}
// ignore error
apiKey, _ := module.getOrCreateAPIKey(ctx, orgID, provider)
// ignore error
ingestionKey, _ := module.getOrCreateIngestionKey(ctx, orgID, provider)
return cloudintegrationtypes.NewCredentials(
signozAPIURL,
apiKey,
module.global.GetConfig(ctx).IngestionURL,
ingestionKey,
), nil
}
func (module *module) CreateAccount(ctx context.Context, account *cloudintegrationtypes.Account) error {
_, err := module.licensing.GetActive(ctx, account.OrgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableCloudIntegration, err := cloudintegrationtypes.NewStorableCloudIntegration(account)
if err != nil {
return err
}
return module.store.CreateAccount(ctx, storableCloudIntegration)
}
func (module *module) GetConnectionArtifact(ctx context.Context, account *cloudintegrationtypes.Account, req *cloudintegrationtypes.GetConnectionArtifactRequest) (*cloudintegrationtypes.ConnectionArtifact, error) {
_, err := module.licensing.GetActive(ctx, account.OrgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProviderModule, err := module.getCloudProvider(account.Provider)
if err != nil {
return nil, err
}
req.Config.SetAgentVersion(module.config.Agent.Version)
return cloudProviderModule.GetConnectionArtifact(ctx, account, req)
}
func (module *module) GetAccount(ctx context.Context, orgID valuer.UUID, accountID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (*cloudintegrationtypes.Account, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccount, err := module.store.GetAccountByID(ctx, orgID, accountID, provider)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAccountFromStorable(storableAccount)
}
func (module *module) GetConnectedAccount(ctx context.Context, orgID, accountID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (*cloudintegrationtypes.Account, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccount, err := module.store.GetConnectedAccount(ctx, orgID, accountID, provider)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAccountFromStorable(storableAccount)
}
// ListAccounts return only agent connected accounts.
func (module *module) ListAccounts(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) ([]*cloudintegrationtypes.Account, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccounts, err := module.store.ListConnectedAccounts(ctx, orgID, provider)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAccountsFromStorables(storableAccounts)
}
func (module *module) AgentCheckIn(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType, req *cloudintegrationtypes.AgentCheckInRequest) (*cloudintegrationtypes.AgentCheckInResponse, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
connectedAccount, err := module.store.GetConnectedAccountByProviderAccountID(ctx, orgID, req.ProviderAccountID, provider)
if err != nil && !errors.Ast(err, errors.TypeNotFound) {
return nil, err
}
// If a different integration is already connected to this provider account ID, reject the check-in.
// Allow re-check-in from the same integration (e.g. agent restarting).
if connectedAccount != nil && connectedAccount.ID != req.CloudIntegrationID {
errMessage := fmt.Sprintf("provider account id %s is already connected to cloud integration id %s", req.ProviderAccountID, connectedAccount.ID)
return nil, errors.New(errors.TypeAlreadyExists, cloudintegrationtypes.ErrCodeCloudIntegrationAlreadyConnected, errMessage)
}
account, err := module.store.GetAccountByID(ctx, orgID, req.CloudIntegrationID, provider)
if err != nil {
return nil, err
}
// If account has been removed (disconnected), return a minimal response with empty integration config.
// The agent uses this response to clean up resources
if account.RemovedAt != nil {
return cloudintegrationtypes.NewAgentCheckInResponse(
req.ProviderAccountID,
account.ID.StringValue(),
new(cloudintegrationtypes.ProviderIntegrationConfig),
account.RemovedAt,
), nil
}
// update account with cloud provider account id and agent report (heartbeat)
account.Update(&req.ProviderAccountID, cloudintegrationtypes.NewAgentReport(req.Data))
err = module.store.UpdateAccount(ctx, account)
if err != nil {
return nil, err
}
// Get account as domain object for config access (enabled regions, etc.)
domainAccount, err := cloudintegrationtypes.NewAccountFromStorable(account)
if err != nil {
return nil, err
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
storedServices, err := module.store.ListServices(ctx, req.CloudIntegrationID)
if err != nil {
return nil, err
}
// Delegate integration config building entirely to the provider module
integrationConfig, err := cloudProvider.BuildIntegrationConfig(ctx, domainAccount, storedServices)
if err != nil {
return nil, err
}
return cloudintegrationtypes.NewAgentCheckInResponse(
req.ProviderAccountID,
account.ID.StringValue(),
integrationConfig,
account.RemovedAt,
), nil
}
func (module *module) UpdateAccount(ctx context.Context, account *cloudintegrationtypes.Account) error {
_, err := module.licensing.GetActive(ctx, account.OrgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
storableAccount, err := cloudintegrationtypes.NewStorableCloudIntegration(account)
if err != nil {
return err
}
return module.store.UpdateAccount(ctx, storableAccount)
}
func (module *module) DisconnectAccount(ctx context.Context, orgID valuer.UUID, accountID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) error {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
return module.store.RemoveAccount(ctx, orgID, accountID, provider)
}
func (module *module) ListServicesMetadata(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType, integrationID valuer.UUID) ([]*cloudintegrationtypes.ServiceMetadata, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
serviceDefinitions, err := cloudProvider.ListServiceDefinitions(ctx)
if err != nil {
return nil, err
}
enabledServiceIDs := map[string]bool{}
if !integrationID.IsZero() {
storedServices, err := module.store.ListServices(ctx, integrationID)
if err != nil {
return nil, err
}
for _, svc := range storedServices {
serviceConfig, err := cloudintegrationtypes.NewServiceConfigFromJSON(provider, svc.Config)
if err != nil {
return nil, err
}
if serviceConfig.IsServiceEnabled(provider) {
enabledServiceIDs[svc.Type.StringValue()] = true
}
}
}
resp := make([]*cloudintegrationtypes.ServiceMetadata, 0, len(serviceDefinitions))
for _, serviceDefinition := range serviceDefinitions {
resp = append(resp, cloudintegrationtypes.NewServiceMetadata(*serviceDefinition, enabledServiceIDs[serviceDefinition.ID]))
}
return resp, nil
}
func (module *module) GetService(ctx context.Context, orgID valuer.UUID, serviceID cloudintegrationtypes.ServiceID, provider cloudintegrationtypes.CloudProviderType, cloudIntegrationID valuer.UUID) (*cloudintegrationtypes.Service, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
serviceDefinition, err := cloudProvider.GetServiceDefinition(ctx, serviceID)
if err != nil {
return nil, err
}
var integrationService *cloudintegrationtypes.CloudIntegrationService
if !cloudIntegrationID.IsZero() {
storedService, err := module.store.GetServiceByServiceID(ctx, cloudIntegrationID, serviceID)
if err != nil && !errors.Ast(err, errors.TypeNotFound) {
return nil, err
}
if storedService != nil {
serviceConfig, err := cloudintegrationtypes.NewServiceConfigFromJSON(provider, storedService.Config)
if err != nil {
return nil, err
}
integrationService = cloudintegrationtypes.NewCloudIntegrationServiceFromStorable(storedService, serviceConfig)
}
}
return cloudintegrationtypes.NewService(*serviceDefinition, integrationService), nil
}
func (module *module) CreateService(ctx context.Context, orgID valuer.UUID, service *cloudintegrationtypes.CloudIntegrationService, provider cloudintegrationtypes.CloudProviderType) error {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return err
}
serviceDefinition, err := cloudProvider.GetServiceDefinition(ctx, service.Type)
if err != nil {
return err
}
configJSON, err := service.Config.ToJSON(provider, service.Type, &serviceDefinition.SupportedSignals)
if err != nil {
return err
}
return module.store.CreateService(ctx, cloudintegrationtypes.NewStorableCloudIntegrationService(service, string(configJSON)))
}
func (module *module) UpdateService(ctx context.Context, orgID valuer.UUID, integrationService *cloudintegrationtypes.CloudIntegrationService, provider cloudintegrationtypes.CloudProviderType) error {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return err
}
serviceDefinition, err := cloudProvider.GetServiceDefinition(ctx, integrationService.Type)
if err != nil {
return err
}
configJSON, err := integrationService.Config.ToJSON(provider, integrationService.Type, &serviceDefinition.SupportedSignals)
if err != nil {
return err
}
storableService := cloudintegrationtypes.NewStorableCloudIntegrationService(integrationService, string(configJSON))
return module.store.UpdateService(ctx, storableService)
}
func (module *module) GetDashboardByID(ctx context.Context, orgID valuer.UUID, id string) (*dashboardtypes.Dashboard, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
_, _, _, err = cloudintegrationtypes.ParseCloudIntegrationDashboardID(id)
if err != nil {
return nil, err
}
allDashboards, err := module.listDashboards(ctx, orgID)
if err != nil {
return nil, err
}
for _, d := range allDashboards {
if d.ID == id {
return d, nil
}
}
return nil, errors.New(errors.TypeNotFound, cloudintegrationtypes.ErrCodeCloudIntegrationNotFound, "cloud integration dashboard not found")
}
func (module *module) ListDashboards(ctx context.Context, orgID valuer.UUID) ([]*dashboardtypes.Dashboard, error) {
_, err := module.licensing.GetActive(ctx, orgID)
if err != nil {
return nil, errors.New(errors.TypeLicenseUnavailable, errors.CodeLicenseUnavailable, "a valid license is not available").WithAdditional("this feature requires a valid license").WithAdditional(err.Error())
}
return module.listDashboards(ctx, orgID)
}
func (module *module) Collect(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
stats := make(map[string]any)
// get connected accounts for AWS
awsAccountsCount, err := module.store.CountConnectedAccounts(ctx, orgID, cloudintegrationtypes.CloudProviderTypeAWS)
if err == nil {
stats["cloudintegration.aws.connectedaccounts.count"] = awsAccountsCount
}
// NOTE: not adding stats for services for now.
// TODO: add more cloud providers when supported
return stats, nil
}
func (module *module) getCloudProvider(provider cloudintegrationtypes.CloudProviderType) (cloudintegration.CloudProviderModule, error) {
if cloudProviderModule, ok := module.cloudProvidersMap[provider]; ok {
return cloudProviderModule, nil
}
return nil, errors.NewInvalidInputf(cloudintegrationtypes.ErrCodeCloudProviderInvalidInput, "invalid cloud provider: %s", provider.StringValue())
}
func (module *module) getOrCreateIngestionKey(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (string, error) {
keyName := cloudintegrationtypes.NewIngestionKeyName(provider)
result, err := module.gateway.SearchIngestionKeysByName(ctx, orgID, keyName, 1, 10)
if err != nil {
return "", errors.WrapInternalf(err, errors.CodeInternal, "couldn't search ingestion keys")
}
// ideally there should be only one key per cloud integration provider
if len(result.Keys) > 0 {
return result.Keys[0].Value, nil
}
createdIngestionKey, err := module.gateway.CreateIngestionKey(ctx, orgID, keyName, []string{"integration"}, time.Time{})
if err != nil {
return "", errors.WrapInternalf(err, errors.CodeInternal, "couldn't create ingestion key")
}
return createdIngestionKey.Value, nil
}
func (module *module) getOrCreateAPIKey(ctx context.Context, orgID valuer.UUID, provider cloudintegrationtypes.CloudProviderType) (string, error) {
domain := module.serviceAccount.Config().Email.Domain
serviceAccount := serviceaccounttypes.NewServiceAccount("integration", domain, serviceaccounttypes.ServiceAccountStatusActive, orgID)
serviceAccount, err := module.serviceAccount.GetOrCreate(ctx, orgID, serviceAccount)
if err != nil {
return "", err
}
err = module.serviceAccount.SetRoleByName(ctx, orgID, serviceAccount.ID, authtypes.SigNozViewerRoleName)
if err != nil {
return "", err
}
factorAPIKey, err := serviceAccount.NewFactorAPIKey(provider.StringValue(), 0)
if err != nil {
return "", err
}
factorAPIKey, err = module.serviceAccount.GetOrCreateFactorAPIKey(ctx, factorAPIKey)
if err != nil {
return "", err
}
return factorAPIKey.Key, nil
}
func (module *module) listDashboards(ctx context.Context, orgID valuer.UUID) ([]*dashboardtypes.Dashboard, error) {
var allDashboards []*dashboardtypes.Dashboard
for provider := range module.cloudProvidersMap {
cloudProvider, err := module.getCloudProvider(provider)
if err != nil {
return nil, err
}
connectedAccounts, err := module.store.ListConnectedAccounts(ctx, orgID, provider)
if err != nil {
return nil, err
}
for _, storableAccount := range connectedAccounts {
storedServices, err := module.store.ListServices(ctx, storableAccount.ID)
if err != nil {
return nil, err
}
for _, storedSvc := range storedServices {
serviceConfig, err := cloudintegrationtypes.NewServiceConfigFromJSON(provider, storedSvc.Config)
if err != nil || !serviceConfig.IsMetricsEnabled(provider) {
continue
}
svcDef, err := cloudProvider.GetServiceDefinition(ctx, storedSvc.Type)
if err != nil || svcDef == nil {
continue
}
dashboards := cloudintegrationtypes.GetDashboardsFromAssets(
storedSvc.Type.StringValue(),
orgID,
provider,
storableAccount.CreatedAt,
svcDef.Assets,
)
allDashboards = append(allDashboards, dashboards...)
}
}
}
sort.Slice(allDashboards, func(i, j int) bool {
return allDashboards[i].ID < allDashboards[j].ID
})
return allDashboards, nil
}

View File

@@ -213,8 +213,8 @@ func (module *module) Update(ctx context.Context, orgID valuer.UUID, id valuer.U
return module.pkgDashboardModule.Update(ctx, orgID, id, updatedBy, data, diff)
}
func (module *module) LockUnlock(ctx context.Context, orgID valuer.UUID, id valuer.UUID, updatedBy string, isAdmin bool, lock bool) error {
return module.pkgDashboardModule.LockUnlock(ctx, orgID, id, updatedBy, isAdmin, lock)
func (module *module) LockUnlock(ctx context.Context, orgID valuer.UUID, id valuer.UUID, updatedBy string, role types.Role, lock bool) error {
return module.pkgDashboardModule.LockUnlock(ctx, orgID, id, updatedBy, role, lock)
}
func (module *module) MustGetTypeables() []authtypes.Typeable {

View File

@@ -65,7 +65,7 @@ func (h *handler) QueryRange(rw http.ResponseWriter, req *http.Request) {
}
if anomalyQuery, ok := queryRangeRequest.IsAnomalyRequest(); ok {
anomalies, err := h.handleAnomalyQuery(ctx, orgID, anomalyQuery, &queryRangeRequest)
anomalies, err := h.handleAnomalyQuery(ctx, orgID, anomalyQuery, queryRangeRequest)
if err != nil {
render.Error(rw, errors.NewInternalf(errors.CodeInternal, "failed to get anomalies: %v", err))
return
@@ -79,7 +79,7 @@ func (h *handler) QueryRange(rw http.ResponseWriter, req *http.Request) {
// Build step intervals from the anomaly query
stepIntervals := make(map[string]uint64)
if anomalyQuery.StepInterval.Duration > 0 {
stepIntervals[anomalyQuery.Name] = uint64(anomalyQuery.StepInterval.Seconds())
stepIntervals[anomalyQuery.Name] = uint64(anomalyQuery.StepInterval.Duration.Seconds())
}
finalResp := &qbtypes.QueryRangeResponse{
@@ -149,7 +149,7 @@ func (h *handler) createAnomalyProvider(seasonality anomalyV2.Seasonality) anoma
}
}
func (h *handler) handleAnomalyQuery(ctx context.Context, orgID valuer.UUID, anomalyQuery *qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation], queryRangeRequest *qbtypes.QueryRangeRequest) (*anomalyV2.AnomaliesResponse, error) {
func (h *handler) handleAnomalyQuery(ctx context.Context, orgID valuer.UUID, anomalyQuery *qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation], queryRangeRequest qbtypes.QueryRangeRequest) (*anomalyV2.AnomaliesResponse, error) {
seasonality := extractSeasonality(anomalyQuery)
provider := h.createAnomalyProvider(seasonality)

View File

@@ -14,9 +14,10 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/modules/user"
basemodel "github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/serviceaccounttypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/gorilla/mux"
)
@@ -49,7 +50,7 @@ func (ah *APIHandler) CloudIntegrationsGenerateConnectionParams(w http.ResponseW
return
}
apiKey, apiErr := ah.getOrCreateCloudIntegrationFactorAPIKey(r.Context(), valuer.MustNewUUID(claims.OrgID), cloudProvider)
apiKey, apiErr := ah.getOrCreateCloudIntegrationPAT(r.Context(), claims.OrgID, cloudProvider)
if apiErr != nil {
RespondError(w, basemodel.WrapApiError(
apiErr, "couldn't provision PAT for cloud integration:",
@@ -109,44 +110,84 @@ func (ah *APIHandler) CloudIntegrationsGenerateConnectionParams(w http.ResponseW
ah.Respond(w, result)
}
func (ah *APIHandler) getOrCreateCloudIntegrationFactorAPIKey(ctx context.Context, orgID valuer.UUID, cloudProvider string) (
func (ah *APIHandler) getOrCreateCloudIntegrationPAT(ctx context.Context, orgId string, cloudProvider string) (
string, *basemodel.ApiError,
) {
integrationPATName := fmt.Sprintf("%s", cloudProvider)
serviceAccount, apiErr := ah.getOrCreateCloudIntegrationServiceAccount(ctx, orgID)
integrationPATName := fmt.Sprintf("%s integration", cloudProvider)
integrationUser, apiErr := ah.getOrCreateCloudIntegrationUser(ctx, orgId, cloudProvider)
if apiErr != nil {
return "", apiErr
}
factorAPIKey, err := serviceAccount.NewFactorAPIKey(integrationPATName, 0)
orgIdUUID, err := valuer.NewUUID(orgId)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't parse orgId: %w", err,
))
}
allPats, err := ah.Signoz.Modules.UserSetter.ListAPIKeys(ctx, orgIdUUID)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't list PATs: %w", err,
))
}
for _, p := range allPats {
if p.UserID == integrationUser.ID && p.Name == integrationPATName {
return p.Token, nil
}
}
slog.InfoContext(ctx, "no PAT found for cloud integration, creating a new one",
"cloud_provider", cloudProvider,
)
newPAT, err := types.NewStorableAPIKey(
integrationPATName,
integrationUser.ID,
types.RoleViewer,
0,
)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't create cloud integration PAT: %w", err,
))
}
factorAPIKey, err = ah.Signoz.Modules.ServiceAccount.GetOrCreateFactorAPIKey(ctx, factorAPIKey)
err = ah.Signoz.Modules.UserSetter.CreateAPIKey(ctx, newPAT)
if err != nil {
return "", basemodel.InternalError(fmt.Errorf(
"couldn't create cloud integration PAT: %w", err,
))
}
return factorAPIKey.Key, nil
return newPAT.Token, nil
}
func (ah *APIHandler) getOrCreateCloudIntegrationServiceAccount(ctx context.Context, orgId valuer.UUID) (*serviceaccounttypes.ServiceAccount, *basemodel.ApiError) {
domain := ah.Signoz.Modules.ServiceAccount.Config().Email.Domain
cloudIntegrationServiceAccount := serviceaccounttypes.NewServiceAccount("integration", domain, serviceaccounttypes.ServiceAccountStatusActive, orgId)
cloudIntegrationServiceAccount, err := ah.Signoz.Modules.ServiceAccount.GetOrCreate(ctx, orgId, cloudIntegrationServiceAccount)
func (ah *APIHandler) getOrCreateCloudIntegrationUser(
ctx context.Context, orgId string, cloudProvider string,
) (*types.User, *basemodel.ApiError) {
cloudIntegrationUserName := fmt.Sprintf("%s-integration", cloudProvider)
email := valuer.MustNewEmail(fmt.Sprintf("%s@signoz.io", cloudIntegrationUserName))
cloudIntegrationUser, err := types.NewUser(cloudIntegrationUserName, email, valuer.MustNewUUID(orgId), types.UserStatusActive)
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration service account: %w", err))
}
err = ah.Signoz.Modules.ServiceAccount.SetRoleByName(ctx, orgId, cloudIntegrationServiceAccount.ID, authtypes.SigNozViewerRoleName)
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration service account: %w", err))
return nil, basemodel.InternalError(fmt.Errorf("couldn't create cloud integration user: %w", err))
}
return cloudIntegrationServiceAccount, nil
password := types.MustGenerateFactorPassword(cloudIntegrationUser.ID.StringValue())
cloudIntegrationUser, err = ah.Signoz.Modules.UserSetter.GetOrCreateUser(
ctx,
cloudIntegrationUser,
user.WithFactorPassword(password),
user.WithRoleNames([]string{authtypes.SigNozViewerRoleName}),
)
if err != nil {
return nil, basemodel.InternalError(fmt.Errorf("couldn't look for integration user: %w", err))
}
return cloudIntegrationUser, nil
}
func (ah *APIHandler) getIngestionUrlAndSigNozAPIUrl(ctx context.Context, licenseKey string) (

View File

@@ -7,10 +7,6 @@ import (
"github.com/SigNoz/signoz/ee/query-service/constants"
"github.com/SigNoz/signoz/ee/query-service/model"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type DayWiseBreakdown struct {
@@ -49,17 +45,15 @@ type details struct {
BillTotal float64 `json:"billTotal"`
}
type billingData struct {
BillingPeriodStart int64 `json:"billingPeriodStart"`
BillingPeriodEnd int64 `json:"billingPeriodEnd"`
Details details `json:"details"`
Discount float64 `json:"discount"`
SubscriptionStatus string `json:"subscriptionStatus"`
}
type billingDetails struct {
Status string `json:"status"`
Data billingData `json:"data"`
Status string `json:"status"`
Data struct {
BillingPeriodStart int64 `json:"billingPeriodStart"`
BillingPeriodEnd int64 `json:"billingPeriodEnd"`
Details details `json:"details"`
Discount float64 `json:"discount"`
SubscriptionStatus string `json:"subscriptionStatus"`
} `json:"data"`
}
func (ah *APIHandler) getBilling(w http.ResponseWriter, r *http.Request) {
@@ -70,33 +64,6 @@ func (ah *APIHandler) getBilling(w http.ResponseWriter, r *http.Request) {
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)
useZeus := ah.Signoz.Flagger.BooleanOrEmpty(r.Context(), flagger.FeatureGetMetersFromZeus, evalCtx)
if useZeus {
data, err := ah.Signoz.Zeus.GetMeters(r.Context(), licenseKey)
if err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
var billing billingData
if err := json.Unmarshal(data, &billing); err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
ah.Respond(w, billing)
return
}
billingURL := fmt.Sprintf("%s/usage?licenseKey=%s", constants.LicenseSignozIo, licenseKey)
hClient := &http.Client{}
@@ -112,11 +79,13 @@ func (ah *APIHandler) getBilling(w http.ResponseWriter, r *http.Request) {
return
}
// decode response body
var billingResponse billingDetails
if err := json.NewDecoder(billingResp.Body).Decode(&billingResponse); err != nil {
RespondError(w, model.InternalError(err), nil)
return
}
// TODO(srikanthccv):Fetch the current day usage and add it to the response
ah.Respond(w, billingResponse.Data)
}

View File

@@ -29,7 +29,6 @@ import (
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/http/middleware"
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/rulestatehistory"
"github.com/SigNoz/signoz/pkg/prometheus"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/signoz"
@@ -49,6 +48,7 @@ import (
opAmpModel "github.com/SigNoz/signoz/pkg/query-service/app/opamp/model"
baseconst "github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/healthcheck"
baseint "github.com/SigNoz/signoz/pkg/query-service/interfaces"
baserules "github.com/SigNoz/signoz/pkg/query-service/rules"
"github.com/SigNoz/signoz/pkg/query-service/utils"
)
@@ -98,6 +98,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
)
rm, err := makeRulesManager(
reader,
signoz.Cache,
signoz.Alertmanager,
signoz.SQLStore,
@@ -105,7 +106,6 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
signoz.TelemetryMetadataStore,
signoz.Prometheus,
signoz.Modules.OrgGetter,
signoz.Modules.RuleStateHistory,
signoz.Querier,
signoz.Instrumentation.ToProviderSettings(),
signoz.QueryParser,
@@ -136,7 +136,6 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
logParsingPipelineController, err := logparsingpipeline.NewLogParsingPipelinesController(
signoz.SQLStore,
integrationsController.GetPipelinesForInstalledIntegrations,
reader,
)
if err != nil {
return nil, err
@@ -227,7 +226,7 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
s.config.APIServer.Timeout.Default,
s.config.APIServer.Timeout.Max,
).Wrap)
r.Use(middleware.NewAudit(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes, s.signoz.Auditor).Wrap)
r.Use(middleware.NewLogging(s.signoz.Instrumentation.Logger(), s.config.APIServer.Logging.ExcludedRoutes).Wrap)
r.Use(middleware.NewComment().Wrap)
apiHandler.RegisterRoutes(r, am)
@@ -240,6 +239,7 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
apiHandler.RegisterWebSocketPaths(r, am)
apiHandler.RegisterMessagingQueuesRoutes(r, am)
apiHandler.RegisterThirdPartyApiRoutes(r, am)
apiHandler.MetricExplorerRoutes(r, am)
apiHandler.RegisterTraceFunnelsRoutes(r, am)
err := s.signoz.APIServer.AddToRouter(r)
@@ -343,28 +343,28 @@ func (s *Server) Stop(ctx context.Context) error {
return nil
}
func makeRulesManager(cache cache.Cache, alertmanager alertmanager.Alertmanager, sqlstore sqlstore.SQLStore, telemetryStore telemetrystore.TelemetryStore, metadataStore telemetrytypes.MetadataStore, prometheus prometheus.Prometheus, orgGetter organization.Getter, ruleStateHistoryModule rulestatehistory.Module, querier querier.Querier, providerSettings factory.ProviderSettings, queryParser queryparser.QueryParser) (*baserules.Manager, error) {
func makeRulesManager(ch baseint.Reader, cache cache.Cache, alertmanager alertmanager.Alertmanager, sqlstore sqlstore.SQLStore, telemetryStore telemetrystore.TelemetryStore, metadataStore telemetrytypes.MetadataStore, prometheus prometheus.Prometheus, orgGetter organization.Getter, querier querier.Querier, providerSettings factory.ProviderSettings, queryParser queryparser.QueryParser) (*baserules.Manager, error) {
ruleStore := sqlrulestore.NewRuleStore(sqlstore, queryParser, providerSettings)
maintenanceStore := sqlrulestore.NewMaintenanceStore(sqlstore)
// create manager opts
managerOpts := &baserules.ManagerOptions{
TelemetryStore: telemetryStore,
MetadataStore: metadataStore,
Prometheus: prometheus,
Context: context.Background(),
Querier: querier,
Logger: providerSettings.Logger,
Cache: cache,
EvalDelay: baseconst.GetEvalDelay(),
PrepareTaskFunc: rules.PrepareTaskFunc,
PrepareTestRuleFunc: rules.TestNotification,
Alertmanager: alertmanager,
OrgGetter: orgGetter,
RuleStore: ruleStore,
MaintenanceStore: maintenanceStore,
SQLStore: sqlstore,
QueryParser: queryParser,
RuleStateHistoryModule: ruleStateHistoryModule,
TelemetryStore: telemetryStore,
MetadataStore: metadataStore,
Prometheus: prometheus,
Context: context.Background(),
Reader: ch,
Querier: querier,
Logger: providerSettings.Logger,
Cache: cache,
EvalDelay: baseconst.GetEvalDelay(),
PrepareTaskFunc: rules.PrepareTaskFunc,
PrepareTestRuleFunc: rules.TestNotification,
Alertmanager: alertmanager,
OrgGetter: orgGetter,
RuleStore: ruleStore,
MaintenanceStore: maintenanceStore,
SqlStore: sqlstore,
QueryParser: queryParser,
}
// create Manager

View File

@@ -5,33 +5,58 @@ import (
"encoding/json"
"fmt"
"log/slog"
"math"
"strings"
"sync"
"time"
"github.com/SigNoz/signoz/ee/query-service/anomaly"
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/types/rulestatehistorytypes"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/transition"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/valuer"
querierV2 "github.com/SigNoz/signoz/pkg/query-service/app/querier/v2"
"github.com/SigNoz/signoz/pkg/query-service/app/queryBuilder"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/utils/labels"
"github.com/SigNoz/signoz/pkg/query-service/utils/times"
"github.com/SigNoz/signoz/pkg/query-service/utils/timestamp"
"github.com/SigNoz/signoz/pkg/units"
baserules "github.com/SigNoz/signoz/pkg/query-service/rules"
"github.com/SigNoz/signoz/ee/anomaly"
querierV5 "github.com/SigNoz/signoz/pkg/querier"
anomalyV2 "github.com/SigNoz/signoz/ee/anomaly"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
)
const (
RuleTypeAnomaly = "anomaly_rule"
)
type AnomalyRule struct {
*baserules.BaseRule
mtx sync.Mutex
// querier is used for alerts migrated after the introduction of new query builder
querier querier.Querier
reader interfaces.Reader
provider anomaly.Provider
// querierV2 is used for alerts created after the introduction of new metrics query builder
querierV2 interfaces.Querier
// querierV5 is used for alerts migrated after the introduction of new query builder
querierV5 querierV5.Querier
provider anomaly.Provider
providerV2 anomalyV2.Provider
version string
logger *slog.Logger
@@ -45,70 +70,125 @@ func NewAnomalyRule(
id string,
orgID valuer.UUID,
p *ruletypes.PostableRule,
querier querier.Querier,
reader interfaces.Reader,
querierV5 querierV5.Querier,
logger *slog.Logger,
cache cache.Cache,
opts ...baserules.RuleOption,
) (*AnomalyRule, error) {
logger.Info("creating new AnomalyRule", slog.String("rule.id", id))
logger.Info("creating new AnomalyRule", "rule_id", id)
opts = append(opts, baserules.WithLogger(logger))
baseRule, err := baserules.NewBaseRule(id, orgID, p, opts...)
baseRule, err := baserules.NewBaseRule(id, orgID, p, reader, opts...)
if err != nil {
return nil, err
}
r := AnomalyRule{
t := AnomalyRule{
BaseRule: baseRule,
querier: querier,
version: p.Version,
logger: logger.With(slog.String("rule.id", id)),
}
switch p.RuleCondition.Seasonality {
case ruletypes.SeasonalityHourly:
r.seasonality = anomaly.SeasonalityHourly
case ruletypes.SeasonalityDaily:
r.seasonality = anomaly.SeasonalityDaily
case ruletypes.SeasonalityWeekly:
r.seasonality = anomaly.SeasonalityWeekly
switch strings.ToLower(p.RuleCondition.Seasonality) {
case "hourly":
t.seasonality = anomaly.SeasonalityHourly
case "daily":
t.seasonality = anomaly.SeasonalityDaily
case "weekly":
t.seasonality = anomaly.SeasonalityWeekly
default:
r.seasonality = anomaly.SeasonalityDaily
t.seasonality = anomaly.SeasonalityDaily
}
r.logger.Info("using seasonality", slog.String("rule.seasonality", r.seasonality.StringValue()))
logger.Info("using seasonality", "seasonality", t.seasonality.String())
if r.seasonality == anomaly.SeasonalityHourly {
r.provider = anomaly.NewHourlyProvider(
anomaly.WithQuerier[*anomaly.HourlyProvider](querier),
anomaly.WithLogger[*anomaly.HourlyProvider](r.logger),
querierOptsV2 := querierV2.QuerierOptions{
Reader: reader,
Cache: cache,
KeyGenerator: queryBuilder.NewKeyGenerator(),
}
t.querierV2 = querierV2.NewQuerier(querierOptsV2)
t.reader = reader
if t.seasonality == anomaly.SeasonalityHourly {
t.provider = anomaly.NewHourlyProvider(
anomaly.WithCache[*anomaly.HourlyProvider](cache),
anomaly.WithKeyGenerator[*anomaly.HourlyProvider](queryBuilder.NewKeyGenerator()),
anomaly.WithReader[*anomaly.HourlyProvider](reader),
)
} else if r.seasonality == anomaly.SeasonalityDaily {
r.provider = anomaly.NewDailyProvider(
anomaly.WithQuerier[*anomaly.DailyProvider](querier),
anomaly.WithLogger[*anomaly.DailyProvider](r.logger),
} else if t.seasonality == anomaly.SeasonalityDaily {
t.provider = anomaly.NewDailyProvider(
anomaly.WithCache[*anomaly.DailyProvider](cache),
anomaly.WithKeyGenerator[*anomaly.DailyProvider](queryBuilder.NewKeyGenerator()),
anomaly.WithReader[*anomaly.DailyProvider](reader),
)
} else if r.seasonality == anomaly.SeasonalityWeekly {
r.provider = anomaly.NewWeeklyProvider(
anomaly.WithQuerier[*anomaly.WeeklyProvider](querier),
anomaly.WithLogger[*anomaly.WeeklyProvider](r.logger),
} else if t.seasonality == anomaly.SeasonalityWeekly {
t.provider = anomaly.NewWeeklyProvider(
anomaly.WithCache[*anomaly.WeeklyProvider](cache),
anomaly.WithKeyGenerator[*anomaly.WeeklyProvider](queryBuilder.NewKeyGenerator()),
anomaly.WithReader[*anomaly.WeeklyProvider](reader),
)
}
return &r, nil
if t.seasonality == anomaly.SeasonalityHourly {
t.providerV2 = anomalyV2.NewHourlyProvider(
anomalyV2.WithQuerier[*anomalyV2.HourlyProvider](querierV5),
anomalyV2.WithLogger[*anomalyV2.HourlyProvider](logger),
)
} else if t.seasonality == anomaly.SeasonalityDaily {
t.providerV2 = anomalyV2.NewDailyProvider(
anomalyV2.WithQuerier[*anomalyV2.DailyProvider](querierV5),
anomalyV2.WithLogger[*anomalyV2.DailyProvider](logger),
)
} else if t.seasonality == anomaly.SeasonalityWeekly {
t.providerV2 = anomalyV2.NewWeeklyProvider(
anomalyV2.WithQuerier[*anomalyV2.WeeklyProvider](querierV5),
anomalyV2.WithLogger[*anomalyV2.WeeklyProvider](logger),
)
}
t.querierV5 = querierV5
t.version = p.Version
t.logger = logger
return &t, nil
}
func (r *AnomalyRule) Type() ruletypes.RuleType {
return ruletypes.RuleTypeAnomaly
return RuleTypeAnomaly
}
func (r *AnomalyRule) prepareQueryRange(ctx context.Context, ts time.Time) *qbtypes.QueryRangeRequest {
func (r *AnomalyRule) prepareQueryRange(ctx context.Context, ts time.Time) (*v3.QueryRangeParamsV3, error) {
r.logger.InfoContext(
ctx, "prepare query range request", slog.Int64("ts", ts.UnixMilli()),
slog.Int64("eval.window_ms", r.EvalWindow().Milliseconds()),
slog.Int64("eval.delay_ms", r.EvalDelay().Milliseconds()),
ctx, "prepare query range request v4", "ts", ts.UnixMilli(), "eval_window", r.EvalWindow().Milliseconds(), "eval_delay", r.EvalDelay().Milliseconds(),
)
st, en := r.Timestamps(ts)
start := st.UnixMilli()
end := en.UnixMilli()
compositeQuery := r.Condition().CompositeQuery
if compositeQuery.PanelType != v3.PanelTypeGraph {
compositeQuery.PanelType = v3.PanelTypeGraph
}
// default mode
return &v3.QueryRangeParamsV3{
Start: start,
End: end,
Step: int64(math.Max(float64(common.MinAllowedStepInterval(start, end)), 60)),
CompositeQuery: compositeQuery,
Variables: make(map[string]interface{}, 0),
NoCache: false,
}, nil
}
func (r *AnomalyRule) prepareQueryRangeV5(ctx context.Context, ts time.Time) (*qbtypes.QueryRangeRequest, error) {
r.logger.InfoContext(ctx, "prepare query range request v5", "ts", ts.UnixMilli(), "eval_window", r.EvalWindow().Milliseconds(), "eval_delay", r.EvalDelay().Milliseconds())
startTs, endTs := r.Timestamps(ts)
start, end := startTs.UnixMilli(), endTs.UnixMilli()
@@ -123,14 +203,25 @@ func (r *AnomalyRule) prepareQueryRange(ctx context.Context, ts time.Time) *qbty
}
req.CompositeQuery.Queries = make([]qbtypes.QueryEnvelope, len(r.Condition().CompositeQuery.Queries))
copy(req.CompositeQuery.Queries, r.Condition().CompositeQuery.Queries)
return req
return req, nil
}
func (r *AnomalyRule) GetSelectedQuery() string {
return r.Condition().GetSelectedQueryName()
}
func (r *AnomalyRule) buildAndRunQuery(ctx context.Context, orgID valuer.UUID, ts time.Time) (ruletypes.Vector, error) {
params := r.prepareQueryRange(ctx, ts)
params, err := r.prepareQueryRange(ctx, ts)
if err != nil {
return nil, err
}
err = r.PopulateTemporality(ctx, orgID, params)
if err != nil {
return nil, fmt.Errorf("internal error while setting temporality")
}
anomalies, err := r.provider.GetAnomalies(ctx, orgID, &anomaly.AnomaliesRequest{
anomalies, err := r.provider.GetAnomalies(ctx, orgID, &anomaly.GetAnomaliesRequest{
Params: params,
Seasonality: r.seasonality,
})
@@ -138,43 +229,87 @@ func (r *AnomalyRule) buildAndRunQuery(ctx context.Context, orgID valuer.UUID, t
return nil, err
}
var queryResult *qbtypes.TimeSeriesData
var queryResult *v3.Result
for _, result := range anomalies.Results {
if result.QueryName == r.SelectedQuery(ctx) {
if result.QueryName == r.GetSelectedQuery() {
queryResult = result
break
}
}
if queryResult == nil {
r.logger.WarnContext(ctx, "nil qb result", slog.Int64("ts", ts.UnixMilli()))
return ruletypes.Vector{}, nil
}
hasData := len(queryResult.Aggregations) > 0 &&
queryResult.Aggregations[0] != nil &&
len(queryResult.Aggregations[0].AnomalyScores) > 0
hasData := len(queryResult.AnomalyScores) > 0
if missingDataAlert := r.HandleMissingDataAlert(ctx, ts, hasData); missingDataAlert != nil {
return ruletypes.Vector{*missingDataAlert}, nil
} else if !hasData {
r.logger.WarnContext(ctx, "no anomaly result")
return ruletypes.Vector{}, nil
}
var resultVector ruletypes.Vector
scoresJSON, _ := json.Marshal(queryResult.Aggregations[0].AnomalyScores)
// TODO(srikanthccv): this could be noisy but we do this to answer false alert requests
r.logger.InfoContext(ctx, "anomaly scores", slog.String("anomaly.scores", string(scoresJSON)))
scoresJSON, _ := json.Marshal(queryResult.AnomalyScores)
r.logger.InfoContext(ctx, "anomaly scores", "scores", string(scoresJSON))
for _, series := range queryResult.AnomalyScores {
if !r.Condition().ShouldEval(series) {
r.logger.InfoContext(ctx, "not enough data points to evaluate series, skipping", "ruleid", r.ID(), "numPoints", len(series.Points), "requiredPoints", r.Condition().RequiredNumPoints)
continue
}
results, err := r.Threshold.Eval(*series, r.Unit(), ruletypes.EvalData{
ActiveAlerts: r.ActiveAlertsLabelFP(),
SendUnmatched: r.ShouldSendUnmatched(),
})
if err != nil {
return nil, err
}
resultVector = append(resultVector, results...)
}
return resultVector, nil
}
func (r *AnomalyRule) buildAndRunQueryV5(ctx context.Context, orgID valuer.UUID, ts time.Time) (ruletypes.Vector, error) {
params, err := r.prepareQueryRangeV5(ctx, ts)
if err != nil {
return nil, err
}
anomalies, err := r.providerV2.GetAnomalies(ctx, orgID, &anomalyV2.AnomaliesRequest{
Params: *params,
Seasonality: anomalyV2.Seasonality{String: valuer.NewString(r.seasonality.String())},
})
if err != nil {
return nil, err
}
var qbResult *qbtypes.TimeSeriesData
for _, result := range anomalies.Results {
if result.QueryName == r.GetSelectedQuery() {
qbResult = result
break
}
}
if qbResult == nil {
r.logger.WarnContext(ctx, "nil qb result", "ts", ts.UnixMilli())
}
queryResult := transition.ConvertV5TimeSeriesDataToV4Result(qbResult)
hasData := len(queryResult.AnomalyScores) > 0
if missingDataAlert := r.HandleMissingDataAlert(ctx, ts, hasData); missingDataAlert != nil {
return ruletypes.Vector{*missingDataAlert}, nil
}
var resultVector ruletypes.Vector
scoresJSON, _ := json.Marshal(queryResult.AnomalyScores)
r.logger.InfoContext(ctx, "anomaly scores", "scores", string(scoresJSON))
// Filter out new series if newGroupEvalDelay is configured
seriesToProcess := queryResult.Aggregations[0].AnomalyScores
seriesToProcess := queryResult.AnomalyScores
if r.ShouldSkipNewGroups() {
filteredSeries, filterErr := r.BaseRule.FilterNewSeries(ctx, ts, seriesToProcess)
// In case of error we log the error and continue with the original series
if filterErr != nil {
r.logger.ErrorContext(ctx, "error filtering new series", errors.Attr(filterErr))
r.logger.ErrorContext(ctx, "Error filtering new series, ", errors.Attr(filterErr), "rule_name", r.Name())
} else {
seriesToProcess = filteredSeries
}
@@ -182,14 +317,10 @@ func (r *AnomalyRule) buildAndRunQuery(ctx context.Context, orgID valuer.UUID, t
for _, series := range seriesToProcess {
if !r.Condition().ShouldEval(series) {
r.logger.InfoContext(
ctx, "not enough data points to evaluate series, skipping",
slog.Int("series.num_points", len(series.Values)),
slog.Int("series.required_points", r.Condition().RequiredNumPoints),
)
r.logger.InfoContext(ctx, "not enough data points to evaluate series, skipping", "ruleid", r.ID(), "numPoints", len(series.Points), "requiredPoints", r.Condition().RequiredNumPoints)
continue
}
results, err := r.Threshold.Eval(series, r.Unit(), ruletypes.EvalData{
results, err := r.Threshold.Eval(*series, r.Unit(), ruletypes.EvalData{
ActiveAlerts: r.ActiveAlertsLabelFP(),
SendUnmatched: r.ShouldSendUnmatched(),
})
@@ -210,9 +341,13 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
var res ruletypes.Vector
var err error
r.logger.InfoContext(ctx, "running query")
res, err = r.buildAndRunQuery(ctx, r.OrgID(), ts)
if r.version == "v5" {
r.logger.InfoContext(ctx, "running v5 query")
res, err = r.buildAndRunQueryV5(ctx, r.OrgID(), ts)
} else {
r.logger.InfoContext(ctx, "running v4 query")
res, err = r.buildAndRunQuery(ctx, r.OrgID(), ts)
}
if err != nil {
return 0, err
}
@@ -236,10 +371,7 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
}
value := valueFormatter.Format(smpl.V, r.Unit())
threshold := valueFormatter.Format(smpl.Target, smpl.TargetUnit)
r.logger.DebugContext(
ctx, "alert template data for rule", slog.String("formatter.name", valueFormatter.Name()),
slog.String("alert.value", value), slog.String("alert.threshold", threshold),
)
r.logger.DebugContext(ctx, "Alert template data for rule", "rule_name", r.Name(), "formatter", valueFormatter.Name(), "value", value, "threshold", threshold)
tmplData := ruletypes.AlertTemplateData(l, value, threshold)
// Inject some convenience variables that are easier to remember for users
@@ -254,34 +386,35 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
defs+text,
"__alert_"+r.Name(),
tmplData,
times.Time(timestamp.FromTime(ts)),
nil,
)
result, err := tmpl.Expand()
if err != nil {
result = fmt.Sprintf("<error expanding template: %s>", err)
r.logger.ErrorContext(ctx, "expanding alert template failed", errors.Attr(err), slog.Any("alert.template_data", tmplData))
r.logger.ErrorContext(ctx, "Expanding alert template failed", errors.Attr(err), "data", tmplData, "rule_name", r.Name())
}
return result
}
lb := ruletypes.NewBuilder(smpl.Metric...).Del(ruletypes.MetricNameLabel).Del(ruletypes.TemporalityLabel)
resultLabels := ruletypes.NewBuilder(smpl.Metric...).Del(ruletypes.MetricNameLabel).Del(ruletypes.TemporalityLabel).Labels()
lb := labels.NewBuilder(smpl.Metric).Del(labels.MetricNameLabel).Del(labels.TemporalityLabel)
resultLabels := labels.NewBuilder(smpl.Metric).Del(labels.MetricNameLabel).Del(labels.TemporalityLabel).Labels()
for name, value := range r.Labels().Map() {
lb.Set(name, expand(value))
}
lb.Set(ruletypes.AlertNameLabel, r.Name())
lb.Set(ruletypes.AlertRuleIDLabel, r.ID())
lb.Set(ruletypes.RuleSourceLabel, r.GeneratorURL())
lb.Set(labels.AlertNameLabel, r.Name())
lb.Set(labels.AlertRuleIdLabel, r.ID())
lb.Set(labels.RuleSourceLabel, r.GeneratorURL())
annotations := make(ruletypes.Labels, 0, len(r.Annotations().Map()))
annotations := make(labels.Labels, 0, len(r.Annotations().Map()))
for name, value := range r.Annotations().Map() {
annotations = append(annotations, ruletypes.Label{Name: name, Value: expand(value)})
annotations = append(annotations, labels.Label{Name: name, Value: expand(value)})
}
if smpl.IsMissing {
lb.Set(ruletypes.AlertNameLabel, "[No data] "+r.Name())
lb.Set(ruletypes.NoDataLabel, "true")
lb.Set(labels.AlertNameLabel, "[No data] "+r.Name())
lb.Set(labels.NoDataLabel, "true")
}
lbs := lb.Labels()
@@ -289,17 +422,17 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
resultFPs[h] = struct{}{}
if _, ok := alerts[h]; ok {
r.logger.ErrorContext(ctx, "the alert query returns duplicate records", slog.Any("alert", alerts[h]))
err = errors.NewInternalf(errors.CodeInternal, "duplicate alert found, vector contains metrics with the same labelset after applying alert labels")
r.logger.ErrorContext(ctx, "the alert query returns duplicate records", "rule_id", r.ID(), "alert", alerts[h])
err = fmt.Errorf("duplicate alert found, vector contains metrics with the same labelset after applying alert labels")
return 0, err
}
alerts[h] = &ruletypes.Alert{
Labels: lbs,
QueryResultLabels: resultLabels,
QueryResultLables: resultLabels,
Annotations: annotations,
ActiveAt: ts,
State: ruletypes.StatePending,
State: model.StatePending,
Value: smpl.V,
GeneratorURL: r.GeneratorURL(),
Receivers: ruleReceiverMap[lbs.Map()[ruletypes.LabelThresholdName]],
@@ -308,12 +441,12 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
}
}
r.logger.InfoContext(ctx, "number of alerts found", slog.Int("alert.count", len(alerts)))
r.logger.InfoContext(ctx, "number of alerts found", "rule_name", r.Name(), "alerts_count", len(alerts))
// alerts[h] is ready, add or update active list now
for h, a := range alerts {
// Check whether we already have alerting state for the identifying label set.
// Update the last value and annotations if so, create a new alert entry otherwise.
if alert, ok := r.Active[h]; ok && alert.State != ruletypes.StateInactive {
if alert, ok := r.Active[h]; ok && alert.State != model.StateInactive {
alert.Value = a.Value
alert.Annotations = a.Annotations
@@ -329,76 +462,76 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
r.Active[h] = a
}
itemsToAdd := []rulestatehistorytypes.RuleStateHistory{}
itemsToAdd := []model.RuleStateHistory{}
// Check if any pending alerts should be removed or fire now. Write out alert timeseries.
for fp, a := range r.Active {
labelsJSON, err := json.Marshal(a.QueryResultLabels)
labelsJSON, err := json.Marshal(a.QueryResultLables)
if err != nil {
r.logger.ErrorContext(ctx, "error marshaling labels", errors.Attr(err), slog.Any("alert.labels", a.Labels))
r.logger.ErrorContext(ctx, "error marshaling labels", errors.Attr(err), "labels", a.Labels)
}
if _, ok := resultFPs[fp]; !ok {
// If the alert was previously firing, keep it around for a given
// retention time so it is reported as resolved to the AlertManager.
if a.State == ruletypes.StatePending || (!a.ResolvedAt.IsZero() && ts.Sub(a.ResolvedAt) > ruletypes.ResolvedRetention) {
if a.State == model.StatePending || (!a.ResolvedAt.IsZero() && ts.Sub(a.ResolvedAt) > ruletypes.ResolvedRetention) {
delete(r.Active, fp)
}
if a.State != ruletypes.StateInactive {
a.State = ruletypes.StateInactive
if a.State != model.StateInactive {
a.State = model.StateInactive
a.ResolvedAt = ts
itemsToAdd = append(itemsToAdd, rulestatehistorytypes.RuleStateHistory{
itemsToAdd = append(itemsToAdd, model.RuleStateHistory{
RuleID: r.ID(),
RuleName: r.Name(),
State: ruletypes.StateInactive,
State: model.StateInactive,
StateChanged: true,
UnixMilli: ts.UnixMilli(),
Labels: rulestatehistorytypes.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLabels.Hash(),
Labels: model.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLables.Hash(),
Value: a.Value,
})
}
continue
}
if a.State == ruletypes.StatePending && ts.Sub(a.ActiveAt) >= r.HoldDuration().Duration() {
a.State = ruletypes.StateFiring
if a.State == model.StatePending && ts.Sub(a.ActiveAt) >= r.HoldDuration().Duration() {
a.State = model.StateFiring
a.FiredAt = ts
state := ruletypes.StateFiring
state := model.StateFiring
if a.Missing {
state = ruletypes.StateNoData
state = model.StateNoData
}
itemsToAdd = append(itemsToAdd, rulestatehistorytypes.RuleStateHistory{
itemsToAdd = append(itemsToAdd, model.RuleStateHistory{
RuleID: r.ID(),
RuleName: r.Name(),
State: state,
StateChanged: true,
UnixMilli: ts.UnixMilli(),
Labels: rulestatehistorytypes.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLabels.Hash(),
Labels: model.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLables.Hash(),
Value: a.Value,
})
}
// We need to change firing alert to recovering if the returned sample meets recovery threshold
changeFiringToRecovering := a.State == ruletypes.StateFiring && a.IsRecovering
changeFiringToRecovering := a.State == model.StateFiring && a.IsRecovering
// We need to change recovering alerts to firing if the returned sample meets target threshold
changeRecoveringToFiring := a.State == ruletypes.StateRecovering && !a.IsRecovering && !a.Missing
changeRecoveringToFiring := a.State == model.StateRecovering && !a.IsRecovering && !a.Missing
// in any of the above case we need to update the status of alert
if changeFiringToRecovering || changeRecoveringToFiring {
state := ruletypes.StateRecovering
state := model.StateRecovering
if changeRecoveringToFiring {
state = ruletypes.StateFiring
state = model.StateFiring
}
a.State = state
r.logger.DebugContext(ctx, "converting alert state", slog.Any("alert.state", state))
itemsToAdd = append(itemsToAdd, rulestatehistorytypes.RuleStateHistory{
r.logger.DebugContext(ctx, "converting alert state", "name", r.Name(), "state", state)
itemsToAdd = append(itemsToAdd, model.RuleStateHistory{
RuleID: r.ID(),
RuleName: r.Name(),
State: state,
StateChanged: true,
UnixMilli: ts.UnixMilli(),
Labels: rulestatehistorytypes.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLabels.Hash(),
Labels: model.LabelsString(labelsJSON),
Fingerprint: a.QueryResultLables.Hash(),
Value: a.Value,
})
}
@@ -413,7 +546,7 @@ func (r *AnomalyRule) Eval(ctx context.Context, ts time.Time) (int, error) {
itemsToAdd[idx] = item
}
_ = r.RecordRuleStateHistory(ctx, itemsToAdd)
r.RecordRuleStateHistory(ctx, prevState, currentState, itemsToAdd)
return len(r.Active), nil
}

View File

@@ -2,19 +2,21 @@ package rules
import (
"context"
"log/slog"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/SigNoz/signoz/ee/query-service/anomaly"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/query-service/app/clickhouseReader"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrystore/telemetrystoretest"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/SigNoz/signoz/ee/anomaly"
)
// mockAnomalyProvider is a mock implementation of anomaly.Provider for testing.
@@ -22,13 +24,13 @@ import (
// time periods (current, past period, current season, past season, past 2 seasons,
// past 3 seasons), making it cumbersome to create mock data.
type mockAnomalyProvider struct {
responses []*anomaly.AnomaliesResponse
responses []*anomaly.GetAnomaliesResponse
callCount int
}
func (m *mockAnomalyProvider) GetAnomalies(ctx context.Context, orgID valuer.UUID, req *anomaly.AnomaliesRequest) (*anomaly.AnomaliesResponse, error) {
func (m *mockAnomalyProvider) GetAnomalies(ctx context.Context, orgID valuer.UUID, req *anomaly.GetAnomaliesRequest) (*anomaly.GetAnomaliesResponse, error) {
if m.callCount >= len(m.responses) {
return &anomaly.AnomaliesResponse{Results: []*qbtypes.TimeSeriesData{}}, nil
return &anomaly.GetAnomaliesResponse{Results: []*v3.Result{}}, nil
}
resp := m.responses[m.callCount]
m.callCount++
@@ -47,46 +49,45 @@ func TestAnomalyRule_NoData_AlertOnAbsent(t *testing.T) {
postableRule := ruletypes.PostableRule{
AlertName: "Test anomaly no data",
AlertType: ruletypes.AlertTypeMetric,
RuleType: ruletypes.RuleTypeAnomaly,
RuleType: RuleTypeAnomaly,
Evaluation: &ruletypes.EvaluationEnvelope{Kind: ruletypes.RollingEvaluation, Spec: ruletypes.RollingWindow{
EvalWindow: evalWindow,
Frequency: valuer.MustParseTextDuration("1m"),
}},
RuleCondition: &ruletypes.RuleCondition{
CompareOperator: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
Target: &target,
CompositeQuery: &ruletypes.AlertCompositeQuery{
QueryType: ruletypes.QueryTypeBuilder,
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
CompareOp: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
Target: &target,
CompositeQuery: &v3.CompositeQuery{
QueryType: v3.QueryTypeBuilder,
BuilderQueries: map[string]*v3.BuilderQuery{
"A": {
QueryName: "A",
Expression: "A",
DataSource: v3.DataSourceMetrics,
Temporality: v3.Unspecified,
},
}},
},
},
SelectedQuery: "A",
Seasonality: ruletypes.SeasonalityDaily,
Seasonality: "daily",
Thresholds: &ruletypes.RuleThresholdData{
Kind: ruletypes.BasicThresholdKind,
Spec: ruletypes.BasicRuleThresholds{{
Name: "Test anomaly no data",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOperator: ruletypes.ValueIsAbove,
Name: "Test anomaly no data",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOp: ruletypes.ValueIsAbove,
}},
},
},
}
responseNoData := &anomaly.AnomaliesResponse{
Results: []*qbtypes.TimeSeriesData{
responseNoData := &anomaly.GetAnomaliesResponse{
Results: []*v3.Result{
{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
AnomalyScores: []*qbtypes.TimeSeries{},
}},
QueryName: "A",
AnomalyScores: []*v3.Series{},
},
},
}
@@ -114,17 +115,23 @@ func TestAnomalyRule_NoData_AlertOnAbsent(t *testing.T) {
t.Run(c.description, func(t *testing.T) {
postableRule.RuleCondition.AlertOnAbsent = c.alertOnAbsent
telemetryStore := telemetrystoretest.New(telemetrystore.Config{}, nil)
options := clickhouseReader.NewOptions("primaryNamespace")
reader := clickhouseReader.NewReader(slog.Default(), nil, telemetryStore, nil, "", time.Second, nil, nil, options)
rule, err := NewAnomalyRule(
"test-anomaly-rule",
valuer.GenerateUUID(),
&postableRule,
reader,
nil,
logger,
nil,
)
require.NoError(t, err)
rule.provider = &mockAnomalyProvider{
responses: []*anomaly.AnomaliesResponse{responseNoData},
responses: []*anomaly.GetAnomaliesResponse{responseNoData},
}
alertsFound, err := rule.Eval(context.Background(), evalTime)
@@ -149,47 +156,46 @@ func TestAnomalyRule_NoData_AbsentFor(t *testing.T) {
postableRule := ruletypes.PostableRule{
AlertName: "Test anomaly no data with AbsentFor",
AlertType: ruletypes.AlertTypeMetric,
RuleType: ruletypes.RuleTypeAnomaly,
RuleType: RuleTypeAnomaly,
Evaluation: &ruletypes.EvaluationEnvelope{Kind: ruletypes.RollingEvaluation, Spec: ruletypes.RollingWindow{
EvalWindow: evalWindow,
Frequency: valuer.MustParseTextDuration("1m"),
}},
RuleCondition: &ruletypes.RuleCondition{
CompareOperator: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
AlertOnAbsent: true,
Target: &target,
CompositeQuery: &ruletypes.AlertCompositeQuery{
QueryType: ruletypes.QueryTypeBuilder,
Queries: []qbtypes.QueryEnvelope{{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
CompareOp: ruletypes.ValueIsAbove,
MatchType: ruletypes.AtleastOnce,
AlertOnAbsent: true,
Target: &target,
CompositeQuery: &v3.CompositeQuery{
QueryType: v3.QueryTypeBuilder,
BuilderQueries: map[string]*v3.BuilderQuery{
"A": {
QueryName: "A",
Expression: "A",
DataSource: v3.DataSourceMetrics,
Temporality: v3.Unspecified,
},
}},
},
},
SelectedQuery: "A",
Seasonality: ruletypes.SeasonalityDaily,
Seasonality: "daily",
Thresholds: &ruletypes.RuleThresholdData{
Kind: ruletypes.BasicThresholdKind,
Spec: ruletypes.BasicRuleThresholds{{
Name: "Test anomaly no data with AbsentFor",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOperator: ruletypes.ValueIsAbove,
Name: "Test anomaly no data with AbsentFor",
TargetValue: &target,
MatchType: ruletypes.AtleastOnce,
CompareOp: ruletypes.ValueIsAbove,
}},
},
},
}
responseNoData := &anomaly.AnomaliesResponse{
Results: []*qbtypes.TimeSeriesData{
responseNoData := &anomaly.GetAnomaliesResponse{
Results: []*v3.Result{
{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
AnomalyScores: []*qbtypes.TimeSeries{},
}},
QueryName: "A",
AnomalyScores: []*v3.Series{},
},
},
}
@@ -223,35 +229,32 @@ func TestAnomalyRule_NoData_AbsentFor(t *testing.T) {
t1 := baseTime.Add(5 * time.Minute)
t2 := t1.Add(c.timeBetweenEvals)
responseWithData := &anomaly.AnomaliesResponse{
Results: []*qbtypes.TimeSeriesData{
responseWithData := &anomaly.GetAnomaliesResponse{
Results: []*v3.Result{
{
QueryName: "A",
Aggregations: []*qbtypes.AggregationBucket{{
AnomalyScores: []*qbtypes.TimeSeries{
{
Labels: []*qbtypes.Label{
{
Key: telemetrytypes.TelemetryFieldKey{Name: "Test"},
Value: "labels",
},
},
Values: []*qbtypes.TimeSeriesValue{
{Timestamp: baseTime.UnixMilli(), Value: 1.0},
{Timestamp: baseTime.Add(time.Minute).UnixMilli(), Value: 1.5},
},
AnomalyScores: []*v3.Series{
{
Labels: map[string]string{"test": "label"},
Points: []v3.Point{
{Timestamp: baseTime.UnixMilli(), Value: 1.0},
{Timestamp: baseTime.Add(time.Minute).UnixMilli(), Value: 1.5},
},
},
}},
},
},
},
}
rule, err := NewAnomalyRule("test-anomaly-rule", valuer.GenerateUUID(), &postableRule, nil, logger)
telemetryStore := telemetrystoretest.New(telemetrystore.Config{}, nil)
options := clickhouseReader.NewOptions("primaryNamespace")
reader := clickhouseReader.NewReader(slog.Default(), nil, telemetryStore, nil, "", time.Second, nil, nil, options)
rule, err := NewAnomalyRule("test-anomaly-rule", valuer.GenerateUUID(), &postableRule, reader, nil, logger, nil)
require.NoError(t, err)
rule.provider = &mockAnomalyProvider{
responses: []*anomaly.AnomaliesResponse{responseWithData, responseNoData},
responses: []*anomaly.GetAnomaliesResponse{responseWithData, responseNoData},
}
alertsFound1, err := rule.Eval(context.Background(), t1)

View File

@@ -11,7 +11,9 @@ import (
"github.com/google/uuid"
"github.com/SigNoz/signoz/pkg/errors"
basemodel "github.com/SigNoz/signoz/pkg/query-service/model"
baserules "github.com/SigNoz/signoz/pkg/query-service/rules"
"github.com/SigNoz/signoz/pkg/query-service/utils/labels"
"github.com/SigNoz/signoz/pkg/types/ruletypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -21,25 +23,24 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
rules := make([]baserules.Rule, 0)
var task baserules.Task
ruleID := baserules.RuleIDFromTaskName(opts.TaskName)
ruleId := baserules.RuleIdFromTaskName(opts.TaskName)
evaluation, err := opts.Rule.Evaluation.GetEvaluation()
if err != nil {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "evaluation is invalid: %v", err)
}
if opts.Rule.RuleType == ruletypes.RuleTypeThreshold {
// create a threshold rule
tr, err := baserules.NewThresholdRule(
ruleID,
ruleId,
opts.OrgID,
opts.Rule,
opts.Reader,
opts.Querier,
opts.Logger,
baserules.WithEvalDelay(opts.ManagerOpts.EvalDelay),
baserules.WithSQLStore(opts.SQLStore),
baserules.WithQueryParser(opts.ManagerOpts.QueryParser),
baserules.WithMetadataStore(opts.ManagerOpts.MetadataStore),
baserules.WithRuleStateHistoryModule(opts.ManagerOpts.RuleStateHistoryModule),
)
if err != nil {
@@ -55,15 +56,15 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
// create promql rule
pr, err := baserules.NewPromRule(
ruleID,
ruleId,
opts.OrgID,
opts.Rule,
opts.Logger,
opts.Reader,
opts.ManagerOpts.Prometheus,
baserules.WithSQLStore(opts.SQLStore),
baserules.WithQueryParser(opts.ManagerOpts.QueryParser),
baserules.WithMetadataStore(opts.ManagerOpts.MetadataStore),
baserules.WithRuleStateHistoryModule(opts.ManagerOpts.RuleStateHistoryModule),
)
if err != nil {
@@ -78,16 +79,17 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
} else if opts.Rule.RuleType == ruletypes.RuleTypeAnomaly {
// create anomaly rule
ar, err := NewAnomalyRule(
ruleID,
ruleId,
opts.OrgID,
opts.Rule,
opts.Reader,
opts.Querier,
opts.Logger,
opts.Cache,
baserules.WithEvalDelay(opts.ManagerOpts.EvalDelay),
baserules.WithSQLStore(opts.SQLStore),
baserules.WithQueryParser(opts.ManagerOpts.QueryParser),
baserules.WithMetadataStore(opts.ManagerOpts.MetadataStore),
baserules.WithRuleStateHistoryModule(opts.ManagerOpts.RuleStateHistoryModule),
)
if err != nil {
return task, err
@@ -99,7 +101,7 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
task = newTask(baserules.TaskTypeCh, opts.TaskName, evaluation.GetFrequency().Duration(), rules, opts.ManagerOpts, opts.NotifyFunc, opts.MaintenanceStore, opts.OrgID)
} else {
return nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported rule type %s. Supported types: %s, %s", opts.Rule.RuleType, ruletypes.RuleTypeProm, ruletypes.RuleTypeThreshold)
return nil, fmt.Errorf("unsupported rule type %s. Supported types: %s, %s", opts.Rule.RuleType, ruletypes.RuleTypeProm, ruletypes.RuleTypeThreshold)
}
return task, nil
@@ -107,12 +109,12 @@ func PrepareTaskFunc(opts baserules.PrepareTaskOptions) (baserules.Task, error)
// TestNotification prepares a dummy rule for given rule parameters and
// sends a test notification. returns alert count and error (if any)
func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
func TestNotification(opts baserules.PrepareTestRuleOptions) (int, *basemodel.ApiError) {
ctx := context.Background()
if opts.Rule == nil {
return 0, errors.NewInvalidInputf(errors.CodeInvalidInput, "rule is required")
return 0, basemodel.BadRequest(fmt.Errorf("rule is required"))
}
parsedRule := opts.Rule
@@ -132,14 +134,15 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
if parsedRule.RuleType == ruletypes.RuleTypeThreshold {
// add special labels for test alerts
parsedRule.Labels[ruletypes.RuleSourceLabel] = ""
parsedRule.Labels[ruletypes.AlertRuleIDLabel] = ""
parsedRule.Labels[labels.RuleSourceLabel] = ""
parsedRule.Labels[labels.AlertRuleIdLabel] = ""
// create a threshold rule
rule, err = baserules.NewThresholdRule(
alertname,
opts.OrgID,
parsedRule,
opts.Reader,
opts.Querier,
opts.Logger,
baserules.WithSendAlways(),
@@ -151,7 +154,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
if err != nil {
slog.Error("failed to prepare a new threshold rule for test", "name", alertname, errors.Attr(err))
return 0, err
return 0, basemodel.BadRequest(err)
}
} else if parsedRule.RuleType == ruletypes.RuleTypeProm {
@@ -162,6 +165,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
opts.OrgID,
parsedRule,
opts.Logger,
opts.Reader,
opts.ManagerOpts.Prometheus,
baserules.WithSendAlways(),
baserules.WithSendUnmatched(),
@@ -172,7 +176,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
if err != nil {
slog.Error("failed to prepare a new promql rule for test", "name", alertname, errors.Attr(err))
return 0, err
return 0, basemodel.BadRequest(err)
}
} else if parsedRule.RuleType == ruletypes.RuleTypeAnomaly {
// create anomaly rule
@@ -180,8 +184,10 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
alertname,
opts.OrgID,
parsedRule,
opts.Reader,
opts.Querier,
opts.Logger,
opts.Cache,
baserules.WithSendAlways(),
baserules.WithSendUnmatched(),
baserules.WithSQLStore(opts.SQLStore),
@@ -190,10 +196,10 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
)
if err != nil {
slog.Error("failed to prepare a new anomaly rule for test", "name", alertname, errors.Attr(err))
return 0, err
return 0, basemodel.BadRequest(err)
}
} else {
return 0, errors.NewInvalidInputf(errors.CodeInvalidInput, "failed to derive ruletype with given information")
return 0, basemodel.BadRequest(fmt.Errorf("failed to derive ruletype with given information"))
}
// set timestamp to current utc time
@@ -202,7 +208,7 @@ func TestNotification(opts baserules.PrepareTestRuleOptions) (int, error) {
alertsFound, err := rule.Eval(ctx, ts)
if err != nil {
slog.Error("evaluating rule failed", "rule", rule.Name(), errors.Attr(err))
return 0, err
return 0, basemodel.InternalError(fmt.Errorf("rule evaluation failed"))
}
rule.SendAlerts(ctx, ts, 0, time.Minute, opts.NotifyFunc)

View File

@@ -114,8 +114,11 @@ func TestManager_TestNotification_SendUnmatched_ThresholdRule(t *testing.T) {
},
})
count, err := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
require.Nil(t, err)
count, apiErr := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
if apiErr != nil {
t.Logf("TestNotification error: %v, type: %s", apiErr.Err, apiErr.Typ)
}
require.Nil(t, apiErr)
assert.Equal(t, tc.ExpectAlerts, count)
if tc.ExpectAlerts > 0 {
@@ -254,7 +257,7 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
WillReturnRows(samplesRows)
// Create Prometheus provider for this test
promProvider = prometheustest.New(context.Background(), instrumentationtest.New().ToProviderSettings(), prometheus.Config{Timeout: 2 * time.Minute}, store)
promProvider = prometheustest.New(context.Background(), instrumentationtest.New().ToProviderSettings(), prometheus.Config{}, store)
},
ManagerOptionsHook: func(opts *rules.ManagerOptions) {
// Set Prometheus provider for PromQL queries
@@ -265,8 +268,11 @@ func TestManager_TestNotification_SendUnmatched_PromRule(t *testing.T) {
},
})
count, err := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
require.Nil(t, err)
count, apiErr := mgr.TestNotification(context.Background(), orgID, string(ruleBytes))
if apiErr != nil {
t.Logf("TestNotification error: %v, type: %s", apiErr.Err, apiErr.Typ)
}
require.Nil(t, apiErr)
assert.Equal(t, tc.ExpectAlerts, count)
if tc.ExpectAlerts > 0 {

View File

@@ -336,10 +336,9 @@ func (dialect *dialect) UpdatePrimaryKey(ctx context.Context, bun bun.IDB, oldMo
}
fkReference := ""
switch reference {
case Org:
if reference == Org {
fkReference = OrgReference
case User:
} else if reference == User {
fkReference = UserReference
}
@@ -393,10 +392,9 @@ func (dialect *dialect) AddPrimaryKey(ctx context.Context, bun bun.IDB, oldModel
}
fkReference := ""
switch reference {
case Org:
if reference == Org {
fkReference = OrgReference
case User:
} else if reference == User {
fkReference = UserReference
}

View File

@@ -14,21 +14,14 @@ import (
"github.com/uptrace/bun/dialect/pgdialect"
)
var _ Pooler = new(provider)
type provider struct {
settings factory.ScopedProviderSettings
sqldb *sql.DB
bundb *sqlstore.BunDB
pgxPool *pgxpool.Pool
dialect *dialect
formatter sqlstore.SQLFormatter
}
type Pooler interface {
Pool() *pgxpool.Pool
}
func NewFactory(hookFactories ...factory.ProviderFactory[sqlstore.SQLStoreHook, sqlstore.Config]) factory.ProviderFactory[sqlstore.SQLStore, sqlstore.Config] {
return factory.NewProviderFactory(factory.MustNewName("postgres"), func(ctx context.Context, providerSettings factory.ProviderSettings, config sqlstore.Config) (sqlstore.SQLStore, error) {
hooks := make([]sqlstore.SQLStoreHook, len(hookFactories))
@@ -54,7 +47,6 @@ func New(ctx context.Context, providerSettings factory.ProviderSettings, config
// Set the maximum number of open connections
pgConfig.MaxConns = int32(config.Connection.MaxOpenConns)
pgConfig.MaxConnLifetime = config.Connection.MaxConnLifetime
// Use pgxpool to create a connection pool
pool, err := pgxpool.NewWithConfig(ctx, pgConfig)
@@ -70,7 +62,6 @@ func New(ctx context.Context, providerSettings factory.ProviderSettings, config
settings: settings,
sqldb: sqldb,
bundb: bunDB,
pgxPool: pool,
dialect: new(dialect),
formatter: newFormatter(bunDB.Dialect()),
}, nil
@@ -84,10 +75,6 @@ func (provider *provider) SQLDB() *sql.DB {
return provider.sqldb
}
func (provider *provider) Pool() *pgxpool.Pool {
return provider.pgxPool
}
func (provider *provider) Dialect() sqlstore.SQLDialect {
return provider.dialect
}

View File

@@ -19,7 +19,7 @@ var (
once sync.Once
)
// initializes the Zeus configuration.
// initializes the Zeus configuration
func Config() zeus.Config {
once.Do(func() {
parsedURL, err := neturl.Parse(url)

View File

@@ -109,21 +109,6 @@ func (provider *Provider) GetDeployment(ctx context.Context, key string) ([]byte
return []byte(gjson.GetBytes(response, "data").String()), nil
}
func (provider *Provider) GetMeters(ctx context.Context, key string) ([]byte, error) {
response, err := provider.do(
ctx,
provider.config.URL.JoinPath("/v1/meters"),
http.MethodGet,
key,
nil,
)
if err != nil {
return nil, err
}
return []byte(gjson.GetBytes(response, "data").String()), nil
}
func (provider *Provider) PutMeters(ctx context.Context, key string, data []byte) error {
_, err := provider.do(
ctx,
@@ -204,7 +189,7 @@ func (provider *Provider) do(ctx context.Context, url *url.URL, method string, k
return nil, provider.errFromStatusCode(response.StatusCode, errorMessage)
}
// This can be taken down to the client package.
// This can be taken down to the client package
func (provider *Provider) errFromStatusCode(statusCode int, errorMessage string) error {
switch statusCode {
case http.StatusBadRequest:

View File

@@ -1,7 +1,5 @@
node_modules
build
eslint-rules/
stylelint-rules/
*.typegen.ts
i18-generate-hash.js
src/parser/TraceOperatorParser/**

View File

@@ -1,8 +1,3 @@
/* eslint-disable @typescript-eslint/no-var-requires */
const path = require('path');
const rulesDirPlugin = require('eslint-plugin-rulesdir');
rulesDirPlugin.RULES_DIR = path.join(__dirname, 'eslint-rules');
/**
* ESLint Configuration for SigNoz Frontend
*/
@@ -37,7 +32,6 @@ module.exports = {
sourceType: 'module',
},
plugins: [
'rulesdir', // Local custom rules
'react', // React-specific rules
'@typescript-eslint', // TypeScript linting
'simple-import-sort', // Auto-sort imports
@@ -62,9 +56,6 @@ module.exports = {
},
},
rules: {
// Asset migration — base-path safety
'rulesdir/no-unsupported-asset-pattern': 'error',
// Code quality rules
'prefer-const': 'error', // Enforces const for variables never reassigned
'no-var': 'error', // Disallows var, enforces let/const
@@ -214,25 +205,6 @@ module.exports = {
],
},
overrides: [
{
files: ['src/**/*.{jsx,tsx,ts}'],
excludedFiles: [
'**/*.test.{js,jsx,ts,tsx}',
'**/*.spec.{js,jsx,ts,tsx}',
'**/__tests__/**/*.{js,jsx,ts,tsx}',
],
rules: {
'no-restricted-properties': [
'error',
{
object: 'navigator',
property: 'clipboard',
message:
'Do not use navigator.clipboard directly since it does not work well with specific browsers. Use hook useCopyToClipboard from react-use library. https://streamich.github.io/react-use/?path=/story/side-effects-usecopytoclipboard--docs',
},
],
},
},
{
files: [
'**/*.test.{js,jsx,ts,tsx}',

View File

@@ -1,9 +0,0 @@
const path = require('path');
module.exports = {
plugins: [path.join(__dirname, 'stylelint-rules/no-unsupported-asset-url.js')],
customSyntax: 'postcss-scss',
rules: {
'local/no-unsupported-asset-url': true,
},
};

View File

@@ -12,7 +12,7 @@
or
`docker build . -t tagname`
**Tag to remote url- Introduce versioning later on**
**Tag to remote url- Introduce versinoing later on**
```
docker tag signoz/frontend:latest 7296823551/signoz:latest

View File

@@ -1 +0,0 @@
export default 'test-file-stub';

View File

@@ -2,7 +2,6 @@
interface SafeNavigateOptions {
replace?: boolean;
state?: unknown;
newTab?: boolean;
}
interface SafeNavigateTo {
@@ -21,7 +20,9 @@ interface UseSafeNavigateReturn {
export const useSafeNavigate = (): UseSafeNavigateReturn => ({
safeNavigate: jest.fn(
(_to: SafeNavigateToType, _options?: SafeNavigateOptions) => {},
(to: SafeNavigateToType, options?: SafeNavigateOptions) => {
console.log(`Mock safeNavigate called with:`, to, options);
},
) as jest.MockedFunction<
(to: SafeNavigateToType, options?: SafeNavigateOptions) => void
>,

View File

@@ -1,197 +0,0 @@
'use strict';
const {
hasAssetExtension,
containsAssetExtension,
extractUrlPath,
isAbsolutePath,
isPublicRelative,
} = require('./shared/asset-patterns');
const PUBLIC_DIR_SEGMENTS = ['/Icons/', '/Images/', '/Logos/', '/svgs/'];
function collectBinaryStringParts(node) {
if (node.type === 'Literal' && typeof node.value === 'string')
return [node.value];
if (node.type === 'BinaryExpression' && node.operator === '+') {
return [
...collectBinaryStringParts(node.left),
...collectBinaryStringParts(node.right),
];
}
if (node.type === 'TemplateLiteral') {
return node.quasis.map((q) => q.value.raw);
}
return [null];
}
module.exports = {
meta: {
type: 'problem',
docs: {
description:
'Disallow Vite-unsafe asset reference patterns that break runtime base-path deployments',
category: 'Asset Migration',
recommended: true,
},
schema: [],
messages: {
absoluteString:
'Absolute asset path "{{ value }}" is not base-path-safe. ' +
"Use an ES import instead: import fooUrl from '@/assets/...' and reference the variable.",
templateLiteral:
'Dynamic asset path with absolute prefix is not base-path-safe. ' +
"Use new URL('./asset.svg', import.meta.url).href for dynamic asset paths.",
absoluteImport:
'Asset imported via absolute path is not supported. ' +
"Use import fooUrl from '@/assets/...' instead.",
publicImport:
"Assets in public/ bypass Vite's module pipeline — their URLs are not base-path-aware and will break when the app is served from a sub-path (e.g. /app/). " +
"Use an ES import instead: import fooUrl from '@/assets/...' so Vite injects the correct base path.",
},
},
create(context) {
return {
Literal(node) {
if (node.parent && node.parent.type === 'ImportDeclaration') {
return;
}
const value = node.value;
if (typeof value !== 'string') return;
if (isAbsolutePath(value) && hasAssetExtension(value)) {
context.report({
node,
messageId: 'absoluteString',
data: { value },
});
return;
}
const urlPath = extractUrlPath(value);
if (urlPath && isAbsolutePath(urlPath) && hasAssetExtension(urlPath)) {
context.report({
node,
messageId: 'absoluteString',
data: { value: urlPath },
});
}
},
TemplateLiteral(node) {
const quasis = node.quasis;
if (!quasis || quasis.length === 0) return;
const firstQuasi = quasis[0].value.raw;
const hasAssetExt = quasis.some((q) => containsAssetExtension(q.value.raw));
if (isAbsolutePath(firstQuasi) && hasAssetExt) {
context.report({
node,
messageId: 'templateLiteral',
});
return;
}
// Expression-first template with known public-dir segment: `${base}/Icons/foo.svg`
const hasPublicSegment = quasis.some((q) =>
PUBLIC_DIR_SEGMENTS.some((seg) => q.value.raw.includes(seg)),
);
if (hasPublicSegment && hasAssetExt) {
context.report({
node,
messageId: 'templateLiteral',
});
return;
}
if (quasis.length === 1) {
const urlPath = extractUrlPath(firstQuasi);
if (urlPath && isAbsolutePath(urlPath) && hasAssetExtension(urlPath)) {
context.report({
node,
messageId: 'templateLiteral',
});
}
return;
}
if (firstQuasi.includes('url(') && hasAssetExt) {
const urlMatch = firstQuasi.match(/^url\(\s*['"]?\//);
if (urlMatch) {
context.report({
node,
messageId: 'templateLiteral',
});
}
}
},
// String concatenation: "/Icons/" + name + ".svg"
BinaryExpression(node) {
if (node.operator !== '+') return;
const parts = collectBinaryStringParts(node);
const prefixParts = [];
for (const part of parts) {
if (part === null) break;
prefixParts.push(part);
}
const staticPrefix = prefixParts.join('');
if (!isAbsolutePath(staticPrefix)) return;
const hasExt = parts.some(
(part) => part !== null && containsAssetExtension(part),
);
if (hasExt) {
context.report({
node,
messageId: 'templateLiteral',
});
}
},
ImportDeclaration(node) {
const src = node.source.value;
if (typeof src !== 'string') return;
if (isAbsolutePath(src) && hasAssetExtension(src)) {
context.report({
node,
messageId: 'absoluteImport',
});
return;
}
if (isPublicRelative(src) && hasAssetExtension(src)) {
context.report({
node,
messageId: 'publicImport',
});
}
},
ImportExpression(node) {
const src = node.source;
if (!src || src.type !== 'Literal' || typeof src.value !== 'string') return;
if (isAbsolutePath(src.value) && hasAssetExtension(src.value)) {
context.report({
node,
messageId: 'absoluteImport',
});
return;
}
if (isPublicRelative(src.value) && hasAssetExtension(src.value)) {
context.report({
node,
messageId: 'publicImport',
});
}
},
};
},
};

View File

@@ -1,3 +0,0 @@
{
"type": "commonjs"
}

View File

@@ -1,69 +0,0 @@
'use strict';
const ASSET_EXTENSIONS = ['.svg', '.png', '.webp', '.jpg', '.jpeg', '.gif'];
/**
* Returns true if the string ends with an asset extension.
* e.g. "/Icons/foo.svg" → true, "/Icons/foo.svg.bak" → false
*/
function hasAssetExtension(str) {
if (typeof str !== 'string') return false;
return ASSET_EXTENSIONS.some((ext) => str.endsWith(ext));
}
// Like hasAssetExtension but also matches mid-string with boundary check,
// e.g. "/foo.svg?v=1" → true, "/icons.svg-dir/" → false
function containsAssetExtension(str) {
if (typeof str !== 'string') return false;
return ASSET_EXTENSIONS.some((ext) => {
const idx = str.indexOf(ext);
if (idx === -1) return false;
const afterIdx = idx + ext.length;
// Accept if extension is at end of string or followed by non-alphanumeric char (?, #, /, etc.)
return afterIdx >= str.length || /[^a-zA-Z0-9]/.test(str[afterIdx]);
});
}
/**
* Extracts the asset path from a CSS url() wrapper.
* Handles single quotes, double quotes, unquoted, and whitespace variations.
* e.g.
* "url('/Icons/foo.svg')" → "/Icons/foo.svg"
* "url( '../assets/bg.png' )" → "../assets/bg.png"
* "url(/Icons/foo.svg)" → "/Icons/foo.svg"
* Returns null if the string is not a url() wrapper.
*/
function extractUrlPath(str) {
if (typeof str !== 'string') return null;
// Match url( [whitespace] [quote?] path [quote?] [whitespace] )
// Capture group: [^'")\s]+ matches path until quote, closing paren, or whitespace
const match = str.match(/^url\(\s*['"]?([^'")\s]+)['"]?\s*\)$/);
return match ? match[1] : null;
}
/**
* Returns true if the string is an absolute path (starts with /).
* Absolute paths in url() bypass <base href> and fail under any URL prefix.
*/
function isAbsolutePath(str) {
if (typeof str !== 'string') return false;
return str.startsWith('/') && !str.startsWith('//');
}
/**
* Returns true if the path imports from the public/ directory.
* Relative imports into public/ cause asset duplication in dist/.
*/
function isPublicRelative(str) {
if (typeof str !== 'string') return false;
return str.includes('/public/') || str.startsWith('public/');
}
module.exports = {
ASSET_EXTENSIONS,
hasAssetExtension,
containsAssetExtension,
extractUrlPath,
isAbsolutePath,
isPublicRelative,
};

View File

@@ -11,9 +11,6 @@ const config: Config.InitialOptions = {
moduleFileExtensions: ['ts', 'tsx', 'js', 'json'],
modulePathIgnorePatterns: ['dist'],
moduleNameMapper: {
'\\.(png|jpg|jpeg|gif|svg|webp|avif|ico|bmp|tiff)$':
'<rootDir>/__mocks__/fileMock.ts',
'^@/(.*)$': '<rootDir>/src/$1',
'\\.(css|less|scss)$': '<rootDir>/__mocks__/cssMock.ts',
'\\.md$': '<rootDir>/__mocks__/cssMock.ts',
'^uplot$': '<rootDir>/__mocks__/uplotMock.ts',

View File

@@ -10,10 +10,9 @@
"preview": "vite preview",
"prettify": "prettier --write .",
"fmt": "prettier --check .",
"lint": "eslint ./src && stylelint \"src/**/*.scss\"",
"lint": "eslint ./src",
"lint:generated": "eslint ./src/api/generated --fix",
"lint:fix": "eslint ./src --fix",
"lint:styles": "stylelint \"src/**/*.scss\"",
"jest": "jest",
"jest:coverage": "jest --coverage",
"jest:watch": "jest --watch",
@@ -69,8 +68,8 @@
"@signozhq/toggle-group": "0.0.1",
"@signozhq/tooltip": "0.0.2",
"@signozhq/ui": "0.0.5",
"@tanstack/react-table": "8.21.3",
"@tanstack/react-virtual": "3.13.22",
"@tanstack/react-table": "8.20.6",
"@tanstack/react-virtual": "3.11.2",
"@uiw/codemirror-theme-copilot": "4.23.11",
"@uiw/codemirror-theme-github": "4.24.1",
"@uiw/react-codemirror": "4.23.10",
@@ -165,7 +164,6 @@
"vite-plugin-html": "3.2.2",
"web-vitals": "^0.2.4",
"xstate": "^4.31.0",
"zod": "4.3.6",
"zustand": "5.0.11"
},
"browserslist": {
@@ -230,7 +228,6 @@
"eslint-plugin-prettier": "^4.0.0",
"eslint-plugin-react": "^7.24.0",
"eslint-plugin-react-hooks": "^4.3.0",
"eslint-plugin-rulesdir": "0.2.2",
"eslint-plugin-simple-import-sort": "^7.0.0",
"eslint-plugin-sonarjs": "^0.12.0",
"husky": "^7.0.4",
@@ -246,7 +243,6 @@
"orval": "7.18.0",
"portfinder-sync": "^0.0.2",
"postcss": "8.5.6",
"postcss-scss": "4.0.9",
"prettier": "2.2.1",
"prop-types": "15.8.1",
"react-hooks-testing-library": "0.6.0",
@@ -254,8 +250,6 @@
"redux-mock-store": "1.5.4",
"sass": "1.97.3",
"sharp": "0.34.5",
"stylelint": "17.7.0",
"stylelint-scss": "7.0.0",
"svgo": "4.0.0",
"ts-api-utils": "2.4.0",
"ts-jest": "29.4.6",
@@ -292,4 +286,4 @@
"tmp": "0.2.4",
"vite": "npm:rolldown-vite@7.3.1"
}
}
}

View File

Before

Width:  |  Height:  |  Size: 9.1 KiB

After

Width:  |  Height:  |  Size: 9.1 KiB

View File

Before

Width:  |  Height:  |  Size: 2.3 KiB

After

Width:  |  Height:  |  Size: 2.3 KiB

View File

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

View File

Before

Width:  |  Height:  |  Size: 2.9 KiB

After

Width:  |  Height:  |  Size: 2.9 KiB

View File

Before

Width:  |  Height:  |  Size: 507 B

After

Width:  |  Height:  |  Size: 507 B

View File

Before

Width:  |  Height:  |  Size: 9.0 KiB

After

Width:  |  Height:  |  Size: 9.0 KiB

View File

Before

Width:  |  Height:  |  Size: 644 B

After

Width:  |  Height:  |  Size: 644 B

View File

Before

Width:  |  Height:  |  Size: 5.2 KiB

After

Width:  |  Height:  |  Size: 5.2 KiB

View File

Before

Width:  |  Height:  |  Size: 1.6 KiB

After

Width:  |  Height:  |  Size: 1.6 KiB

View File

Before

Width:  |  Height:  |  Size: 4.5 KiB

After

Width:  |  Height:  |  Size: 4.5 KiB

View File

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

Before

Width:  |  Height:  |  Size: 2.0 KiB

After

Width:  |  Height:  |  Size: 2.0 KiB

View File

Before

Width:  |  Height:  |  Size: 5.2 KiB

After

Width:  |  Height:  |  Size: 5.2 KiB

View File

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 15 KiB

View File

Before

Width:  |  Height:  |  Size: 2.0 KiB

After

Width:  |  Height:  |  Size: 2.0 KiB

View File

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 5.9 KiB

View File

Before

Width:  |  Height:  |  Size: 5.6 KiB

After

Width:  |  Height:  |  Size: 5.6 KiB

View File

Before

Width:  |  Height:  |  Size: 1.2 KiB

After

Width:  |  Height:  |  Size: 1.2 KiB

View File

Before

Width:  |  Height:  |  Size: 418 B

After

Width:  |  Height:  |  Size: 418 B

View File

Before

Width:  |  Height:  |  Size: 604 B

After

Width:  |  Height:  |  Size: 604 B

View File

Before

Width:  |  Height:  |  Size: 878 B

After

Width:  |  Height:  |  Size: 878 B

View File

Before

Width:  |  Height:  |  Size: 7.3 KiB

After

Width:  |  Height:  |  Size: 7.3 KiB

View File

Before

Width:  |  Height:  |  Size: 7.5 KiB

After

Width:  |  Height:  |  Size: 7.5 KiB

View File

Before

Width:  |  Height:  |  Size: 3.2 KiB

After

Width:  |  Height:  |  Size: 3.2 KiB

View File

Before

Width:  |  Height:  |  Size: 6.1 KiB

After

Width:  |  Height:  |  Size: 6.1 KiB

View File

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 88 KiB

View File

Before

Width:  |  Height:  |  Size: 1.0 KiB

After

Width:  |  Height:  |  Size: 1.0 KiB

View File

Before

Width:  |  Height:  |  Size: 3.4 KiB

After

Width:  |  Height:  |  Size: 3.4 KiB

View File

Before

Width:  |  Height:  |  Size: 303 B

After

Width:  |  Height:  |  Size: 303 B

View File

Before

Width:  |  Height:  |  Size: 2.3 KiB

After

Width:  |  Height:  |  Size: 2.3 KiB

View File

Before

Width:  |  Height:  |  Size: 629 B

After

Width:  |  Height:  |  Size: 629 B

View File

Before

Width:  |  Height:  |  Size: 439 B

After

Width:  |  Height:  |  Size: 439 B

View File

Before

Width:  |  Height:  |  Size: 467 B

After

Width:  |  Height:  |  Size: 467 B

View File

Before

Width:  |  Height:  |  Size: 2.1 KiB

After

Width:  |  Height:  |  Size: 2.1 KiB

View File

Before

Width:  |  Height:  |  Size: 2.1 KiB

After

Width:  |  Height:  |  Size: 2.1 KiB

View File

Before

Width:  |  Height:  |  Size: 300 B

After

Width:  |  Height:  |  Size: 300 B

View File

Before

Width:  |  Height:  |  Size: 348 B

After

Width:  |  Height:  |  Size: 348 B

View File

Before

Width:  |  Height:  |  Size: 3.6 KiB

After

Width:  |  Height:  |  Size: 3.6 KiB

View File

Before

Width:  |  Height:  |  Size: 3.2 KiB

After

Width:  |  Height:  |  Size: 3.2 KiB

View File

Before

Width:  |  Height:  |  Size: 7.2 KiB

After

Width:  |  Height:  |  Size: 7.2 KiB

View File

Before

Width:  |  Height:  |  Size: 6.8 KiB

After

Width:  |  Height:  |  Size: 6.8 KiB

View File

Before

Width:  |  Height:  |  Size: 910 B

After

Width:  |  Height:  |  Size: 910 B

View File

Before

Width:  |  Height:  |  Size: 105 KiB

After

Width:  |  Height:  |  Size: 105 KiB

View File

Before

Width:  |  Height:  |  Size: 214 KiB

After

Width:  |  Height:  |  Size: 214 KiB

Some files were not shown because too many files have changed in this diff Show More