mirror of
https://github.com/SigNoz/signoz.git
synced 2026-08-06 13:10:40 +01:00
ns/scope
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Naman Verma
|
d013eab29d |
chore: migrate integration dashboards to v2 (#12242)
* chore: add migration script from current to perses dashboard * fix: move WrapInV5Envelope to types package * fix: generate internal name from title * fix: match with lower case signal for variables * fix: sanitize tags instead of throwing error * fix: extract row and widget positions to build expanded sections * chore: add a catch all panic check to log migration error * chore: catch typecast errors * fix: remove nil nil return * test: fix ut to check for v2 internal name * chore: remove spec md file * chore: separate error type for migration * feat: note down all errors in migration * fix: allow zero value for threshold values * fix: ignore react placeholders in layout * fix: note error in query parsing * fix: read threshold value properly during migration * fix: adjust to schema changes in variables * test: fix unit tests * fix: catch error from decodeTelemetryFields * fix: make threshold value non-nullable * fix: handle list of widget IDs in panel maps * fix: add better error note down for failed conversions * fix: normalize having array from v4 schema * fix: remove metric aggregation from non metric queries (malformed json) * fix: normalize telemetry field keys in query * chore: add temporary doc for other malformation handlers needed * fix: add more malformed query handling * fix: add more malformation handling * chore: reuse transition package (adds dependency, need to see what to do) * fix: fix go lint issue * revert: remove md file * fix: make none the default fill mode during migration * fix: add nil check on limit value when normalizing page size * fix: validate dashboard spec after migration * fix: remove unused method * fix: match shape safe version to existing version * chore: improve variable names * chore: better comments and names in layout migrator * chore: add explanation comment for continuing on empty id widget * fix: ditch dynamic vars with no attr, rearrange overlapping layouts * fix: add default aggregation count() if none is present in logs/traces * fix: add empty list tolerance to v1 variables * fix: add fixes based on backup migration testing * chore: skip over unreferenced widgets * chore: remove widgets that have a single query that has no metric name * chore: assign missing formula names like ui does * chore: assign query names to queries if missing * chore: drop legacy filter fields that ui also ignores * chore: make migration drop widgets that have no aggregations for metrics * chore: normalize group by * chore: remove expressions from metric aggregations * chore: add more error notes * chore: replace em dash with hyphen in widget id * chore: add more panel id sanitization * chore: replace faulty order by keys with aggregation expression * chore: fill valid span gaps * chore: handle misformed threshold operators * chore: remove repeat occurences of widgets in layouts * chore: remove limits that cross max limit * chore: restore function args malformed by v4->v5 migration * chore: remove invalid functions * chore: skip variables in wrong type * chore: add wdiget id type check in layout migration * chore: add a.count to malformed order by keys list * chore: remove bad aggregations that accompany valid ones * chore: drop trace operators with no expression * chore: drop invalid formulas * chore: skip unknown panel kind * chore: drop widgets with >1 and all aggregations as malformed * chore: skip empty type variables * chore: pass threshold format raw without typecasting * chore: add handling for trace operators around wrapinv5env * chore: handle no string value for variable sort * test: fix span gaps test * chore: undo schema change in threshold value * chore: final set of error fixes/hacks * fix: fix build errors post merge conflicts * test: remove test for floating pt value * test: use appendf instead of sprintf * chore: migrate integration dashboards to v2 * feat: add scaffolding to migrate dashboards * fix: fill in ref path when building grid layout * feat: handle v1 payloads in v2 create api * fix: return empty objects instead of null if no variables/panels/layouts are there * test: fix unit test for empty layouts * fix: add empty links array in migration * chore: rerun migration after links and signal change * test: add unit test to show v5 takes precedence over v4 * fix: change EXISTS to exists so that migrateCommon handles it properly * test: add unit test for count with attributes * fix: run order by handling after default aggregation handling * fix: restore formula fields that wrapinv5envelope removes * fix: keep thresholds that do not have an optional label * fix: force allowAll to be false if allowMultiple is false * chore: rerun migration * fix: migrate name column even if data column migration fails * fix: use slog for logging * refactor(dashboard): re-host ConvertAllV1ToV2 off the Module interface Mirror base's removal of ConvertAllV1ToV2 from the shared files so future base→branch merges stay conflict-free, and re-host the capability in dedicated branch-only files (V1ToV2Migrator interface + module method + result types). * refactor(dashboard): drop ConvertAllV1ToV2 module scaffolding from base Remove the temporary v1→v2 bulk-conversion module method (ConvertAllV1ToV2 + migrateOneV1ToV2), its result types (V1ToV2MigrationResult / V1ToV2MigrationItem), the Module interface entry, and the EE delegation. This scaffolding lives on the migration feature branches, not the base branch that merges to main. V1Name and store.UpdateName are kept (small helpers; UpdateName is on the shared Store interface). * fix: add image validation and base64 to asset conversion * fix: remove unneeded interface * chore: rerun migration * fix: make ReencodeInlineSVGImage so that migration script can call it * chore: remove temp script * fix: remove local laptop default value --------- Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com> |
||
|
Vikrant Gupta
|
1644e35d9c |
feat(authz): make serviceaccount_role and user_role first class citizens (#11977)
* feat(authz): add serviceaccount and user role handlers * feat(authz): add support for role and service account extractors * feat(authz): make the user apis v2 * feat(authz): return the existing service account role id |
||
|
Swapnil Nakade
|
028ac27496 |
feat: adding cloud integration API changes for GCP (#11892)
* feat: adding cloud integration API changes for GCP * chore: generating openapi specs * fix: integration tests * ci: fixing golang ci lint |
||
|
swapnil-signoz
|
7932917f5f |
refactor: service response changes and dashboardID handling (#11485)
* refactor: service response changes and dashboardID handling * refactor: removing unused enrichDashboardIDs func * revert: restore frontend files to main state * refactor: updating response * refactor: updating nullable tag * chore: adding required tags * chore: adding required tags for ServiceDashboard struct * refactor(integrations): align service-details UI with new cloud-integration API + disabled-dashboard state (#11547) * refactor: cloud integration service * fix: update schemas * fix: update schema * fix: minor fixes * refactor: review comments --------- Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com> --------- Co-authored-by: Gaurav Tewari <gauravtewari111@gmail.com> Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com> |
||
|
swapnil-signoz
|
0671c5f416 |
feat: installed integration dashboards migration to DB (#11415)
* chore: added migration setup * feat(sqlmigration): add integration_dashboards table (migration 079) Adds the `integration_dashboards` relations table that stores the integration-specific identity for dashboards provisioned from cloud or builtin integrations. Columns: id, org_id, dashboard_id, provider, slug, created_at, updated_at. Includes a unique index on dashboard_id. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(sqlmigration): backfill cloud integration dashboards to DB (migration 080) One-time idempotent migration that provisions dashboard rows for all orgs with existing cloud integration services where metrics are enabled. Each dashboard is inserted into the `dashboard` table with source="integration" and locked=true, and a companion row is added to `integration_dashboards` with provider="cloud_integrations" and slug="{provider}-{service}-{dashboard}" (e.g. aws-alb-overview). Idempotency is enforced by checking (org_id, provider, slug) on integration_dashboards before each insert. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(sqlmigration): clean up stale 079 artifacts, add 079 schema migration Remove the pre-rename 079_migrate_cloud_integration_dashboards.go and 079_cloud_integration_dashboards/ directory that were left behind when the backfill migration was renumbered to 080. Add the missing 079_add_integration_dashboards.go (schema-only migration creating the integration_dashboards table) which provider.go already references. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: adding comment for fk * refactor: renaming table name * refactor: rename and restructure cloud integration dashboard migration types * chore: file rename * refactor: dashboard creation and listing flow change * refactor: removing loose strings * refactor: adding DeleteBySource on dashboard module * refactor: review changes and update service flow change * refactor: simplify comments * ci: lint staticcheck fix * refactor: renaming migration and adding integration tests * ci: py fmt lint fixes * feat: adding ListSharedServices store method * ci: golangci-lint fix * feat(integrations): persist installed integration dashboards in DB Provisions dashboard DB rows when an integration is installed and deprovisions them on uninstall. Adds a backfill migration (087) for users with already-installed integrations. Removes the on-the-fly filesystem serving path from http_handler in favor of the standard dashboard module. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor: changing dashboard ID and other cleanup * chore: update code structure for better readability and maintainability * refactor: removing deprecated cloud integrations and merging integration types * refactor: renaming migration files and removing deprecated tests * refactor: using BunDBCtx method instead * ci: fix py fmt lint --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
swapnil-signoz
|
832930239e |
refactor: cloud integration dashboards migration to DB (#11382)
Some checks failed
build-staging / prepare (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
* chore: added migration setup * feat(sqlmigration): add integration_dashboards table (migration 079) Adds the `integration_dashboards` relations table that stores the integration-specific identity for dashboards provisioned from cloud or builtin integrations. Columns: id, org_id, dashboard_id, provider, slug, created_at, updated_at. Includes a unique index on dashboard_id. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(sqlmigration): backfill cloud integration dashboards to DB (migration 080) One-time idempotent migration that provisions dashboard rows for all orgs with existing cloud integration services where metrics are enabled. Each dashboard is inserted into the `dashboard` table with source="integration" and locked=true, and a companion row is added to `integration_dashboards` with provider="cloud_integrations" and slug="{provider}-{service}-{dashboard}" (e.g. aws-alb-overview). Idempotency is enforced by checking (org_id, provider, slug) on integration_dashboards before each insert. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(sqlmigration): clean up stale 079 artifacts, add 079 schema migration Remove the pre-rename 079_migrate_cloud_integration_dashboards.go and 079_cloud_integration_dashboards/ directory that were left behind when the backfill migration was renumbered to 080. Add the missing 079_add_integration_dashboards.go (schema-only migration creating the integration_dashboards table) which provider.go already references. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: adding comment for fk * refactor: renaming table name * refactor: rename and restructure cloud integration dashboard migration types * chore: file rename * refactor: dashboard creation and listing flow change * refactor: removing loose strings * refactor: adding DeleteBySource on dashboard module * refactor: review changes and update service flow change * refactor: simplify comments * ci: lint staticcheck fix * refactor: renaming migration and adding integration tests * ci: py fmt lint fixes * feat: adding ListSharedServices store method * ci: golangci-lint fix * refactor: code cleanup * chore: revert changed due to js lint * refactor: test assertion changes * refactor: using bindparam for sql generation * chore: migrate integration dashboards json to v5 (#11419) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com> |
||
|
swapnil-signoz
|
97885babe8 |
feat: adding cloud integration implementation details for Azure (#11058)
* refactor: moving types to cloud provider specific namespace/pkg * refactor: separating cloud provider types * refactor: using upper case key for AWS * feat: adding cloud integration azure types * feat: adding azure services * refactor: updating omitempty tags * refactor: updating azure integration config * feat: completing azure types * refactor: lint issues * feat: adding service definitions for azure * refactor: update service names for Azure Blob Storage telemetry * refactor: updating definitions with metrics and strategy * refactor: updating command key * fix: handle optional connection URL in AWS integration * feat: wip * refactor: updating strategy struct * refactor: updating telemetry strategy * refactor: updating connection artifact struct * refactor: updating blob storage service name * refactor: updating azure blob storage service name * refactor: update Azure service identifiers * refactor: updating service defs * fix: update integration account ID and add agent version to Azure CLI and PowerShell commands * refactor: updating deny settings mode * refactor: updating types * refactor: adding missing case for azure service update * feat: implement Azure connection commands and add unit tests * refactor: using template for Azure connection artifact creation and update tests |
||
|
swapnil-signoz
|
a9b458f1f6 |
refactor: moving types to cloud provider specific namespace/pkg (#10976)
* refactor: moving types to cloud provider specific namespace/pkg * refactor: separating cloud provider types * refactor: using upper case key for AWS |
||
|
swapnil-signoz
|
dce496d099 |
refactor: cloud integration modules implementation (#10718)
* feat: adding cloud integration type for refactor * refactor: store interfaces to use local types and error * feat: adding sql store implementation * refactor: removing interface check * feat: adding updated types for cloud integration * refactor: using struct for map * refactor: update cloud integration types and module interface * fix: correct GetService signature and remove shadowed Data field * feat: implement cloud integration store * refactor: adding comments and removed wrong code * refactor: streamlining types * refactor: add comments for backward compatibility in PostableAgentCheckInRequest * refactor: update Dashboard struct comments and remove unused fields * refactor: split upsert store method * feat: adding integration test * refactor: clean up types * refactor: renaming service type to service id * refactor: using serviceID type * feat: adding method for service id creation * refactor: updating store methods * refactor: clean up * refactor: clean up * refactor: review comments * refactor: clean up * feat: adding handlers * fix: lint and ci issues * fix: lint issues * fix: update error code for service not found * feat: adding handler skeleton * chore: removing todo comment * feat: adding frontend openapi schema * feat: adding module implementation for create account * fix: returning valid error instead of panic * fix: module test * refactor: simplify ingestion key retrieval logic * feat: adding module implementation for AWS * refactor: ci lint changes * refactor: python formatting change * fix: new storable account func was unsetting provider account id * refactor: python lint changes * refactor: adding validation on update account request * refactor: reverting older tests and adding new tests * chore: lint changes * feat: using service account for API key * refactor: renaming tests and cleanup * refactor: removing dashboard overview images * feat: adding service definition store * chore: adding TODO comments * feat: adding API for getting connection credentials * feat: adding openapi spec for the endpoint * feat: adding tests for credential API * feat: adding cloud integration config * refactor: updating test with new env variable for config * refactor: moving few cloud provider interface methods to types * refactor: review comments resolution * refactor: lint changes * refactor: code clean up * refactor: removing email domain function * refactor: review comments and clean up * refactor: lint fixes * refactor: review changes - Added get connected account module method - Fixed integration tests - Removed cloud integration store as callback function's param * refactor: changing wrong dashboard id for EKS definition |