Commit Graph

25 Commits

Author SHA1 Message Date
Naman Verma
d013eab29d chore: migrate integration dashboards to v2 (#12242)
* chore: add migration script from current to perses dashboard

* fix: move WrapInV5Envelope to types package

* fix: generate internal name from title

* fix: match with lower case signal for variables

* fix: sanitize tags instead of throwing error

* fix: extract row and widget positions to build expanded sections

* chore: add a catch all panic check to log migration error

* chore: catch typecast errors

* fix: remove nil nil return

* test: fix ut to check for v2 internal name

* chore: remove spec md file

* chore: separate error type for migration

* feat: note down all errors in migration

* fix: allow zero value for threshold values

* fix: ignore react placeholders in layout

* fix: note error in query parsing

* fix: read threshold value properly during migration

* fix: adjust to schema changes in variables

* test: fix unit tests

* fix: catch error from decodeTelemetryFields

* fix: make threshold value non-nullable

* fix: handle list of widget IDs in panel maps

* fix: add better error note down for failed conversions

* fix: normalize having array from v4 schema

* fix: remove metric aggregation from non metric queries (malformed json)

* fix: normalize telemetry field keys in query

* chore: add temporary doc for other malformation handlers needed

* fix: add more malformed query handling

* fix: add more malformation handling

* chore: reuse transition package (adds dependency, need to see what to do)

* fix: fix go lint issue

* revert: remove md file

* fix: make none the default fill mode during migration

* fix: add nil check on limit value when normalizing page size

* fix: validate dashboard spec after migration

* fix: remove unused method

* fix: match shape safe version to existing version

* chore: improve variable names

* chore: better comments and names in layout migrator

* chore: add explanation comment for continuing on empty id widget

* fix: ditch dynamic vars with no attr, rearrange overlapping layouts

* fix: add default aggregation count() if none is present in logs/traces

* fix: add empty list tolerance to v1 variables

* fix: add fixes based on backup migration testing

* chore: skip over unreferenced widgets

* chore: remove widgets that have a single query that has no metric name

* chore: assign missing formula names like ui does

* chore: assign query names to queries if missing

* chore: drop legacy filter fields that ui also ignores

* chore: make migration drop widgets that have no aggregations for metrics

* chore: normalize group by

* chore: remove expressions from metric aggregations

* chore: add more error notes

* chore: replace em dash with hyphen in widget id

* chore: add more panel id sanitization

* chore: replace faulty order by keys with aggregation expression

* chore: fill valid span gaps

* chore: handle misformed threshold operators

* chore: remove repeat occurences of widgets in layouts

* chore: remove limits that cross max limit

* chore: restore function args malformed by v4->v5 migration

* chore: remove invalid functions

* chore: skip variables in wrong type

* chore: add wdiget id type check in layout migration

* chore: add a.count to malformed order by keys list

* chore: remove bad aggregations that accompany valid ones

* chore: drop trace operators with no expression

* chore: drop invalid formulas

* chore: skip unknown panel kind

* chore: drop widgets with >1 and all aggregations as malformed

* chore: skip empty type variables

* chore: pass threshold format raw without typecasting

* chore: add handling for trace operators around wrapinv5env

* chore: handle no string value for variable sort

* test: fix span gaps test

* chore: undo schema change in threshold value

* chore: final set of error fixes/hacks

* fix: fix build errors post merge conflicts

* test: remove test for floating pt value

* test: use appendf instead of sprintf

* chore: migrate integration dashboards to v2

* feat: add scaffolding to migrate dashboards

* fix: fill in ref path when building grid layout

* feat: handle v1 payloads in v2 create api

* fix: return empty objects instead of null if no variables/panels/layouts are there

* test: fix unit test for empty layouts

* fix: add empty links array in migration

* chore: rerun migration after links and signal change

* test: add unit test to show v5  takes precedence over v4

* fix: change EXISTS to exists so that migrateCommon handles it properly

* test: add unit test for count with attributes

* fix: run order by handling after default aggregation handling

* fix: restore formula fields that wrapinv5envelope removes

* fix: keep thresholds that do not have an optional label

* fix: force allowAll to be false if allowMultiple is false

* chore: rerun migration

* fix: migrate name column even if data column migration fails

* fix: use slog for logging

* refactor(dashboard): re-host ConvertAllV1ToV2 off the Module interface

Mirror base's removal of ConvertAllV1ToV2 from the shared files so future
base→branch merges stay conflict-free, and re-host the capability in dedicated
branch-only files (V1ToV2Migrator interface + module method + result types).

* refactor(dashboard): drop ConvertAllV1ToV2 module scaffolding from base

Remove the temporary v1→v2 bulk-conversion module method (ConvertAllV1ToV2 +
migrateOneV1ToV2), its result types (V1ToV2MigrationResult / V1ToV2MigrationItem),
the Module interface entry, and the EE delegation. This scaffolding lives on the
migration feature branches, not the base branch that merges to main.

V1Name and store.UpdateName are kept (small helpers; UpdateName is on the shared
Store interface).

* fix: add image validation and base64 to asset conversion

* fix: remove unneeded interface

* chore: rerun migration

* fix: make ReencodeInlineSVGImage so that migration script can call it

* chore: remove temp script

* fix: remove local laptop default value

---------

Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-07-26 14:15:54 +00:00
Swapnil Nakade
8c6a0de8c3 refactor: gcp integration memorystore redis dashboard update (#12229)
Some checks failed
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
2026-07-22 11:42:12 +00:00
Swapnil Nakade
842d44d874 feat: adding gcp memorystore redis service (#12118)
* feat: adding gcp memorystore redis service

* refactor: updating dashboard title

* refactor: extending width of uptime gauge panel

* refactor: updating cpu utilization panel

* refactor: updating dashboard panel to use rate function instead of hack
2026-07-22 07:08:26 +00:00
Swapnil Nakade
86313ae561 feat: clousql - adding metrics in definition and dashboard JSON (#12072)
* feat: adding metrics in definition and dashboard JSON

* refactor: renaming cloudsql to cloudsql_postgres

* refactor: generating openapi specs
2026-07-13 09:57:49 +00:00
Swapnil Nakade
028ac27496 feat: adding cloud integration API changes for GCP (#11892)
* feat: adding cloud integration API changes for GCP

* chore: generating openapi specs

* fix: integration tests

* ci: fixing golang ci lint
2026-06-30 17:13:53 +00:00
swapnil-signoz
5d711377ec feat: adding support for Azure DB services (#11679)
* feat: adding support for Azure SQL DB

* refactor: adding missing metrics in data collected and panels in dash

* feat: adding support for Azure SQL DB Managed Instances

* feat: generating openapi spec

* feat: adding nosql and cache services in azure integration
2026-06-16 14:20:49 +00:00
swapnil-signoz
73c2c15200 feat: adding support for VMs in Azure integration (#11573)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat: adding supoprt for virtual machines to azure one click integration

* chore: generating openapi spec

* chore: updating dashboard title
2026-06-09 15:12:57 +00:00
swapnil-signoz
1ba9b90855 feat: adding assets and dashboards for azure app services (#11565)
* feat: adding assets and dashboards for azure app services

* refactor: updating metric names

* chore: openapi spec changes

* chore: generating frontend types
2026-06-09 14:30:53 +00:00
swapnil-signoz
922800ff98 feat: adding support for Azure Container Apps (#11580) 2026-06-09 09:58:26 +00:00
swapnil-signoz
b072095a9d feat: adding support for Azure AKS (#11615)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat: adding support for Azure AKS

* refactor: updating metric names in integration config

* chore: generating OpenAPI spec

* chore: adding note in overview

* refactor: updating spatial aggregation and other names

* refactor: updating reduceTo to empty value

* refactor: updating title
2026-06-09 09:13:38 +00:00
swapnil-signoz
b3b245ebc5 feat: adding get cloud integration service for account handler (#11497)
* feat: adding get cloud integration service for account handler

* feat: adding integration test for new API endpoint

* ci: fix py fmt lint

* refactor: cloudfront integration service (#11570)

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>

* fix: update tests

---------

Co-authored-by: Gaurav Tewari <gauravtewari111@gmail.com>
Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-06-05 12:42:18 +00:00
swapnil-signoz
ce26458b9f feat(cloud-integrations): adding endpoint services metadata for account (#11563)
* feat: adding endpoint services metadata for account

* fix: adding missing response writer in error

* refactor(cloud-integrations): use account-scoped  ListAccountServices endpoint when an account is connected (#11569)

* fix: frontend changes for issue 4616

* fix: update frontend changes

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>

---------

Co-authored-by: Gaurav Tewari <gauravtewari111@gmail.com>
Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-06-04 18:07:01 +00:00
swapnil-signoz
7932917f5f refactor: service response changes and dashboardID handling (#11485)
* refactor: service response changes and dashboardID handling

* refactor: removing unused enrichDashboardIDs func

* revert: restore frontend files to main state

* refactor: updating response

* refactor: updating nullable tag

* chore: adding required tags

* chore: adding required tags for ServiceDashboard struct

* refactor(integrations): align service-details UI with new cloud-integration API + disabled-dashboard state (#11547)

* refactor: cloud integration service

* fix: update schemas

* fix: update schema

* fix: minor fixes

* refactor: review comments

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>

---------

Co-authored-by: Gaurav Tewari <gauravtewari111@gmail.com>
Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-06-04 17:32:56 +00:00
swapnil-signoz
832930239e refactor: cloud integration dashboards migration to DB (#11382)
Some checks failed
build-staging / prepare (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
* chore: added migration setup

* feat(sqlmigration): add integration_dashboards table (migration 079)

Adds the `integration_dashboards` relations table that stores the
integration-specific identity for dashboards provisioned from cloud
or builtin integrations. Columns: id, org_id, dashboard_id, provider,
slug, created_at, updated_at. Includes a unique index on dashboard_id.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(sqlmigration): backfill cloud integration dashboards to DB (migration 080)

One-time idempotent migration that provisions dashboard rows for all
orgs with existing cloud integration services where metrics are enabled.
Each dashboard is inserted into the `dashboard` table with
source="integration" and locked=true, and a companion row is added to
`integration_dashboards` with provider="cloud_integrations" and
slug="{provider}-{service}-{dashboard}" (e.g. aws-alb-overview).
Idempotency is enforced by checking (org_id, provider, slug) on
integration_dashboards before each insert.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(sqlmigration): clean up stale 079 artifacts, add 079 schema migration

Remove the pre-rename 079_migrate_cloud_integration_dashboards.go and
079_cloud_integration_dashboards/ directory that were left behind when
the backfill migration was renumbered to 080. Add the missing
079_add_integration_dashboards.go (schema-only migration creating the
integration_dashboards table) which provider.go already references.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: adding comment for fk

* refactor: renaming table name

* refactor: rename and restructure cloud integration dashboard migration types

* chore: file rename

* refactor: dashboard creation and listing flow change

* refactor: removing loose strings

* refactor: adding DeleteBySource on dashboard module

* refactor: review changes and update service flow change

* refactor: simplify comments

* ci: lint staticcheck fix

* refactor: renaming migration and adding integration tests

* ci: py fmt lint fixes

* feat: adding ListSharedServices store method

* ci: golangci-lint fix

* refactor: code cleanup

* chore: revert changed due to js lint

* refactor: test assertion changes

* refactor: using bindparam for sql generation

* chore: migrate integration dashboards json to v5 (#11419)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
2026-05-23 03:05:06 +00:00
swapnil-signoz
69212ef9b1 feat(cloudintegrations): add azure dashboards (#11094)
* refactor: moving types to cloud provider specific namespace/pkg

* refactor: separating cloud provider types

* refactor: using upper case key for AWS

* feat: adding cloud integration azure types

* feat: adding azure services

* refactor: updating omitempty tags

* refactor: updating azure integration config

* feat: completing azure types

* refactor: lint issues

* feat: adding service definitions for azure

* refactor: update service names for Azure Blob Storage telemetry

* refactor: updating definitions with metrics and strategy

* refactor: updating command key

* fix: handle optional connection URL in AWS integration

* feat: wip

* refactor: updating strategy struct

* refactor: updating telemetry strategy

* refactor: updating connection artifact struct

* refactor: updating blob storage service name

* refactor: updating azure blob storage service name

* refactor: update Azure service identifiers

* refactor: updating service defs

* fix: update integration account ID and add agent version to Azure CLI and PowerShell commands

* refactor: updating deny settings mode

* refactor: updating types

* refactor: adding missing case for azure service update

* feat: implement Azure connection commands and add unit tests

* refactor: using template for Azure connection artifact creation and update tests

* feat: adding dashboards for azure

* refactor: updated aggregrations

* refactor: updating aggregations
2026-04-28 17:46:29 +00:00
swapnil-signoz
97885babe8 feat: adding cloud integration implementation details for Azure (#11058)
* refactor: moving types to cloud provider specific namespace/pkg

* refactor: separating cloud provider types

* refactor: using upper case key for AWS

* feat: adding cloud integration azure types

* feat: adding azure services

* refactor: updating omitempty tags

* refactor: updating azure integration config

* feat: completing azure types

* refactor: lint issues

* feat: adding service definitions for azure

* refactor: update service names for Azure Blob Storage telemetry

* refactor: updating definitions with metrics and strategy

* refactor: updating command key

* fix: handle optional connection URL in AWS integration

* feat: wip

* refactor: updating strategy struct

* refactor: updating telemetry strategy

* refactor: updating connection artifact struct

* refactor: updating blob storage service name

* refactor: updating azure blob storage service name

* refactor: update Azure service identifiers

* refactor: updating service defs

* fix: update integration account ID and add agent version to Azure CLI and PowerShell commands

* refactor: updating deny settings mode

* refactor: updating types

* refactor: adding missing case for azure service update

* feat: implement Azure connection commands and add unit tests

* refactor: using template for Azure connection artifact creation and update tests
2026-04-27 11:23:52 +00:00
swapnil-signoz
04552fa2e9 feat: cloud integration azure service definitions (#11006)
* refactor: moving types to cloud provider specific namespace/pkg

* refactor: separating cloud provider types

* refactor: using upper case key for AWS

* feat: adding cloud integration azure types

* feat: adding azure services

* refactor: updating omitempty tags

* refactor: updating azure integration config

* feat: completing azure types

* refactor: lint issues

* feat: adding service definitions for azure

* refactor: update service names for Azure Blob Storage telemetry

* refactor: updating definitions with metrics and strategy

* refactor: updating command key

* fix: handle optional connection URL in AWS integration

* refactor: updating strategy struct

* refactor: updating telemetry strategy

* refactor: updating blob storage service name

* refactor: updating azure blob storage service name

* refactor: update Azure service identifiers

* refactor: updating service defs

* refactor: updating types
2026-04-23 18:42:10 +00:00
swapnil-signoz
21ce7a663a feat: adding cloud integration azure types and openapi spec (#11005)
* refactor: moving types to cloud provider specific namespace/pkg

* refactor: separating cloud provider types

* refactor: using upper case key for AWS

* feat: adding cloud integration azure types

* feat: adding azure services

* refactor: updating omitempty tags

* refactor: updating azure integration config

* feat: completing azure types

* refactor: lint issues

* refactor: updating command key

* fix: handle optional connection URL in AWS integration

* refactor: updating strategy struct

* refactor: updating azure blob storage service name

* refactor: update Azure service identifiers

* refactor: updating types
2026-04-23 14:48:04 +00:00
swapnil-signoz
3df0da3a4e chore: removing underscore attribute based cloud integration dashboards (#10929)
Some checks failed
build-staging / js-build (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
2026-04-14 13:08:07 +00:00
swapnil-signoz
dce496d099 refactor: cloud integration modules implementation (#10718)
* feat: adding cloud integration type for refactor

* refactor: store interfaces to use local types and error

* feat: adding sql store implementation

* refactor: removing interface check

* feat: adding updated types for cloud integration

* refactor: using struct for map

* refactor: update cloud integration types and module interface

* fix: correct GetService signature and remove shadowed Data field

* feat: implement cloud integration store

* refactor: adding comments and removed wrong code

* refactor: streamlining types

* refactor: add comments for backward compatibility in PostableAgentCheckInRequest

* refactor: update Dashboard struct comments and remove unused fields

* refactor: split upsert store method

* feat: adding integration test

* refactor: clean up types

* refactor: renaming service type to service id

* refactor: using serviceID type

* feat: adding method for service id creation

* refactor: updating store methods

* refactor: clean up

* refactor: clean up

* refactor: review comments

* refactor: clean up

* feat: adding handlers

* fix: lint and ci issues

* fix: lint issues

* fix: update error code for service not found

* feat: adding handler skeleton

* chore: removing todo comment

* feat: adding frontend openapi schema

* feat: adding module implementation for create account

* fix: returning valid error instead of panic

* fix: module test

* refactor: simplify ingestion key retrieval logic

* feat: adding module implementation for AWS

* refactor: ci lint changes

* refactor: python formatting change

* fix: new storable account func was unsetting provider account id

* refactor: python lint changes

* refactor: adding validation on update account request

* refactor: reverting older tests and adding new tests

* chore: lint changes

* feat: using service account for API key

* refactor: renaming tests and cleanup

* refactor: removing dashboard overview images

* feat: adding service definition store

* chore: adding TODO comments

* feat: adding API for getting connection credentials

* feat: adding openapi spec for the endpoint

* feat: adding tests for credential API

* feat: adding cloud integration config

* refactor: updating test with new env variable for config

* refactor: moving few cloud provider interface methods to types

* refactor: review comments resolution

* refactor: lint changes

* refactor: code clean up

* refactor: removing email domain function

* refactor: review comments and clean up

* refactor: lint fixes

* refactor: review changes

- Added get connected account module method
- Fixed integration tests
- Removed cloud integration store as callback function's param

* refactor: changing wrong dashboard id for EKS definition
2026-04-13 15:16:26 +00:00
swapnil-signoz
92660b457d feat: adding types changes and openapi spec (#10866)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* feat: adding types changes and openapi spec

* refactor: review changes

* feat: generating OpenAPI spec

* refactor: updating create account types

* refactor: removing email domain function
2026-04-08 20:11:49 +00:00
swapnil-signoz
4f273b296e refactor: removing dashboard overview images (#10801)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
2026-04-01 16:25:55 +00:00
swapnil-signoz
3dc0a7c8ce feat: adding aws service definitions in types (#10798)
* feat: adding aws service definitions in types

* refactor: moving definitions to module fs
2026-04-01 14:08:46 +00:00
swapnil-signoz
4b09f057b9 feat: adding handlers with OpenAPI specs (#10643)
* feat: adding cloud integration type for refactor

* refactor: store interfaces to use local types and error

* feat: adding sql store implementation

* refactor: removing interface check

* feat: adding updated types for cloud integration

* refactor: using struct for map

* refactor: update cloud integration types and module interface

* fix: correct GetService signature and remove shadowed Data field

* feat: implement cloud integration store

* refactor: adding comments and removed wrong code

* refactor: streamlining types

* refactor: add comments for backward compatibility in PostableAgentCheckInRequest

* refactor: update Dashboard struct comments and remove unused fields

* refactor: split upsert store method

* feat: adding integration test

* refactor: clean up types

* refactor: renaming service type to service id

* refactor: using serviceID type

* feat: adding method for service id creation

* refactor: updating store methods

* refactor: clean up

* refactor: clean up

* refactor: review comments

* refactor: clean up

* feat: adding handlers

* fix: lint and ci issues

* fix: lint issues

* fix: update error code for service not found

* feat: adding handler skeleton

* chore: removing todo comment

* feat: adding frontend openapi schema

* refactor: making review changes

* feat: regenerating openapi specs
2026-03-24 10:24:38 +00:00
swapnil-signoz
866e541e29 refactor: cloud integration store implementation (#10469)
* feat: adding cloud integration type for refactor

* refactor: store interfaces to use local types and error

* feat: adding sql store implementation

* refactor: removing interface check

* feat: adding updated types for cloud integration

* refactor: using struct for map

* refactor: update cloud integration types and module interface

* fix: correct GetService signature and remove shadowed Data field

* feat: implement cloud integration store

* refactor: adding comments and removed wrong code

* refactor: streamlining types

* refactor: add comments for backward compatibility in PostableAgentCheckInRequest

* refactor: update Dashboard struct comments and remove unused fields

* refactor: split upsert store method

* feat: adding integration test

* refactor: clean up types

* refactor: renaming service type to service id

* refactor: using serviceID type

* feat: adding method for service id creation

* refactor: updating store methods

* refactor: clean up

* refactor: clean up

* refactor: review comments

* refactor: clean up

* fix: update error code for service not found

* refactor: returning error only for create methods

* refactor: method chaining formatting
2026-03-23 14:25:32 +00:00