Compare commits

..

5 Commits

Author SHA1 Message Date
Vikrant Gupta
e69d3fcc8c fix(metricreduction): type the no-kept-labels preview count so the driver can scan it (#13221)
Some checks are pending
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
cacheci / tests (push) Waiting to run
Release Drafter / update_release_draft (push) Waiting to run
#### Description

- Previewing a rule that drops every non-protected label of a metric
returned 500 "failed to estimate reduction impact". With no kept labels
the cardinality query selects a bare `1`, which ClickHouse types as
`UInt8`, and the driver refuses to scan that into the `uint64` the code
reads it into.
- The literal is now `toUInt64(1)`, matching the scan target, so such
previews report one retained series as intended.
2026-10-21 12:33:51 +00:00
Naman Verma
354214ca87 chore: drop sloglint key-naming-case (#13220)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description

Dotted keys like rule.id are already the convention across the codebase.
2026-10-21 12:04:26 +00:00
Vikrant Gupta
e3488c1592 feat(authz): enable FGA for metrics explorer routes (#13150)
#### Description

- Metrics explorer routes were still on the coarse `ViewAccess` gate, so
a custom role scoped to a workspace key on metrics could list, inspect
and read stats for every metric through them while being denied on
`query_range`.
- Stats, treemap, inspect, highlights and onboarding derive the resource
from the caller's `filter.expression` or resolve to the metrics
wildcard, exactly as `query_range` does. Key-scoped custom roles get 403
unless the filter pins a granted key.
- A new `metrics-field` metaresource covers the metrics list, the
metadata and attributes reads, and the metadata update, with wildcard
grants: admin and editor may update, every managed role may list and
read. Existing organizations get the tuples through a migration.
- The alerts and dashboards lookups read the rule and dashboard stores,
so they require `rule:list` and `dashboard:list`.
- The new `metricsexplorer` integration suite is added to the CI matrix.

#### Issues closed by this PR

Part of SigNoz/keystone-pod#141
2026-10-21 11:40:56 +00:00
Swapnil Nakade
574bfa62b8 feat: enabled logs and added collection strategy for gcp integration (#13184)
<!--A few plain bullets saying what changed and why, for a reviewer
skimming it - not a wall of text, not a restatement of the diff, not
generated boilerplate.-->
#### Description
- with release of 0.163.0 of collector contrib, `googlecloudpubsubpush`
receiver is published
- enabling logs services in gcp integration for manual flow

<!--Reference issues using `Closes #issue-number` to enable automatic
closure on merge. -->
#### Issues closed by this PR
Contributes to https://github.com/SigNoz/keystone-pod/issues/164

Co-authored-by: Vikrant Gupta <vikrant@signoz.io>
2026-10-21 11:40:25 +00:00
Vikrant Gupta
ba6c4f83a6 docs(authz): explain telemetry resources versus metaresources (#13186)
Some checks failed
build-staging / staging (push) Has been cancelled
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
cacheci / tests (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
#### Description

- The authz guide listed the `telemetryresource` and `metaresource`
types but never said when a route belongs to which, so new routes have
been landing on role gates or the wrong type.
- Adds a section contrasting the two: telemetry resources are the data,
read-only, with the selector derived from the query; metaresources are
objects about the data with CRUD verbs and ID or wildcard selectors. The
metrics explorer is the worked example, with `metrics:read` for stats
against `metrics-field:list` and `metrics-field:update` for the
catalogue and metadata write.
- Adds the rule of thumb to the summary list.

#### Additional Information

Contributes to: https://github.com/SigNoz/keystone-pod/issues/141
2026-10-21 08:10:56 +00:00
28 changed files with 912 additions and 114 deletions

View File

@@ -75,6 +75,7 @@ jobs:
- ttl
- clickhousecluster
- metricreduction
- metricsexplorer
sqlstore-provider:
- postgres
- sqlite

View File

@@ -41,7 +41,6 @@ linters:
no-global: all
context: all
static-msg: true
key-naming-case: snake
exclusions:
generated: lax
presets:

View File

@@ -3581,6 +3581,7 @@ components:
- meter-metrics
- logs-field
- traces-field
- metrics-field
type: string
CoretypesObject:
properties:
@@ -20637,9 +20638,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics-field:list
- tokenizer:
- VIEWER
- metrics-field:list
summary: List metric names
tags:
- metrics
@@ -20706,9 +20707,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- rule:list
- tokenizer:
- VIEWER
- rule:list
summary: Get metric alerts
tags:
- metrics
@@ -20790,9 +20791,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics-field:read
- tokenizer:
- VIEWER
- metrics-field:read
summary: Get metric attributes
tags:
- metrics
@@ -20860,9 +20861,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics:read
- tokenizer:
- VIEWER
- metrics:read
summary: Get metric highlights
tags:
- metrics
@@ -20919,9 +20920,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics:read
- tokenizer:
- VIEWER
- metrics:read
summary: Inspect raw metric data points
tags:
- metrics
@@ -20989,9 +20990,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics-field:read
- tokenizer:
- VIEWER
- metrics-field:read
summary: Get metric metadata
tags:
- metrics
@@ -21035,9 +21036,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- EDITOR
- metrics-field:update
- tokenizer:
- EDITOR
- metrics-field:update
summary: Update metric metadata
tags:
- metrics
@@ -21083,9 +21084,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics:read
- tokenizer:
- VIEWER
- metrics:read
summary: Check if non-SigNoz metrics have been received
tags:
- metrics
@@ -21142,9 +21143,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics:read
- tokenizer:
- VIEWER
- metrics:read
summary: Get metrics statistics
tags:
- metrics
@@ -21201,9 +21202,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- metrics:read
- tokenizer:
- VIEWER
- metrics:read
summary: Get metrics treemap
tags:
- metrics
@@ -28342,9 +28343,9 @@ paths:
description: Internal Server Error
security:
- api_key:
- VIEWER
- dashboard:list
- tokenizer:
- VIEWER
- dashboard:list
summary: Get metric dashboards (v2)
tags:
- metrics

View File

@@ -43,6 +43,25 @@ type Resource interface {
All resources are registered in [pkg/types/coretypes/registry_resource.go](/pkg/types/coretypes/registry_resource.go) using constructors like `NewResourceMetaResource(KindDashboard)` or `NewResourceTelemetryResource(KindLogs)`.
### Telemetry resources and metaresources
Almost every feature resource is one of these two types, and they answer different questions.
- A **telemetry resource** (`logs`, `traces`, `metrics`, `meter-metrics`, `audit-logs`) is the data itself. It only supports `read`. Its selector is not an ID but the query: the route wires a `TelemetryResourceDef` whose extractor turns the caller's filter into `<query_type>/<key>/<value>` IDs, and the check ladders each one through `<query_type>/<key>/*`, `<query_type>/*` and `*`. A grant such as `builder_query/signoz.workspace.key.id/key-a` therefore decides which rows a role may read; a query that does not pin a granted key resolves to `<query_type>/*` and is denied for key-scoped roles. Managed roles hold `*`.
- A **metaresource** (`dashboard`, `rule`, `logs-field`, ...) is an object that describes or configures the system. It supports whichever of `create`, `read`, `update`, `delete` and `list` the resource declares, its selectors are object IDs or `*`, and routes wire a `BasicResourceDef`. It says nothing about rows: `dashboard:read` on `*` does not grant the telemetry a dashboard's panels query.
The metrics explorer uses both on one surface:
| Route | Resource | Why |
|---|---|---|
| `POST /api/v2/metrics/stats` | `metrics:read` (telemetry) | Returns sample and series counts, i.e. the data. The selector comes from `filter.expression`, so a role scoped to a workspace key sees only its own samples. |
| `GET /api/v2/metrics` | `metrics-field:list` (meta) | Lists metric names, types and units: the catalogue, not the samples. |
| `POST /api/v2/metrics/metadata` | `metrics-field:update` (meta) | Edits a metric's type, unit and description. |
The two are independent: a role holding `metrics-field` `list` and `read` can browse the catalogue but cannot read a sample, and a role scoped to `builder_query/signoz.workspace.key.id/key-a` on `metrics` can query its samples but cannot list the catalogue.
For a new route, ask what it returns. Rows or aggregates over them go on the signal's telemetry resource with an extractor; anything that names, describes or configures telemetry goes on a metaresource.
### Selector
A `Selector` identifies *which* instance(s) of a resource a check is about — a UUID, a role name, or the wildcard `*`. A `SelectorFunc` maps the extracted resource ID to selectors at request time. Two prebuilt ones cover most routes ([pkg/types/coretypes/selector.go](/pkg/types/coretypes/selector.go)):
@@ -142,6 +161,7 @@ Because both paths go through the same middleware and the same `ResourceDef` dec
## What should I remember?
- Declare authz in the registries ([pkg/types/coretypes](/pkg/types/coretypes)), not in migrations — tuples for new organizations are derived from code at bootstrap.
- Telemetry resources are the data: `read` only, selector derived from the query. Metaresources are objects about the data: CRUD and list, ID or wildcard selectors. A route that returns rows or aggregates is telemetry; one that names or configures them is meta, as `metrics` versus `metrics-field` shows.
- A new kind never needs an OpenFGA schema change; only a new type does, and then **both** [pkg/authz/openfgaschema/base.fga](/pkg/authz/openfgaschema/base.fga) and [ee/authz/openfgaschema/base.fga](/ee/authz/openfgaschema/base.fga) must be updated together.
- Prefer `CheckResources` + `ResourceDef` over the coarse `ViewAccess`/`EditAccess`/`AdminAccess` gates for new routes.
- Use `WildcardSelector` for `create`/`list`, `IDSelector` for instance operations, and a custom `SelectorFunc` when the request ID is not the FGA selector.

View File

@@ -116,7 +116,7 @@ func (c *clickhouse) EstimateCardinality(ctx context.Context, metricName string,
sb := sqlbuilder.NewSelectBuilder()
reducedExpr := "1"
reducedExpr := "toUInt64(1)"
if len(keptLabels) > 0 {
reducedExpr = "uniq(("
for i, label := range keptLabels {

View File

@@ -3291,6 +3291,7 @@ export enum CoretypesKindDTO {
'meter-metrics' = 'meter-metrics',
'logs-field' = 'logs-field',
'traces-field' = 'traces-field',
'metrics-field' = 'metrics-field',
}
export enum CoretypesTypeDTO {
user = 'user',

View File

@@ -3,7 +3,6 @@ import { Provider } from 'react-redux';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
import { message } from 'antd';
import configureStore from 'redux-mock-store';
import { PANEL_TYPES } from 'constants/queryBuilder';
import store from 'store';
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import { EQueryType } from 'types/common/dashboard';
@@ -79,14 +78,11 @@ const createMockStagedQuery = (dataSource: DataSource): Query => ({
clickhouse_sql: [],
});
const renderWithStore = (
dataSource: DataSource,
panelType?: PANEL_TYPES,
): void => {
const renderWithStore = (dataSource: DataSource): void => {
const mockReduxStore = createMockReduxStore();
render(
<Provider store={mockReduxStore}>
<DownloadOptionsMenu dataSource={dataSource} panelType={panelType} />
<DownloadOptionsMenu dataSource={dataSource} />
</Provider>,
);
};
@@ -121,14 +117,11 @@ describe.each([
expect(screen.getByText('FORMAT')).toBeInTheDocument();
expect(screen.getByText('Number of Rows')).toBeInTheDocument();
expect(screen.getByText('Columns')).toBeInTheDocument();
});
it('hides columns scope in the trace view', () => {
renderWithStore(dataSource, PANEL_TYPES.TRACE);
fireEvent.click(screen.getByTestId(testId));
expect(screen.queryByText('Columns')).not.toBeInTheDocument();
if (dataSource === DataSource.TRACES) {
expect(screen.queryByText('Columns')).not.toBeInTheDocument();
} else {
expect(screen.getByText('Columns')).toBeInTheDocument();
}
});
it('allows changing export format', () => {
@@ -158,6 +151,17 @@ describe.each([
});
it('allows changing columns scope', () => {
if (dataSource === DataSource.TRACES) {
renderWithStore(dataSource);
fireEvent.click(screen.getByTestId(testId));
expect(screen.queryByRole('radio', { name: 'All' })).not.toBeInTheDocument();
expect(
screen.queryByRole('radio', { name: 'Selected' }),
).not.toBeInTheDocument();
return;
}
renderWithStore(dataSource);
fireEvent.click(screen.getByTestId(testId));
@@ -224,7 +228,12 @@ describe.each([
mockUseQueryBuilder.mockReturnValue({ stagedQuery: mockQuery });
renderWithStore(dataSource);
fireEvent.click(screen.getByTestId(testId));
fireEvent.click(screen.getByRole('radio', { name: 'Selected' }));
// For traces, column scope is always Selected and the radio is hidden
if (dataSource !== DataSource.TRACES) {
fireEvent.click(screen.getByRole('radio', { name: 'Selected' }));
}
fireEvent.click(screen.getByText('Export'));
await waitFor(() => {
@@ -241,6 +250,11 @@ describe.each([
});
it('sends no selectFields when column scope is All', async () => {
// For traces, column scope is always Selected — this test only applies to other sources
if (dataSource === DataSource.TRACES) {
return;
}
renderWithStore(dataSource);
fireEvent.click(screen.getByTestId(testId));
fireEvent.click(screen.getByRole('radio', { name: 'All' }));

View File

@@ -45,7 +45,9 @@ export default function DownloadOptionsMenu({
await handleExportRawData({
format: exportFormat,
rowLimit,
clearSelectColumns: columnsScope === DownloadColumnsScopes.ALL,
clearSelectColumns:
dataSource !== DataSource.TRACES &&
columnsScope === DownloadColumnsScopes.ALL,
selectedColumns,
});
}, [
@@ -54,6 +56,7 @@ export default function DownloadOptionsMenu({
columnsScope,
selectedColumns,
handleExportRawData,
dataSource,
]);
const popoverContent = useMemo(
@@ -93,7 +96,7 @@ export default function DownloadOptionsMenu({
/>
</div>
{panelType !== PANEL_TYPES.TRACE && (
{dataSource !== DataSource.TRACES && (
<>
<div className="horizontal-line" />
@@ -131,7 +134,7 @@ export default function DownloadOptionsMenu({
columnsScope,
isDownloading,
handleExport,
panelType,
dataSource,
],
);

View File

@@ -2,8 +2,11 @@ import { useCallback, useEffect, useState } from 'react';
import { Button, InputNumber, Popover, Tooltip } from 'antd';
import { Typography } from '@signozhq/ui/typography';
import cx from 'classnames';
import { LogViewMode } from 'container/OptionsMenu/types';
import { FontSize, OptionsMenuConfig } from 'container/OptionsMenu/types';
import {
LogViewMode,
FontSize,
OptionsMenuConfig,
} from 'container/OptionsMenu/types';
import {
Check,
ChevronLeft,

View File

@@ -73,7 +73,8 @@
"audit-logs",
"meter-metrics",
"logs-field",
"traces-field"
"traces-field",
"metrics-field"
],
"type": "string"
},

View File

@@ -4,14 +4,16 @@ import (
"net/http"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/coretypes"
"github.com/SigNoz/signoz/pkg/types/metricsexplorertypes"
"github.com/gorilla/mux"
)
func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
if err := router.Handle("/api/v2/metrics", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.ListMetrics),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.ListMetrics, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "ListMetrics",
Tags: []string{"metrics"},
@@ -25,13 +27,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceMetricsField.Scope(coretypes.VerbList)}),
}, handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceMetricsField,
Verb: coretypes.VerbList,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}))).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/stats", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetStats),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetStats, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricsStats",
Tags: []string{"metrics"},
@@ -44,13 +51,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodPost).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceTelemetryResourceMetrics.Scope(coretypes.VerbRead)}),
}, handler.WithResourceDefs(handler.TelemetryResourceDef{
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: querybuilder.TelemetrySelector,
Resources: querybuilder.BuilderQueryWithFilterExpressionResources(coretypes.ResourceTelemetryResourceMetrics, "filter.expression"),
}))).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/treemap", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetTreemap),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetTreemap, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricsTreemap",
Tags: []string{"metrics"},
@@ -63,13 +75,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodPost).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceTelemetryResourceMetrics.Scope(coretypes.VerbRead)}),
}, handler.WithResourceDefs(handler.TelemetryResourceDef{
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: querybuilder.TelemetrySelector,
Resources: querybuilder.BuilderQueryWithFilterExpressionResources(coretypes.ResourceTelemetryResourceMetrics, "filter.expression"),
}))).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/attributes", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetMetricAttributes),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetMetricAttributes, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricAttributes",
Tags: []string{"metrics"},
@@ -83,13 +100,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusNotFound, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceMetricsField.Scope(coretypes.VerbRead)}),
}, handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceMetricsField,
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}))).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/metadata", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetMetricMetadata),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetMetricMetadata, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricMetadata",
Tags: []string{"metrics"},
@@ -103,13 +125,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusNotFound, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceMetricsField.Scope(coretypes.VerbRead)}),
}, handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceMetricsField,
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}))).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/metadata", handler.New(
provider.authzMiddleware.EditAccess(provider.metricsExplorerHandler.UpdateMetricMetadata),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.UpdateMetricMetadata, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName),
handler.OpenAPIDef{
ID: "UpdateMetricMetadata",
Tags: []string{"metrics"},
@@ -122,13 +149,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleEditor),
})).Methods(http.MethodPost).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceMetricsField.Scope(coretypes.VerbUpdate)}),
}, handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceMetricsField,
Verb: coretypes.VerbUpdate,
Category: coretypes.ActionCategoryConfigurationChange,
Selector: coretypes.WildcardSelector,
}))).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/highlights", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetMetricHighlights),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetMetricHighlights, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricHighlights",
Tags: []string{"metrics"},
@@ -142,13 +174,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusNotFound, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceTelemetryResourceMetrics.Scope(coretypes.VerbRead)}),
}, handler.WithResourceDefs(handler.TelemetryResourceDef{
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: querybuilder.TelemetrySelector,
Resources: querybuilder.BuilderQueryWildcardResources(coretypes.ResourceTelemetryResourceMetrics),
}))).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/alerts", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetMetricAlerts),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetMetricAlerts, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricAlerts",
Tags: []string{"metrics"},
@@ -162,13 +199,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusNotFound, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceRule.Scope(coretypes.VerbList)}),
}, handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceRule,
Verb: coretypes.VerbList,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}))).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v3/metrics/dashboards", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetMetricDashboardsV2),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetMetricDashboardsV2, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricDashboardsV2",
Tags: []string{"metrics"},
@@ -182,13 +224,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusNotFound, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceDashboard.Scope(coretypes.VerbList)}),
}, handler.WithResourceDefs(handler.BasicResourceDef{
Resource: coretypes.ResourceMetaResourceDashboard,
Verb: coretypes.VerbList,
Category: coretypes.ActionCategoryDataAccess,
Selector: coretypes.WildcardSelector,
}))).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/inspect", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.InspectMetrics),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.InspectMetrics, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "InspectMetrics",
Tags: []string{"metrics"},
@@ -201,13 +248,18 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodPost).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceTelemetryResourceMetrics.Scope(coretypes.VerbRead)}),
}, handler.WithResourceDefs(handler.TelemetryResourceDef{
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: querybuilder.TelemetrySelector,
Resources: querybuilder.BuilderQueryWithFilterExpressionResources(coretypes.ResourceTelemetryResourceMetrics, "filter.expression"),
}))).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/metrics/onboarding", handler.New(
provider.authzMiddleware.ViewAccess(provider.metricsExplorerHandler.GetOnboardingStatus),
provider.authzMiddleware.CheckResources(provider.metricsExplorerHandler.GetOnboardingStatus, authtypes.SigNozAdminRoleName, authtypes.SigNozEditorRoleName, authtypes.SigNozViewerRoleName),
handler.OpenAPIDef{
ID: "GetMetricsOnboardingStatus",
Tags: []string{"metrics"},
@@ -220,8 +272,13 @@ func (provider *provider) addMetricsExplorerRoutes(router *mux.Router) error {
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusUnauthorized, http.StatusInternalServerError},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodGet).GetError(); err != nil {
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceTelemetryResourceMetrics.Scope(coretypes.VerbRead)}),
}, handler.WithResourceDefs(handler.TelemetryResourceDef{
Verb: coretypes.VerbRead,
Category: coretypes.ActionCategoryDataAccess,
Selector: querybuilder.TelemetrySelector,
Resources: querybuilder.BuilderQueryWildcardResources(coretypes.ResourceTelemetryResourceMetrics),
}))).Methods(http.MethodGet).GetError(); err != nil {
return err
}

View File

@@ -118,10 +118,47 @@
"description": ""
}
],
"logs": []
"logs": [
{
"name": "Project",
"path": "gcp.project",
"type": "string"
},
{
"name": "Resource Type",
"path": "gcp.resource_type",
"type": "string"
}
]
},
"telemetryCollectionStrategy": {
"gcp": {}
"gcp": {
"metrics": {
"metricNames": [
"cloudsql.googleapis.com/database/up",
"cloudsql.googleapis.com/database/instance_state",
"cloudsql.googleapis.com/database/replication/replica_lag",
"cloudsql.googleapis.com/database/cpu/utilization",
"cloudsql.googleapis.com/database/memory/utilization",
"cloudsql.googleapis.com/database/disk/utilization",
"cloudsql.googleapis.com/database/network/connections",
"cloudsql.googleapis.com/database/mysql/queries",
"cloudsql.googleapis.com/database/mysql/dml_operations_count",
"cloudsql.googleapis.com/database/mysql/threads",
"cloudsql.googleapis.com/database/mysql/innodb/buffer_pool_reads_count",
"cloudsql.googleapis.com/database/mysql/innodb/buffer_pool_read_requests_count",
"cloudsql.googleapis.com/database/mysql/slow_queries_count",
"cloudsql.googleapis.com/database/mysql/aborted_connects_count",
"cloudsql.googleapis.com/database/mysql/innodb/deadlocks_count",
"cloudsql.googleapis.com/database/mysql/innodb/row_lock_waits_count",
"cloudsql.googleapis.com/database/disk/read_ops_count",
"cloudsql.googleapis.com/database/disk/write_ops_count"
]
},
"logs": {
"sinkFilter": "resource.type=\"cloudsql_database\" AND (logName:\"/logs/cloudsql.googleapis.com%2Fmysql\" OR (log_id(\"cloudaudit.googleapis.com/data_access\") AND protoPayload.request.@type=\"type.googleapis.com/google.cloud.sql.audit.v1.MysqlAuditEntry\"))"
}
}
},
"assets": {
"dashboards": [

View File

@@ -5,7 +5,7 @@
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": false
"logs": true
},
"dataCollected": {
"metrics": [
@@ -75,6 +75,12 @@
"type": "Gauge",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/postgresql/insights/aggregate/io_time",
"unit": "Microseconds",
"type": "Sum",
"description": ""
},
{
"name": "cloudsql.googleapis.com/database/postgresql/insights/perquery/execution_time",
"unit": "Microseconds",
@@ -88,10 +94,43 @@
"description": ""
}
],
"logs": []
"logs": [
{
"name": "Project",
"path": "gcp.project",
"type": "string"
},
{
"name": "Resource Type",
"path": "gcp.resource_type",
"type": "string"
}
]
},
"telemetryCollectionStrategy": {
"gcp": {}
"gcp": {
"metrics": {
"metricNames": [
"cloudsql.googleapis.com/database/up",
"cloudsql.googleapis.com/database/cpu/utilization",
"cloudsql.googleapis.com/database/memory/utilization",
"cloudsql.googleapis.com/database/memory/usage",
"cloudsql.googleapis.com/database/disk/bytes_used",
"cloudsql.googleapis.com/database/postgresql/num_backends",
"cloudsql.googleapis.com/database/postgresql/num_backends_by_state",
"cloudsql.googleapis.com/database/postgresql/transaction_count",
"cloudsql.googleapis.com/database/postgresql/deadlock_count",
"cloudsql.googleapis.com/database/postgresql/vacuum/oldest_transaction_age",
"cloudsql.googleapis.com/database/postgresql/insights/aggregate/execution_time",
"cloudsql.googleapis.com/database/postgresql/insights/aggregate/io_time",
"cloudsql.googleapis.com/database/postgresql/insights/perquery/execution_time",
"cloudsql.googleapis.com/database/postgresql/replication/replica_byte_lag"
]
},
"logs": {
"sinkFilter": "resource.type=\"cloudsql_database\" AND (logName:\"/logs/cloudsql.googleapis.com%2Fpostgres\" OR (log_id(\"cloudaudit.googleapis.com/data_access\") AND protoPayload.request.@type=\"type.googleapis.com/google.cloud.sql.audit.v1.PgAuditEntry\"))"
}
}
},
"assets": {
"dashboards": [

View File

@@ -5,7 +5,7 @@
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": false
"logs": true
},
"dataCollected": {
"metrics": [
@@ -52,10 +52,36 @@
"description": ""
}
],
"logs": []
"logs": [
{
"name": "Project",
"path": "gcp.project",
"type": "string"
},
{
"name": "Resource Type",
"path": "gcp.resource_type",
"type": "string"
}
]
},
"telemetryCollectionStrategy": {
"gcp": {}
"gcp": {
"metrics": {
"metricNames": [
"storage.googleapis.com/storage/v2/total_bytes",
"storage.googleapis.com/storage/v2/total_count",
"storage.googleapis.com/storage/v2/deleted_bytes",
"storage.googleapis.com/api/request_count",
"storage.googleapis.com/network/received_bytes_count",
"storage.googleapis.com/network/sent_bytes_count",
"storage.googleapis.com/authn/authentication_count"
]
},
"logs": {
"sinkFilter": "resource.type=\"gcs_bucket\""
}
}
},
"assets": {
"dashboards": [

View File

@@ -5,7 +5,7 @@
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": false
"logs": true
},
"dataCollected": {
"metrics": [
@@ -106,10 +106,45 @@
"description": ""
}
],
"logs": []
"logs": [
{
"name": "Project",
"path": "gcp.project",
"type": "string"
},
{
"name": "Resource Type",
"path": "gcp.resource_type",
"type": "string"
}
]
},
"telemetryCollectionStrategy": {
"gcp": {}
"gcp": {
"metrics": {
"metricNames": [
"compute.googleapis.com/instance/uptime_total",
"compute.googleapis.com/instance/cpu/utilization",
"compute.googleapis.com/guest/memory/bytes_used",
"compute.googleapis.com/guest/memory/percent_used",
"agent.googleapis.com/memory/percent_used",
"compute.googleapis.com/instance/disk/average_io_latency",
"compute.googleapis.com/instance/disk/read_bytes_count",
"compute.googleapis.com/instance/disk/write_bytes_count",
"compute.googleapis.com/instance/disk/read_ops_count",
"compute.googleapis.com/instance/disk/write_ops_count",
"compute.googleapis.com/instance/disk/performance_status",
"compute.googleapis.com/instance/network/received_bytes_count",
"compute.googleapis.com/instance/network/sent_bytes_count",
"compute.googleapis.com/instance/network/received_packets_count",
"compute.googleapis.com/instance/network/sent_packets_count",
"compute.googleapis.com/firewall/dropped_packets_count"
]
},
"logs": {
"sinkFilter": "resource.type=\"gce_instance\" OR protoPayload.serviceName=\"compute.googleapis.com\""
}
}
},
"assets": {
"dashboards": [

View File

@@ -5,7 +5,7 @@
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": false
"logs": true
},
"dataCollected": {
"metrics": [
@@ -106,10 +106,45 @@
"description": ""
}
],
"logs": []
"logs": [
{
"name": "Project",
"path": "gcp.project",
"type": "string"
},
{
"name": "Resource Type",
"path": "gcp.resource_type",
"type": "string"
}
]
},
"telemetryCollectionStrategy": {
"gcp": {}
"gcp": {
"metrics": {
"metricNames": [
"kubernetes.io/container/cpu/limit_utilization",
"kubernetes.io/container/cpu/request_utilization",
"kubernetes.io/container/memory/limit_utilization",
"kubernetes.io/container/memory/request_utilization",
"kubernetes.io/container/restart_count",
"kubernetes.io/pod/latencies/pod_first_ready",
"kubernetes.io/pod/network/received_bytes_count",
"kubernetes.io/pod/network/sent_bytes_count",
"kubernetes.io/pod/volume/utilization",
"kubernetes.io/node/cpu/allocatable_utilization",
"kubernetes.io/node/memory/allocatable_utilization",
"kubernetes.io/node/ephemeral_storage/used_bytes",
"kubernetes.io/node/ephemeral_storage/allocatable_bytes",
"kubernetes.io/node/network/received_bytes_count",
"kubernetes.io/node/network/sent_bytes_count",
"kubernetes.io/node/status_condition"
]
},
"logs": {
"sinkFilter": "resource.type=(\"k8s_container\" OR \"k8s_pod\" OR \"k8s_node\" OR \"k8s_cluster\" OR \"k8s_control_plane_component\" OR \"gke_cluster\" OR \"gke_nodepool\")"
}
}
},
"assets": {
"dashboards": [

View File

@@ -5,7 +5,7 @@
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": false
"logs": true
},
"dataCollected": {
"metrics": [
@@ -76,10 +76,40 @@
"description": ""
}
],
"logs": []
"logs": [
{
"name": "Project",
"path": "gcp.project",
"type": "string"
},
{
"name": "Resource Type",
"path": "gcp.resource_type",
"type": "string"
}
]
},
"telemetryCollectionStrategy": {
"gcp": {}
"gcp": {
"metrics": {
"metricNames": [
"redis.googleapis.com/server/uptime",
"redis.googleapis.com/stats/memory/usage_ratio",
"redis.googleapis.com/stats/memory/system_memory_usage_ratio",
"redis.googleapis.com/stats/cpu_utilization_main_thread",
"redis.googleapis.com/clients/connected",
"redis.googleapis.com/stats/cache_hit_ratio",
"redis.googleapis.com/keyspace/keys",
"redis.googleapis.com/commands/usec_per_call",
"redis.googleapis.com/commands/calls",
"redis.googleapis.com/clients/blocked",
"redis.googleapis.com/stats/reject_connections_count"
]
},
"logs": {
"sinkFilter": "resource.type=\"redis_instance\" OR (protoPayload.serviceName=\"redis.googleapis.com\" AND protoPayload.methodName:\".CloudRedis.\") OR (log_id(\"maintenance.googleapis.com/activity_log\") AND labels.activity_type_name:\"CloudRedisRollout\")"
}
}
},
"assets": {
"dashboards": [

View File

@@ -263,6 +263,7 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddFieldTuplesFactory(sqlstore),
sqlmigration.NewKeepRelevantLLMPricingRulesFactory(),
sqlmigration.NewBackfillTelegramChannelSpecFactory(),
sqlmigration.NewAddMetricsFieldTuplesFactory(sqlstore),
)
}

View File

@@ -0,0 +1,162 @@
package sqlmigration
import (
"context"
"database/sql"
"encoding/json"
"time"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/coretypes"
"github.com/oklog/ulid/v2"
"github.com/uptrace/bun"
"github.com/uptrace/bun/dialect"
"github.com/uptrace/bun/migrate"
)
type addMetricsFieldTuples struct {
sqlstore sqlstore.SQLStore
}
func NewAddMetricsFieldTuplesFactory(sqlstore sqlstore.SQLStore) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(factory.MustNewName("add_metrics_field_tuples"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &addMetricsFieldTuples{sqlstore: sqlstore}, nil
})
}
func (migration *addMetricsFieldTuples) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
func (migration *addMetricsFieldTuples) Up(ctx context.Context, db *bun.DB) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var storeID string
err = tx.QueryRowContext(ctx, `SELECT id FROM store WHERE name = ? LIMIT 1`, "signoz").Scan(&storeID)
if err != nil {
return err
}
var orgIDs []string
err = tx.NewSelect().
Table("organizations").
Column("id").
Scan(ctx, &orgIDs)
if err != nil && err != sql.ErrNoRows {
return err
}
isPG := migration.sqlstore.BunDB().Dialect().Name() == dialect.PG
tuples := []migrationTuple{
{authtypes.SigNozAdminRoleName, "metaresource", "metrics-field", "read"},
{authtypes.SigNozAdminRoleName, "metaresource", "metrics-field", "update"},
{authtypes.SigNozAdminRoleName, "metaresource", "metrics-field", "list"},
{authtypes.SigNozEditorRoleName, "metaresource", "metrics-field", "read"},
{authtypes.SigNozEditorRoleName, "metaresource", "metrics-field", "update"},
{authtypes.SigNozEditorRoleName, "metaresource", "metrics-field", "list"},
{authtypes.SigNozViewerRoleName, "metaresource", "metrics-field", "read"},
{authtypes.SigNozViewerRoleName, "metaresource", "metrics-field", "list"},
}
for _, orgID := range orgIDs {
for _, tuple := range tuples {
entropy := ulid.DefaultEntropy()
now := time.Now().UTC()
tupleID := ulid.MustNew(ulid.Timestamp(now), entropy).String()
objectID := "organization/" + orgID + "/" + tuple.objectName + "/*"
roleSubject := "organization/" + orgID + "/role/" + tuple.roleName
if isPG {
user := "role:" + roleSubject + "#assignee"
result, err := tx.ExecContext(ctx, `
INSERT INTO tuple (store, object_type, object_id, relation, _user, user_type, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, object_type, object_id, relation, _user) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, user, "userset", tupleID, now,
)
if err != nil {
return err
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
continue
}
_, err = tx.ExecContext(ctx, `
INSERT INTO changelog (store, object_type, object_id, relation, _user, operation, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, user, 0, tupleID, now,
)
if err != nil {
return err
}
} else {
result, err := tx.ExecContext(ctx, `
INSERT INTO tuple (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, user_type, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", "userset", tupleID, now,
)
if err != nil {
return err
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return err
}
if rowsAffected == 0 {
continue
}
_, err = tx.ExecContext(ctx, `
INSERT INTO changelog (store, object_type, object_id, relation, user_object_type, user_object_id, user_relation, operation, ulid, inserted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (store, ulid, object_type) DO NOTHING`,
storeID, tuple.objectType, objectID, tuple.relation, "role", roleSubject, "assignee", 0, tupleID, now,
)
if err != nil {
return err
}
}
}
}
managedRoleGroups := make(map[string]string, len(coretypes.ManagedRoleToTransactions))
for roleName, transactions := range coretypes.ManagedRoleToTransactions {
data, err := json.Marshal(authtypes.NewTransactionGroupsFromTransactions(transactions))
if err != nil {
return err
}
managedRoleGroups[roleName] = string(data)
}
for _, orgID := range orgIDs {
for roleName, data := range managedRoleGroups {
if _, err := tx.NewUpdate().
Model(new(roles)).
Set("transaction_groups = ?", data).
Where("org_id = ?", orgID).
Where("type = ?", authtypes.RoleTypeManaged.StringValue()).
Where("name = ?", roleName).
Exec(ctx); err != nil {
return err
}
}
}
return tx.Commit()
}
func (migration *addMetricsFieldTuples) Down(context.Context, *bun.DB) error {
return nil
}

View File

@@ -24,7 +24,19 @@ type GCPConnectionArtifact struct{}
type GCPIntegrationConfig struct{}
type GCPTelemetryCollectionStrategy struct{}
type GCPTelemetryCollectionStrategy struct {
Metrics *GCPMetricsCollectionStrategy `json:"metrics" required:"false" nullable:"false"`
Logs *GCPLogsCollectionStrategy `json:"logs" required:"false" nullable:"false"`
}
type GCPMetricsCollectionStrategy struct {
MetricNames []string `json:"metricNames" required:"true" nullable:"false"`
}
type GCPLogsCollectionStrategy struct {
// Cloud Logging filter for the log sink, https://cloud.google.com/logging/docs/view/logging-query-language
SinkFilter string `json:"sinkFilter" required:"true"`
}
type GCPServiceConfig struct {
Logs *GCPServiceLogsConfig `json:"logs,omitempty" required:"false"`

View File

@@ -79,6 +79,7 @@ func (name Kind) Enum() []any {
KindMeterMetrics,
KindLogsField,
KindTracesField,
KindMetricsField,
}
}

View File

@@ -39,6 +39,7 @@ var Kinds = []Kind{
KindMeterMetrics,
KindLogsField,
KindTracesField,
KindMetricsField,
}
var (
@@ -80,4 +81,5 @@ var (
KindMeterMetrics = MustNewKind("meter-metrics")
KindLogsField = MustNewKind("logs-field")
KindTracesField = MustNewKind("traces-field")
KindMetricsField = MustNewKind("metrics-field")
)

View File

@@ -214,6 +214,10 @@ var ManagedRoleToTransactions = map[string][]Transaction{
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
{Verb: VerbUpdate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
// metrics-field — editor+admin update, viewer reads
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
{Verb: VerbUpdate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
},
SigNozEditorRoleName: {
// cloud-integration — editor can read accounts
@@ -309,6 +313,10 @@ var ManagedRoleToTransactions = map[string][]Transaction{
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
{Verb: VerbUpdate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
// metrics-field — editor reads+updates
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
{Verb: VerbUpdate, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
},
SigNozViewerRoleName: {
// cloud-integration — viewer can read accounts
@@ -370,6 +378,9 @@ var ManagedRoleToTransactions = map[string][]Transaction{
// traces-field — viewer reads
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindTracesField}, WildCardSelectorString)},
// metrics-field — viewer reads
{Verb: VerbRead, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
{Verb: VerbList, Object: *MustNewObject(ResourceRef{Type: TypeMetaResource, Kind: KindMetricsField}, WildCardSelectorString)},
},
SigNozAnonymousRoleName: {
// public-dashboard — anonymous read

View File

@@ -39,6 +39,7 @@ var Resources = []Resource{
ResourceTelemetryResourceMeterMetrics,
ResourceMetaResourceLogsField,
ResourceMetaResourceTracesField,
ResourceMetaResourceMetricsField,
}
var (
@@ -80,4 +81,5 @@ var (
ResourceTelemetryResourceMeterMetrics = NewResourceTelemetryResource(KindMeterMetrics)
ResourceMetaResourceLogsField = NewResourceMetaResource(KindLogsField)
ResourceMetaResourceTracesField = NewResourceMetaResource(KindTracesField)
ResourceMetaResourceMetricsField = NewResourceMetaResource(KindMetricsField, VerbList, VerbRead, VerbUpdate)
)

View File

@@ -49,9 +49,6 @@ class ProviderAccountSpec:
class ProviderServiceSpec:
provider: str
service_id: str
# GCP ships every service with supportedSignals.logs false, so a logs block
# is neither required on write nor persisted.
supports_logs: bool
account_config: dict
# id shown in parametrized test names; defaults to the provider slug.
id: str = field(default="")
@@ -61,10 +58,12 @@ class ProviderServiceSpec:
object.__setattr__(self, "id", self.provider)
def build_service_config(self, metrics_enabled: bool, logs_enabled: bool | None = None) -> dict:
config: dict = {"metrics": {"enabled": metrics_enabled}}
if self.supports_logs:
config["logs"] = {"enabled": metrics_enabled if logs_enabled is None else logs_enabled}
return {self.provider: config}
return {
self.provider: {
"metrics": {"enabled": metrics_enabled},
"logs": {"enabled": metrics_enabled if logs_enabled is None else logs_enabled},
}
}
@pytest.fixture(scope="function")

View File

@@ -288,6 +288,15 @@
"list"
]
},
{
"type": "metaresource",
"kind": "metrics-field",
"verbs": [
"read",
"update",
"list"
]
},
{
"type": "metaresource",
"kind": "ttl-setting",
@@ -510,6 +519,15 @@
"list"
]
},
{
"type": "metaresource",
"kind": "metrics-field",
"verbs": [
"read",
"update",
"list"
]
},
{
"type": "metaresource",
"kind": "ttl-setting",
@@ -681,6 +699,14 @@
"list"
]
},
{
"type": "metaresource",
"kind": "metrics-field",
"verbs": [
"read",
"list"
]
},
{
"type": "metaresource",
"kind": "ttl-setting",

View File

@@ -19,14 +19,12 @@ logger = setup_logger(__name__)
AWS_SERVICE_SPEC = ProviderServiceSpec(
provider="aws",
service_id="rds",
supports_logs=True,
account_config={"aws": {"deploymentRegion": "us-east-1", "regions": ["us-east-1"]}},
)
GCP_SERVICE_SPEC = ProviderServiceSpec(
provider="gcp",
service_id="cloudsql_postgres",
supports_logs=False,
account_config={
"gcp": {
"deploymentProjectId": "signoz-test-project",
@@ -162,7 +160,7 @@ def test_get_service_details_without_account(
assert data["cloudIntegrationService"] is None, "cloudIntegrationService should be null without account context"
assert data["supportedSignals"]["metrics"] is True, "metrics should be a supported signal"
assert data["supportedSignals"]["logs"] is spec.supports_logs, f"logs support should be {spec.supports_logs} for {spec.provider}"
assert data["supportedSignals"]["logs"] is True, "logs should be a supported signal"
@provider_spec
@@ -253,10 +251,7 @@ def test_update_service_config(
assert svc["config"][spec.provider]["metrics"]["enabled"] is True, "metrics should be enabled"
assert svc["cloudIntegrationId"] == account_id, "cloudIntegrationId should match the account"
if spec.supports_logs:
assert svc["config"][spec.provider]["logs"]["enabled"] is True, "logs should be enabled"
else:
assert svc["config"][spec.provider].get("logs") is None, f"logs should not be stored for {spec.provider}, got: {svc['config'][spec.provider]}"
assert svc["config"][spec.provider]["logs"]["enabled"] is True, "logs should be enabled"
@provider_spec
@@ -307,8 +302,7 @@ def test_update_service_config_disable(
assert svc is not None, "cloudIntegrationService should still be present after disable"
assert svc["config"][spec.provider]["metrics"]["enabled"] is False, "metrics should be disabled"
if spec.supports_logs:
assert svc["config"][spec.provider]["logs"]["enabled"] is False, "logs should be disabled"
assert svc["config"][spec.provider]["logs"]["enabled"] is False, "logs should be disabled"
@provider_spec
@@ -337,7 +331,7 @@ def test_update_gcp_service_without_metrics_config(
get_token: Callable[[str, str], str],
create_cloud_integration_account: Callable,
) -> None:
"""GCP services support metrics only, so a config omitting metrics is rejected."""
"""A GCP config omitting metrics is rejected."""
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
account = create_cloud_integration_account(admin_token, "gcp", config=GCP_SERVICE_SPEC.account_config)

View File

@@ -0,0 +1,286 @@
from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
import pytest
import requests
from wiremock.resources.mappings import Mapping
from fixtures import types
from fixtures.auth import (
USER_ADMIN_EMAIL,
USER_ADMIN_PASSWORD,
add_license,
change_user_role,
create_active_user,
)
from fixtures.metrics import Metrics
from fixtures.role import transaction_group
user_password = "password123Z$"
scoped_role = "metrics-explorer-scope-key-a"
scoped_email = "metrics-explorer-scope-key-a@telemetry.test"
wildcard_role = "metrics-explorer-wildcard"
wildcard_email = "metrics-explorer-wildcard@telemetry.test"
metadata_role = "metrics-explorer-metadata-reader"
metadata_email = "metrics-explorer-metadata-reader@telemetry.test"
viewer_email = "metrics-explorer-viewer@telemetry.test"
editor_email = "metrics-explorer-editor@telemetry.test"
def test_setup(
signoz: types.SigNoz,
create_user_admin: types.Operation, # pylint: disable=unused-argument
make_http_mocks: Callable[[types.TestContainerDocker, list[Mapping]], None],
get_token: Callable[[str, str], str],
create_role: Callable[..., str],
) -> None:
add_license(signoz, make_http_mocks, get_token)
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
create_role(admin_token, scoped_role, [transaction_group("read", "telemetryresource", "metrics", ["builder_query/signoz.workspace.key.id/key-a"])])
scoped_user = create_active_user(signoz, admin_token, email=scoped_email, role="signoz-viewer", password=user_password)
change_user_role(signoz, admin_token, scoped_user, "signoz-viewer", scoped_role)
create_role(admin_token, wildcard_role, [transaction_group("read", "telemetryresource", "metrics", ["builder_query/*"])])
wildcard_user = create_active_user(signoz, admin_token, email=wildcard_email, role="signoz-viewer", password=user_password)
change_user_role(signoz, admin_token, wildcard_user, "signoz-viewer", wildcard_role)
create_role(
admin_token,
metadata_role,
[transaction_group("read", "metaresource", "metrics-field", ["*"]), transaction_group("list", "metaresource", "metrics-field", ["*"])],
)
metadata_user = create_active_user(signoz, admin_token, email=metadata_email, role="signoz-viewer", password=user_password)
change_user_role(signoz, admin_token, metadata_user, "signoz-viewer", metadata_role)
create_active_user(signoz, admin_token, email=viewer_email, role="signoz-viewer", password=user_password)
create_active_user(signoz, admin_token, email=editor_email, role="signoz-editor", password=user_password)
@pytest.mark.parametrize(
("method", "path", "query", "body"),
[
("POST", "/api/v2/metrics/stats", None, {"limit": 10}),
("POST", "/api/v2/metrics/treemap", None, {"limit": 10, "mode": "samples"}),
("GET", "/api/v2/metrics/highlights", {"metricName": "explorer.requests"}, None),
("POST", "/api/v2/metrics/inspect", None, {"metricName": "explorer.requests"}),
("GET", "/api/v2/metrics/onboarding", {}, None),
],
ids=["stats", "treemap", "highlights", "inspect", "onboarding"],
)
@pytest.mark.parametrize(
("email", "expected"),
[(scoped_email, HTTPStatus.FORBIDDEN), (metadata_email, HTTPStatus.FORBIDDEN), (wildcard_email, HTTPStatus.OK), (USER_ADMIN_EMAIL, HTTPStatus.OK)],
ids=["scoped", "metadata_reader", "wildcard", "admin"],
)
def test_telemetry_routes_without_filter(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
method: str,
path: str,
query: dict | None,
body: dict | None,
email: str,
expected: HTTPStatus,
) -> None:
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics([Metrics(metric_name="explorer.requests", labels={"signoz.workspace.key.id": "key-a"}, timestamp=now - timedelta(minutes=i + 1), value=float(i)) for i in range(3)])
start, end = int((now - timedelta(minutes=10)).timestamp() * 1000), int(now.timestamp() * 1000)
token = get_token(email, USER_ADMIN_PASSWORD if email == USER_ADMIN_EMAIL else user_password)
response = requests.request(
method,
signoz.self.host_configs["8080"].get(path),
params={"start": start, "end": end, **query} if query is not None else None,
json={"start": start, "end": end, **body} if body is not None else None,
headers={"Authorization": f"Bearer {token}"},
timeout=10,
)
assert response.status_code == expected, response.text
@pytest.mark.parametrize(
("path", "query"),
[
("/api/v2/metrics", {"limit": 10}),
("/api/v2/metrics/attributes", {"metricName": "explorer.requests"}),
("/api/v2/metrics/metadata", {"metricName": "explorer.requests"}),
],
ids=["list", "attributes", "metadata"],
)
@pytest.mark.parametrize(
("email", "expected"),
[
(scoped_email, HTTPStatus.FORBIDDEN),
(wildcard_email, HTTPStatus.FORBIDDEN),
(metadata_email, HTTPStatus.OK),
(viewer_email, HTTPStatus.OK),
(USER_ADMIN_EMAIL, HTTPStatus.OK),
],
ids=["metrics_scoped", "metrics_wildcard", "metadata_reader", "viewer", "admin"],
)
def test_metadata_reads_need_metadata_grant(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
path: str,
query: dict,
email: str,
expected: HTTPStatus,
) -> None:
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics([Metrics(metric_name="explorer.requests", labels={"signoz.workspace.key.id": "key-a"}, timestamp=now - timedelta(minutes=i + 1), value=float(i)) for i in range(3)])
token = get_token(email, USER_ADMIN_PASSWORD if email == USER_ADMIN_EMAIL else user_password)
response = requests.get(
signoz.self.host_configs["8080"].get(path),
params={"start": int((now - timedelta(minutes=10)).timestamp() * 1000), "end": int(now.timestamp() * 1000), **query},
headers={"Authorization": f"Bearer {token}"},
timeout=10,
)
assert response.status_code == expected, response.text
if expected == HTTPStatus.FORBIDDEN:
assert "metrics-field" in response.text
@pytest.mark.parametrize(
("email", "expected"),
[
(scoped_email, HTTPStatus.FORBIDDEN),
(metadata_email, HTTPStatus.FORBIDDEN),
(viewer_email, HTTPStatus.FORBIDDEN),
(editor_email, HTTPStatus.OK),
(USER_ADMIN_EMAIL, HTTPStatus.OK),
],
ids=["metrics_scoped", "metadata_reader", "viewer", "editor", "admin"],
)
def test_metadata_update_needs_update_grant(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
email: str,
expected: HTTPStatus,
) -> None:
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics([Metrics(metric_name="explorer.requests", labels={"signoz.workspace.key.id": "key-a"}, timestamp=now - timedelta(minutes=i + 1), value=float(i)) for i in range(3)])
token = get_token(email, USER_ADMIN_PASSWORD if email == USER_ADMIN_EMAIL else user_password)
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v2/metrics/metadata"),
json={"metricName": "explorer.requests", "type": "sum", "description": "request count", "unit": "1", "temporality": "cumulative", "isMonotonic": True},
headers={"Authorization": f"Bearer {token}"},
timeout=10,
)
assert response.status_code == expected, response.text
@pytest.mark.parametrize(
("email", "expression", "expected", "expected_metrics"),
[
(scoped_email, "signoz.workspace.key.id = 'key-a'", HTTPStatus.OK, {"explorer.key.a"}),
(scoped_email, "signoz.workspace.key.id = 'key-b'", HTTPStatus.FORBIDDEN, set()),
(scoped_email, "signoz.workspace.key.id IN ('key-a', 'key-b')", HTTPStatus.FORBIDDEN, set()),
(wildcard_email, None, HTTPStatus.OK, {"explorer.key.a", "explorer.key.b"}),
],
ids=["scoped_granted_key", "scoped_other_key", "scoped_both_keys", "wildcard_unfiltered"],
)
def test_stats_scoped_read(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
email: str,
expression: str | None,
expected: HTTPStatus,
expected_metrics: set[str],
) -> None:
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics(
[Metrics(metric_name="explorer.key.a", labels={"signoz.workspace.key.id": "key-a"}, timestamp=now - timedelta(minutes=i + 1), value=float(i)) for i in range(3)]
+ [Metrics(metric_name="explorer.key.b", labels={"signoz.workspace.key.id": "key-b"}, timestamp=now - timedelta(minutes=i + 1), value=float(i)) for i in range(3)]
)
body: dict = {"start": int((now - timedelta(minutes=10)).timestamp() * 1000), "end": int(now.timestamp() * 1000), "limit": 50}
if expression:
body["filter"] = {"expression": expression}
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v2/metrics/stats"),
json=body,
headers={"Authorization": f"Bearer {get_token(email, user_password)}"},
timeout=10,
)
assert response.status_code == expected, response.text
if expected == HTTPStatus.OK:
assert {stat["metricName"] for stat in response.json()["data"]["metrics"]} == expected_metrics
@pytest.mark.parametrize(
("path", "body"),
[
("/api/v2/metrics/treemap", {"limit": 10, "mode": "samples"}),
("/api/v2/metrics/inspect", {"metricName": "explorer.key.a"}),
],
ids=["treemap", "inspect"],
)
def test_filtered_routes_allow_granted_key(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
path: str,
body: dict,
) -> None:
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics([Metrics(metric_name="explorer.key.a", labels={"signoz.workspace.key.id": "key-a"}, timestamp=now - timedelta(minutes=i + 1), value=float(i)) for i in range(3)])
start, end = int((now - timedelta(minutes=10)).timestamp() * 1000), int(now.timestamp() * 1000)
token = get_token(scoped_email, user_password)
allowed = requests.post(
signoz.self.host_configs["8080"].get(path),
json={"start": start, "end": end, "filter": {"expression": "signoz.workspace.key.id = 'key-a'"}, **body},
headers={"Authorization": f"Bearer {token}"},
timeout=10,
)
assert allowed.status_code == HTTPStatus.OK, allowed.text
denied = requests.post(
signoz.self.host_configs["8080"].get(path),
json={"start": start, "end": end, "filter": {"expression": "signoz.workspace.key.id = 'key-b'"}, **body},
headers={"Authorization": f"Bearer {token}"},
timeout=10,
)
assert denied.status_code == HTTPStatus.FORBIDDEN, denied.text
assert "builder_query/signoz.workspace.key.id/key-b" in denied.text
@pytest.mark.parametrize(
("path", "scope"),
[("/api/v2/metrics/alerts", "rule:list"), ("/api/v3/metrics/dashboards", "dashboard:list")],
ids=["alerts", "dashboards"],
)
@pytest.mark.parametrize(
("email", "expected"),
[(scoped_email, HTTPStatus.FORBIDDEN), (wildcard_email, HTTPStatus.FORBIDDEN), (USER_ADMIN_EMAIL, HTTPStatus.OK)],
ids=["metrics_scoped", "metrics_wildcard", "admin"],
)
def test_metaresource_lookups_need_list_grant(
signoz: types.SigNoz,
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
path: str,
scope: str,
email: str,
expected: HTTPStatus,
) -> None:
now = datetime.now(tz=UTC).replace(microsecond=0)
insert_metrics([Metrics(metric_name="explorer.requests", labels={"signoz.workspace.key.id": "key-a"}, timestamp=now - timedelta(minutes=i + 1), value=float(i)) for i in range(3)])
token = get_token(email, USER_ADMIN_PASSWORD if email == USER_ADMIN_EMAIL else user_password)
response = requests.get(
signoz.self.host_configs["8080"].get(path),
params={"metricName": "explorer.requests"},
headers={"Authorization": f"Bearer {token}"},
timeout=10,
)
assert response.status_code == expected, response.text
if expected == HTTPStatus.FORBIDDEN:
assert scope.split(":", maxsplit=1)[0] in response.text