Compare commits

...

27 Commits

Author SHA1 Message Date
swapnil-signoz
e769b208a5 refactor: updating dashboard and integration config 2026-07-30 00:51:31 +05:30
swapnil-signoz
3a6442bd70 Merge branch 'issue_5737' into issue-5739 2026-07-30 00:02:28 +05:30
swapnil-signoz
9685fb6591 Merge branch 'main' of https://github.com/SigNoz/signoz into issue-5739 2026-07-29 23:58:55 +05:30
swapnil-signoz
22359a24b6 Merge branch 'main' of https://github.com/SigNoz/signoz into issue_5737 2026-07-29 23:45:32 +05:30
swapnil-signoz
2c032a1874 feat: adding gcp cloud storage service 2026-07-29 23:42:24 +05:30
swapnil-signoz
3acb5e5d68 Merge branch 'issue_5737' into issue-5739 2026-07-29 23:40:49 +05:30
swapnil-signoz
8366a31edd refactor: updating icon in dashboard JSON 2026-07-29 23:15:09 +05:30
Vinicius Lourenço
b88ee12cd5 chore(docs): add authz guide (#12317)
* chore(docs): add authz guide

* chore(assets): clarify what withAuthZContent means
2026-07-29 17:18:15 +00:00
Pandey
6f3dd0b7ad fix(querybuilder): give each clickhouse sql refusal its own error code (#12339)
Some checks failed
build-staging / prepare (push) Has been cancelled
build-staging / js-build (push) Has been cancelled
build-staging / go-build (push) Has been cancelled
build-staging / staging (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
* fix(querybuilder): give each clickhouse sql refusal its own error code

Every refusal used CodeInvalidInput, so the only way to tell why a statement was
refused was to match on the message. Counting the failures on a live deployment
meant regexing the parser's text out of the log, and grouping by cause meant
knowing which of five different messages belonged to the same underlying gap.

One code per reason instead, which arrives as exception.code on the warning that
LogIfStatementIsNotValid writes, so the failures can be grouped and alerted on
directly.

Pin the expected code on each failing case, so the mapping is checked rather
than assumed.

* chore(querybuilder): drop the comment above the error codes

The names say it.

* fix(querybuilder): separate the parser panic code from the parse failure

A panic is a parser defect worth alerting on; a parse failure is a grammar gap
that shows up in ordinary traffic. Sharing one code lumps the two together.
2026-07-29 15:36:10 +00:00
Pandey
d32d93cd39 chore(deps): bump clickhouse-sql-parser to v0.5.3 (#12331)
* chore(deps): bump clickhouse-sql-parser to v0.5.3

v0.5.3 stops lexing a signed number after a closing bracket as one literal, so
`(now()-1)` and `arr[1]-1` parse where they used to fail. That was the third of
the three gaps the validator trips over on real dashboard SQL, and the only one
where the statement had to be rewritten to be accepted.

Move the two queries it covers out of the failing set, which is what that test
exists to prompt. On the production corpus the rejection rate goes from 6.1% to
5.6% of v5 query shapes; the remaining two gaps, `interval` used as a column name
and the standard trim(BOTH x FROM y) syntax, are still open upstream.

* chore(querybuilder): link the upstream issue on the signed literal cases
2026-07-29 15:33:00 +00:00
swapnil-signoz
28f10b3567 chore: generating openapi spec 2026-07-29 20:28:55 +05:30
swapnil-signoz
5d6ee1b97a Merge branch 'main' of https://github.com/SigNoz/signoz into issue_5737 2026-07-29 20:16:24 +05:30
swapnil-signoz
a61cb1fc15 feat: adding gcp gke service 2026-07-29 20:15:24 +05:30
swapnil-signoz
642c49db37 refactor: migrating dashboard to v6 2026-07-29 01:32:45 +05:30
swapnil-signoz
f98d2291fd Merge branch 'main' into issue_5721 2026-07-28 23:01:05 +05:30
swapnil-signoz
22833ea8fe Merge branch 'issue_5721' of https://github.com/SigNoz/signoz into issue_5721 2026-07-22 17:30:14 +05:30
swapnil-signoz
8eb0a24492 fix: correct typo and unit in compute engine dashboard 2026-07-22 17:29:22 +05:30
swapnil-signoz
7e0b19800c Merge branch 'main' into issue_5721 2026-07-22 17:13:37 +05:30
Swapnil Nakade
3bc476c2ed Merge branch 'main' into issue_5721 2026-07-22 16:32:23 +05:30
swapnil-signoz
6a834ee944 Merge branch 'main' into issue_5721 2026-07-22 12:57:39 +05:30
swapnil-signoz
ba6a78aea7 refactor: updating dashboard panels to use rate aggregation 2026-07-22 10:38:46 +05:30
swapnil-signoz
ca98231b18 feat: adding compute engine service 2026-07-22 10:10:59 +05:30
swapnil-signoz
1644ecd6fc refactor: updating dashboard panel to use rate function instead of hack 2026-07-22 01:32:13 +05:30
swapnil-signoz
f1e2e9f4f7 refactor: updating cpu utilization panel 2026-07-19 22:25:47 +05:30
swapnil-signoz
a59e372f07 refactor: extending width of uptime gauge panel 2026-07-14 22:59:52 +05:30
swapnil-signoz
ca368a5b38 refactor: updating dashboard title 2026-07-14 22:51:25 +05:30
swapnil-signoz
dab5ceee0d feat: adding gcp memorystore redis service 2026-07-14 22:40:31 +05:30
22 changed files with 3202 additions and 55 deletions

View File

@@ -1497,6 +1497,8 @@ components:
- cloudsql_postgres
- memorystore_redis
- computeengine
- gke
- cloudstorage
type: string
CloudintegrationtypesServiceMetadata:
properties:

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 139 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 49 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 86 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 64 KiB

View File

@@ -0,0 +1,136 @@
# AuthZ Guide
How to structure a page so it works with the permission system.
We are migrating from the `ADMIN | EDITOR | VIEWER` roles to per-action checks. Instead of granting `VIEWER` and
exposing everything, a user can now be granted access to a single resource, eg: `Logs` only.
- [Prerequisites](#prerequisites)
- [Core rules](#core-rules)
- [Page patterns](#page-patterns)
- [What "blocked" means](#what-blocked-means)
- [Migration checklist](#migration-checklist)
- [Testing](#testing)
## Prerequisites
Check whether the resource your page represents is supported: see
[permissions.type.ts](../src/lib/authz/hooks/useAuthZ/permissions.type.ts) for the current resources and their allowed
verbs.
**If the resource is not listed there, skip authz for now**. The backend does not enforce it yet, so any frontend
check would be decorative. Revisit once the resource is generated into that file (it is auto-generated from the
backend).
## Core rules
These hold for every page. The per-pattern sections below only add to them.
1. **A page is always reachable, regardless of permission.** The intended action must be known before permission can
be checked, so the route renders first and individual pieces are gated.
2. **`update` requires both `read` and `update`.** A user who can write but cannot read the current value must not get
an edit affordance.
3. **`delete` is independent of `read`.** Delete controls stay visible for a user who holds `delete` but not `read`.
4. **Never gate per row.** If a user can `list`, render every row. Check `read` only when the row is opened (drawer or
detail route).
5. **Gate the narrowest thing that works**, a button over a section, a section over a page.
6. **A resource may be gated while a sub-resource is not.** A user without `read` on Service Accounts can still hold
`create` on API Keys, so blocking the outer container would hide work they are allowed to do.
7. **Verbs not covered here** (`attach`, `detach`, `assignee`) behave like `delete`: gate the control that triggers
them, not the surrounding content.
## Page patterns
### List page
![Visual rules for structuring a list page](./assets/list-example.svg)
Without `list`, but with any of `read` / `create` / `update`, only the table is blocked:
- Title, description, search filters and action buttons stay visible.
- Filters and any control that drives the table are non-interactive.
- The create button stays enabled if the user holds `create`.
### Edit page
![Visual rules for structuring an edit page](./assets/edit-example.svg)
Without `read`:
- Block the content with `withAuthZContent`, not the whole route.
- Keep delete visible (rule 3).
- Keep update blocked (rule 2).
### Drawer
![Visual rules for structuring a drawer](./assets/drawer-example.svg)
Without `read`:
- Block the drawer body, not the drawer itself.
- Prefer routing that carries the resource ID in the URL, so delete stays reachable without `read`.
- Keep update blocked (rule 2).
- Watch for sub-resources the user may still be allowed to act on (rule 6).
### Create
Without `create`:
| Entry point | Gate with |
| --------------------- | ------------------------------------------------------------- |
| Button | `AuthZButton` |
| Dedicated create page | `withAuthZPage` |
| Create drawer opened directly (deep link) | `withAuthZContent` on the body + `AuthZButton` on footer actions |
### Delete
Gate the control with `AuthZButton`, or `AuthZTooltip` for a non-button trigger such as an icon or menu item.
### Quick filters
![Visual rules for structuring a page with quick filters](./assets/quick-filters-example.svg)
## What "blocked" means
Blocking is always a visible denial, never a silent removal. Use the components in
[`lib/authz/components`](../src/lib/authz/components/README.md) rather than hand-rolling a check, they carry the
denial message and the loading state.
| Scope | Component | Denied state |
| --------------- | ----------------------------------------- | ------------------------------------------- |
| Button | `AuthZButton` | Disabled + tooltip |
| Any element | `AuthZTooltip` | Disabled child + tooltip |
| Section | `withAuthZContent` / `AuthZGuardContent` | Inline `PermissionDeniedCallout` |
| Page / route | `withAuthZPage` / `AuthZGuardPage` | `PermissionDeniedFullPage` |
| Custom fallback | `withAuthZ` / `AuthZGuard` | Whatever you pass as `fallback` |
Prefer the HOC (`withAuthZ*`); reach for the JSX guard (`AuthZGuard*`) when the gate depends on conditional rendering
and an HOC cannot express it. The components README has the full decision tree and how to build the `checks` array.
## Migration checklist
Use the existing components. Create a new one only if none fit.
- [ ] Can I apply a `withAuthZ*` variant directly, or must I extract the content into a component first?
- If extraction is needed, declare the new content component in the same file to keep the diff small, then move it
to its own file in a follow-up commit or PR.
- [ ] Is the layout structured to respect the [core rules](#core-rules)?
- If not, raise it in the frontend Slack channel before reshaping the page.
- [ ] Am I gating a shared component rather than a page or section?
- If so, it must stay functional with no permission. Example: the Query Builder works without field suggestions when
the user cannot read them.
## Testing
### Devtool
Press `Cmd + K` (`Ctrl + K` on Windows/Linux) to open the shortcuts, search for `AuthZ`, and pick the first
result. The devtool simulates granted and denied permissions across the UI.
Available only in local development, it is stripped from production builds.
### Unit tests
[`lib/authz/utils/README.md`](../src/lib/authz/utils/README.md) covers the `*.authz.test.tsx` naming convention, the
MSW handlers (`setupAuthzAdmin`, `setupAuthzDenyAll`, `setupAuthzDeny`, `setupAuthzAllow`,
`setupAuthzGrantByPrefix`), and how to test the loading state.

View File

@@ -2816,6 +2816,7 @@ export enum CloudintegrationtypesServiceIDDTO {
cloudsql_postgres = 'cloudsql_postgres',
memorystore_redis = 'memorystore_redis',
computeengine = 'computeengine',
gke = 'gke',
}
export type CloudintegrationtypesCloudIntegrationServiceDTOAnyOf = {
/**

View File

@@ -1 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="1200" height="800" viewBox="-19.2 -28.483 166.401 170.898"><g transform="translate(0 -7.034)"><linearGradient id="a" x1="64" x2="64" y1="7.034" y2="120.789" gradientUnits="userSpaceOnUse"><stop offset="0" stop-color="#4387fd"/><stop offset="1" stop-color="#4683ea"/></linearGradient><path fill="url(#a)" d="M27.79 115.217 1.54 69.749a11.5 11.5 0 0 1 0-11.499l26.25-45.467a11.5 11.5 0 0 1 9.96-5.75h52.5a11.5 11.5 0 0 1 9.959 5.75l26.25 45.467a11.5 11.5 0 0 1 0 11.5l-26.25 45.466a11.5 11.5 0 0 1-9.959 5.75h-52.5a11.5 11.5 0 0 1-9.96-5.75z"/></g><g transform="translate(0 -7.034)"><defs><path id="b" d="M27.791 115.217 1.541 69.749a11.5 11.5 0 0 1 0-11.499l26.25-45.467a11.5 11.5 0 0 1 9.959-5.75h52.5a11.5 11.5 0 0 1 9.96 5.75l26.25 45.467a11.5 11.5 0 0 1 0 11.5l-26.25 45.466a11.5 11.5 0 0 1-9.96 5.75h-52.5a11.5 11.5 0 0 1-9.959-5.75z"/></defs><clipPath id="c"><use xlink:href="#b" width="100%" height="100%" overflow="visible"/></clipPath><path d="m49.313 53.875-7.01 6.99 5.957 5.958-5.898 10.476 44.635 44.636 10.816.002L118.936 84 85.489 50.55z" clip-path="url(#c)" opacity=".07"/></g><path fill="#fff" d="M84.7 43.236H43.264c-.667 0-1.212.546-1.212 1.214v8.566c0 .666.546 1.212 1.212 1.212H84.7c.667 0 1.213-.546 1.213-1.212v-8.568c0-.666-.545-1.213-1.212-1.213m-6.416 7.976a2.484 2.484 0 0 1-2.477-2.48 2.475 2.475 0 0 1 2.477-2.477c1.37 0 2.48 1.103 2.48 2.477a2.48 2.48 0 0 1-2.48 2.48m6.415 8.491-41.436.002c-.667 0-1.212.546-1.212 1.214v8.565c0 .666.546 1.213 1.212 1.213H84.7c.667 0 1.213-.547 1.213-1.213v-8.567c0-.666-.545-1.214-1.212-1.214m-6.416 7.976a2.483 2.483 0 0 1-2.477-2.48 2.475 2.475 0 0 1 2.477-2.477 2.48 2.48 0 1 1 0 4.956"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" width="24px" height="24px" viewBox="0 0 24 24"><defs><style>.cls-1{fill:#aecbfa;}.cls-2{fill:#669df6;}.cls-3{fill:#4285f4;}</style></defs><title>Icon_24px_CloudStorage_Color</title><g data-name="Product Icons"><path class="cls-1" d="M2,4.5H22a0,0,0,0,1,0,0v5a0,0,0,0,1,0,0H2a0,0,0,0,1,0,0v-5A0,0,0,0,1,2,4.5Z"/><path class="cls-2" d="M2,14.5H22a0,0,0,0,1,0,0v5a0,0,0,0,1,0,0H2a0,0,0,0,1,0,0v-5A0,0,0,0,1,2,14.5Z"/><rect class="cls-3" x="4.5" y="6.33" width="3.33" height="1.33"/><rect class="cls-3" x="4.5" y="16.33" width="3.33" height="1.33"/><circle class="cls-3" cx="18.5" cy="7" r="1"/><circle class="cls-3" cx="18.5" cy="17" r="1"/></g></svg>

Before

Width:  |  Height:  |  Size: 1.7 KiB

After

Width:  |  Height:  |  Size: 689 B

View File

@@ -1 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24"><path d="M19.73 6.56a1.73 1.73 0 0 0-1.68 1.68 1.83 1.83 0 0 0 .89 1.48v6.9l-5.16 3.06.8 1.28 5.55-3.25a.84.84 0 0 0 .4-.69v-7.3a1.64 1.64 0 0 0 .89-1.48 1.61 1.61 0 0 0-1.69-1.68" style="fill:#aecbfa"/><path d="m18 5.48-5.61-3.16a1.18 1.18 0 0 0-.79 0L5.25 6a1.72 1.72 0 0 0-2.68 1.35A1.73 1.73 0 0 0 4.26 9a1.73 1.73 0 0 0 1.68-1.65L12 3.9l5.15 3ZM11.2 18.5a1.57 1.57 0 0 0-.89.29l-5.16-3V9.92H3.56v6.31a.84.84 0 0 0 .4.69L9.52 20v.09a1.69 1.69 0 0 0 3.37 0 1.65 1.65 0 0 0-1.69-1.59" data-name="Path" style="fill:#aecbfa"/><path d="M16.96 8.63 12.1 5.78 7.13 8.63l4.97 2.77z" data-name="Path" style="fill:#669df6"/><path d="M12.1 12.38 6.84 9.32v2.47l5.26 2.96zM12.1 15.73l-5.26-3.05v2.07l5.26 3.06z" style="fill:#669df6"/><path d="M12.09 12.38v2.37l5.26-3.06V9.33Zm4.32-.94a.38.38 0 1 1 0-.76.38.38 0 0 1 0 .76M12.09 15.73v2.07l5.26-3.05v-2.07Zm4.32-1.07a.38.38 0 1 1 0-.76.38.38 0 0 1 0 .76" style="fill:#4285f4"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" width="24px" height="24px" viewBox="0 0 24 24"><defs><style>.cls-1{fill:#4285f4;}.cls-1,.cls-2,.cls-4{fill-rule:evenodd;}.cls-2{fill:#669df6;}.cls-3,.cls-4{fill:#aecbfa;}</style></defs><title>Icon_24px_K8Engine_Color</title><g data-name="Product Icons"><g ><polygon class="cls-1" points="14.68 13.06 19.23 15.69 19.23 16.68 14.29 13.83 14.68 13.06"/><polygon class="cls-2" points="9.98 13.65 4.77 16.66 4.45 15.86 9.53 12.92 9.98 13.65"/><rect class="cls-3" x="11.55" y="3.29" width="0.86" height="5.78"/><path class="cls-4" d="M3.25,7V17L12,22l8.74-5V7L12,2Zm15.63,8.89L12,19.78,5.12,15.89V8.11L12,4.22l6.87,3.89v7.78Z"/><polygon class="cls-4" points="11.98 11.5 15.96 9.21 11.98 6.91 8.01 9.21 11.98 11.5"/><polygon class="cls-2" points="11.52 12.3 7.66 10.01 7.66 14.6 11.52 16.89 11.52 12.3"/><polygon class="cls-1" points="12.48 12.3 12.48 16.89 16.34 14.6 16.34 10.01 12.48 12.3"/></g></g></svg>

Before

Width:  |  Height:  |  Size: 988 B

After

Width:  |  Height:  |  Size: 941 B

2
go.mod
View File

@@ -4,7 +4,7 @@ go 1.25.7
require (
dario.cat/mergo v1.0.2
github.com/AfterShip/clickhouse-sql-parser v0.5.2
github.com/AfterShip/clickhouse-sql-parser v0.5.3
github.com/ClickHouse/clickhouse-go/v2 v2.44.0
github.com/DATA-DOG/go-sqlmock v1.5.2
github.com/SigNoz/clickhouse-go-mock v0.14.0

4
go.sum
View File

@@ -66,8 +66,8 @@ dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/AfterShip/clickhouse-sql-parser v0.5.2 h1:KCxdmvuVawVF4yl0ZS33q7olgmLZwvZG5BjWHp6o414=
github.com/AfterShip/clickhouse-sql-parser v0.5.2/go.mod h1:Qi3qvPTfZb/aFwI5V4WFOahgjsLJa4MzVijIAfwOhDw=
github.com/AfterShip/clickhouse-sql-parser v0.5.3 h1:6iap8XGjuSjD3w7r1UNrg66ljBugcv2P39s4eo/ZLRw=
github.com/AfterShip/clickhouse-sql-parser v0.5.3/go.mod h1:Qi3qvPTfZb/aFwI5V4WFOahgjsLJa4MzVijIAfwOhDw=
github.com/Azure/azure-sdk-for-go v68.0.0+incompatible h1:fcYLmCpyNYRnvJbPerq7U0hS+6+I79yEDJBqVNcqUzU=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0 h1:fou+2+WFTib47nS+nz/ozhEBnvU96bKHy6LjRsY4E28=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0/go.mod h1:t76Ruy8AHvUAC8GfMWJMa0ElSbuIcO03NLpynfbgsPA=

View File

@@ -0,0 +1,956 @@
{
"schemaVersion": "v6",
"image": "/assets/Logos/gcp-cloud-storage",
"name": "",
"generateName": true,
"tags": [
{
"key": "tag",
"value": "observability"
}
],
"spec": {
"display": {
"name": "GCP Cloud Storage Overview",
"description": "Dashboard for GCP Cloud Storage overview"
},
"variables": [
{
"kind": "ListVariable",
"spec": {
"display": {
"name": "project_id",
"description": "GCP Project ID"
},
"allowAllValue": false,
"allowMultiple": false,
"customAllValue": "",
"capturingRegexp": "",
"sort": "none",
"plugin": {
"kind": "signoz/DynamicVariable",
"spec": {
"name": "project_id",
"signal": "metrics"
}
},
"name": "project_id"
}
},
{
"kind": "ListVariable",
"spec": {
"display": {
"name": "bucket_name",
"description": "GCS bucket name"
},
"allowAllValue": true,
"allowMultiple": true,
"customAllValue": "",
"capturingRegexp": "",
"sort": "none",
"plugin": {
"kind": "signoz/DynamicVariable",
"spec": {
"name": "bucket_name",
"signal": "metrics"
}
},
"name": "bucket_name"
}
}
],
"panels": {
"1f0a4d5b-6c2e-4a71-8b3d-9e5c07a41d20": {
"kind": "Panel",
"spec": {
"display": {
"name": "Bucket size",
"description": "Total size of all objects in the bucket, grouped by storage class."
},
"plugin": {
"kind": "signoz/TablePanel",
"spec": {
"visualization": {
"timePreference": "global_time"
},
"formatting": {
"columnUnits": {
"A": "By"
},
"decimalPrecision": "2"
},
"thresholds": null
}
},
"queries": [
{
"kind": "scalar",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/storage/v2/total_bytes",
"temporality": "",
"timeAggregation": "latest",
"spaceAggregation": "sum",
"reduceTo": "last"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "storage_class",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "Size"
}
}
}
}
],
"links": []
}
},
"29ed9824-21db-44c3-9537-12be458f5c20": {
"kind": "Panel",
"spec": {
"display": {
"name": "Total object bytes",
"description": "Total size of all objects in the bucket, grouped by storage class and type."
},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {
"visualization": {
"timePreference": "global_time",
"fillSpans": false
},
"formatting": {
"unit": "By",
"decimalPrecision": "2"
},
"chartAppearance": {
"lineInterpolation": "spline",
"showPoints": false,
"lineStyle": "solid",
"fillMode": "none",
"spanGaps": {
"fillOnlyBelow": false,
"fillLessThan": ""
}
},
"axes": {
"softMin": 0,
"softMax": 0,
"isLogScale": false
},
"legend": {
"position": "right",
"mode": "list",
"customColors": null
},
"thresholds": null
}
},
"queries": [
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/storage/v2/total_bytes",
"temporality": "",
"timeAggregation": "latest",
"spaceAggregation": "sum",
"reduceTo": "avg"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "storage_class",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "type",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{bucket_name}}/{{storage_class}}/{{type}}"
}
}
}
}
],
"links": []
}
},
"cbc53174-5527-4bce-b32d-317403849e48": {
"kind": "Panel",
"spec": {
"display": {
"name": "Total object count",
"description": "Total number of objects and multipart-uploads per bucket, grouped by storage class and type, where type can be live-object, noncurrent-object, soft-deleted-object or multipart-upload."
},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {
"visualization": {
"timePreference": "global_time",
"fillSpans": false
},
"formatting": {
"unit": "{count}",
"decimalPrecision": "2"
},
"chartAppearance": {
"lineInterpolation": "spline",
"showPoints": false,
"lineStyle": "solid",
"fillMode": "none",
"spanGaps": {
"fillOnlyBelow": false,
"fillLessThan": ""
}
},
"axes": {
"softMin": 0,
"softMax": 0,
"isLogScale": false
},
"legend": {
"position": "right",
"mode": "list",
"customColors": null
},
"thresholds": null
}
},
"queries": [
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/storage/v2/total_count",
"temporality": "",
"timeAggregation": "latest",
"spaceAggregation": "sum",
"reduceTo": "avg"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "storage_class",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "type",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{bucket_name}}/{{storage_class}}/{{type}}"
}
}
}
}
],
"links": []
}
},
"76e70ce5-47e4-4b5b-9715-49fa21c58b0c": {
"kind": "Panel",
"spec": {
"display": {
"name": "Deleted bytes",
"description": "Rate of deleted bytes per bucket, grouped by storage class."
},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {
"visualization": {
"timePreference": "global_time",
"fillSpans": false
},
"formatting": {
"unit": "By/s",
"decimalPrecision": "2"
},
"chartAppearance": {
"lineInterpolation": "spline",
"showPoints": false,
"lineStyle": "solid",
"fillMode": "none",
"spanGaps": {
"fillOnlyBelow": false,
"fillLessThan": ""
}
},
"axes": {
"softMin": 0,
"softMax": 0,
"isLogScale": false
},
"legend": {
"position": "right",
"mode": "list",
"customColors": null
},
"thresholds": null
}
},
"queries": [
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/storage/v2/deleted_bytes",
"temporality": "",
"timeAggregation": "rate",
"spaceAggregation": "sum",
"reduceTo": "sum"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "storage_class",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{bucket_name}}/{{storage_class}}"
}
}
}
}
],
"links": []
}
},
"c5af2813-34ec-41a9-8d53-c9acd67f569f": {
"kind": "Panel",
"spec": {
"display": {
"name": "Request count",
"description": "Rate of API calls, grouped by the API method name and response code."
},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {
"visualization": {
"timePreference": "global_time",
"fillSpans": false
},
"formatting": {
"unit": "{count}/s",
"decimalPrecision": "2"
},
"chartAppearance": {
"lineInterpolation": "spline",
"showPoints": false,
"lineStyle": "solid",
"fillMode": "none",
"spanGaps": {
"fillOnlyBelow": false,
"fillLessThan": ""
}
},
"axes": {
"softMin": 0,
"softMax": 0,
"isLogScale": false
},
"legend": {
"position": "right",
"mode": "list",
"customColors": null
},
"thresholds": null
}
},
"queries": [
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/api/request_count",
"temporality": "",
"timeAggregation": "rate",
"spaceAggregation": "sum",
"reduceTo": "sum"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "method",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "response_code",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{bucket_name}}/{{method}} - {{response_code}}"
}
}
}
}
],
"links": []
}
},
"70333210-68c3-4387-a6b7-afa2459d27dd": {
"kind": "Panel",
"spec": {
"display": {
"name": "Authentication count",
"description": "Rate of HMAC/RSA signed requests, grouped by authentication method, API method name and response code."
},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {
"visualization": {
"timePreference": "global_time",
"fillSpans": false
},
"formatting": {
"unit": "{count}/s",
"decimalPrecision": "2"
},
"chartAppearance": {
"lineInterpolation": "spline",
"showPoints": false,
"lineStyle": "solid",
"fillMode": "none",
"spanGaps": {
"fillOnlyBelow": false,
"fillLessThan": ""
}
},
"axes": {
"softMin": 0,
"softMax": 0,
"isLogScale": false
},
"legend": {
"position": "right",
"mode": "list",
"customColors": null
},
"thresholds": null
}
},
"queries": [
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/authn/authentication_count",
"temporality": "",
"timeAggregation": "rate",
"spaceAggregation": "sum",
"reduceTo": "sum"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "authentication_method",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "method",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "response_code",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{bucket_name}}/{{authentication_method}}/{{method}} - {{response_code}}"
}
}
}
}
],
"links": []
}
},
"7a98084c-2cd3-4146-a831-f5382162ffe5": {
"kind": "Panel",
"spec": {
"display": {
"name": "Received bytes",
"description": "Rate of bytes received over the network, grouped by the API method name and response code."
},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {
"visualization": {
"timePreference": "global_time",
"fillSpans": false
},
"formatting": {
"unit": "By/s",
"decimalPrecision": "2"
},
"chartAppearance": {
"lineInterpolation": "spline",
"showPoints": false,
"lineStyle": "solid",
"fillMode": "none",
"spanGaps": {
"fillOnlyBelow": false,
"fillLessThan": ""
}
},
"axes": {
"softMin": 0,
"softMax": 0,
"isLogScale": false
},
"legend": {
"position": "right",
"mode": "list",
"customColors": null
},
"thresholds": null
}
},
"queries": [
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/network/received_bytes_count",
"temporality": "",
"timeAggregation": "rate",
"spaceAggregation": "sum",
"reduceTo": "sum"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "method",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "response_code",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{bucket_name}}/{{method}} - {{response_code}}"
}
}
}
}
],
"links": []
}
},
"7138bf98-b41a-46d8-90ef-8e7470819cdf": {
"kind": "Panel",
"spec": {
"display": {
"name": "Sent bytes",
"description": "Rate of bytes sent over the network, grouped by the API method name and response code."
},
"plugin": {
"kind": "signoz/TimeSeriesPanel",
"spec": {
"visualization": {
"timePreference": "global_time",
"fillSpans": false
},
"formatting": {
"unit": "By/s",
"decimalPrecision": "2"
},
"chartAppearance": {
"lineInterpolation": "spline",
"showPoints": false,
"lineStyle": "solid",
"fillMode": "none",
"spanGaps": {
"fillOnlyBelow": false,
"fillLessThan": ""
}
},
"axes": {
"softMin": 0,
"softMax": 0,
"isLogScale": false
},
"legend": {
"position": "right",
"mode": "list",
"customColors": null
},
"thresholds": null
}
},
"queries": [
{
"kind": "time_series",
"spec": {
"name": "A",
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "metrics",
"source": "",
"aggregations": [
{
"metricName": "storage.googleapis.com/network/sent_bytes_count",
"temporality": "",
"timeAggregation": "rate",
"spaceAggregation": "sum",
"reduceTo": "sum"
}
],
"disabled": false,
"filter": {
"expression": "project_id = $project_id AND bucket_name in $bucket_name AND gcp.resource_type = 'gcs_bucket'"
},
"groupBy": [
{
"name": "bucket_name",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "method",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
},
{
"name": "response_code",
"signal": "",
"fieldContext": "attribute",
"fieldDataType": "string"
}
],
"order": [],
"having": {
"expression": ""
},
"functions": [],
"legend": "{{bucket_name}}/{{method}} - {{response_code}}"
}
}
}
}
],
"links": []
}
}
},
"layouts": [
{
"kind": "Grid",
"spec": {
"items": [
{
"x": 0,
"y": 0,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/1f0a4d5b-6c2e-4a71-8b3d-9e5c07a41d20"
}
}
]
}
},
{
"kind": "Grid",
"spec": {
"display": {
"title": "Storage",
"collapse": {
"open": true
}
},
"items": [
{
"x": 0,
"y": 0,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/29ed9824-21db-44c3-9537-12be458f5c20"
}
},
{
"x": 6,
"y": 0,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/cbc53174-5527-4bce-b32d-317403849e48"
}
},
{
"x": 0,
"y": 6,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/76e70ce5-47e4-4b5b-9715-49fa21c58b0c"
}
}
]
}
},
{
"kind": "Grid",
"spec": {
"display": {
"title": "API & Authentication",
"collapse": {
"open": true
}
},
"items": [
{
"x": 0,
"y": 0,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/c5af2813-34ec-41a9-8d53-c9acd67f569f"
}
},
{
"x": 6,
"y": 0,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/70333210-68c3-4387-a6b7-afa2459d27dd"
}
}
]
}
},
{
"kind": "Grid",
"spec": {
"display": {
"title": "Network",
"collapse": {
"open": true
}
},
"items": [
{
"x": 0,
"y": 0,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/7a98084c-2cd3-4146-a831-f5382162ffe5"
}
},
{
"x": 6,
"y": 0,
"width": 6,
"height": 6,
"content": {
"$ref": "#/spec/panels/7138bf98-b41a-46d8-90ef-8e7470819cdf"
}
}
]
}
}
],
"duration": "",
"refreshInterval": "",
"links": []
}
}

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24px" height="24px" viewBox="0 0 24 24"><defs><style>.cls-1{fill:#aecbfa;}.cls-2{fill:#669df6;}.cls-3{fill:#4285f4;}</style></defs><title>Icon_24px_CloudStorage_Color</title><g data-name="Product Icons"><path class="cls-1" d="M2,4.5H22a0,0,0,0,1,0,0v5a0,0,0,0,1,0,0H2a0,0,0,0,1,0,0v-5A0,0,0,0,1,2,4.5Z"/><path class="cls-2" d="M2,14.5H22a0,0,0,0,1,0,0v5a0,0,0,0,1,0,0H2a0,0,0,0,1,0,0v-5A0,0,0,0,1,2,14.5Z"/><rect class="cls-3" x="4.5" y="6.33" width="3.33" height="1.33"/><rect class="cls-3" x="4.5" y="16.33" width="3.33" height="1.33"/><circle class="cls-3" cx="18.5" cy="7" r="1"/><circle class="cls-3" cx="18.5" cy="17" r="1"/></g></svg>

After

Width:  |  Height:  |  Size: 689 B

View File

@@ -0,0 +1,70 @@
{
"id": "cloudstorage",
"title": "GCP Cloud Storage",
"icon": "file://icon.svg",
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": true
},
"dataCollected": {
"metrics": [
{
"name": "storage.googleapis.com/storage/v2/total_bytes",
"unit": "Bytes",
"type": "Gauge",
"description": ""
},
{
"name": "storage.googleapis.com/storage/v2/total_count",
"unit": "Count",
"type": "Gauge",
"description": ""
},
{
"name": "storage.googleapis.com/storage/v2/deleted_bytes",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "storage.googleapis.com/api/request_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "storage.googleapis.com/network/received_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "storage.googleapis.com/network/sent_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "storage.googleapis.com/authn/authentication_count",
"unit": "Count",
"type": "Sum",
"description": ""
}
],
"logs": []
},
"telemetryCollectionStrategy": {
"gcp": {}
},
"assets": {
"dashboards": [
{
"id": "overview",
"title": "GCP Cloud Storage Overview",
"description": "Overview of GCP Cloud Storage metrics",
"definition": "file://assets/dashboards/overview.json"
}
]
}
}

View File

@@ -0,0 +1,3 @@
### Monitor GCP Cloud Storage with SigNoz
Collect key GCP Cloud Storage metrics and view them with an out of the box dashboard.

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24px" height="24px" viewBox="0 0 24 24"><defs><style>.cls-1{fill:#4285f4;}.cls-1,.cls-2,.cls-4{fill-rule:evenodd;}.cls-2{fill:#669df6;}.cls-3,.cls-4{fill:#aecbfa;}</style></defs><title>Icon_24px_K8Engine_Color</title><g data-name="Product Icons"><g ><polygon class="cls-1" points="14.68 13.06 19.23 15.69 19.23 16.68 14.29 13.83 14.68 13.06"/><polygon class="cls-2" points="9.98 13.65 4.77 16.66 4.45 15.86 9.53 12.92 9.98 13.65"/><rect class="cls-3" x="11.55" y="3.29" width="0.86" height="5.78"/><path class="cls-4" d="M3.25,7V17L12,22l8.74-5V7L12,2Zm15.63,8.89L12,19.78,5.12,15.89V8.11L12,4.22l6.87,3.89v7.78Z"/><polygon class="cls-4" points="11.98 11.5 15.96 9.21 11.98 6.91 8.01 9.21 11.98 11.5"/><polygon class="cls-2" points="11.52 12.3 7.66 10.01 7.66 14.6 11.52 16.89 11.52 12.3"/><polygon class="cls-1" points="12.48 12.3 12.48 16.89 16.34 14.6 16.34 10.01 12.48 12.3"/></g></g></svg>

After

Width:  |  Height:  |  Size: 941 B

View File

@@ -0,0 +1,124 @@
{
"id": "gke",
"title": "GCP Kubernetes Engine",
"icon": "file://icon.svg",
"overview": "file://overview.md",
"supportedSignals": {
"metrics": true,
"logs": true
},
"dataCollected": {
"metrics": [
{
"name": "kubernetes.io/container/cpu/limit_utilization",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/container/cpu/request_utilization",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/container/memory/limit_utilization",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/container/memory/request_utilization",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/container/restart_count",
"unit": "Count",
"type": "Sum",
"description": ""
},
{
"name": "kubernetes.io/pod/latencies/pod_first_ready",
"unit": "Seconds",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/pod/network/received_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "kubernetes.io/pod/network/sent_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "kubernetes.io/pod/volume/utilization",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/node/cpu/allocatable_utilization",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/node/memory/allocatable_utilization",
"unit": "None",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/node/ephemeral_storage/used_bytes",
"unit": "Bytes",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/node/ephemeral_storage/allocatable_bytes",
"unit": "Bytes",
"type": "Gauge",
"description": ""
},
{
"name": "kubernetes.io/node/network/received_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "kubernetes.io/node/network/sent_bytes_count",
"unit": "Bytes",
"type": "Sum",
"description": ""
},
{
"name": "kubernetes.io/node/status_condition",
"unit": "None",
"type": "Gauge",
"description": ""
}
],
"logs": []
},
"telemetryCollectionStrategy": {
"gcp": {}
},
"assets": {
"dashboards": [
{
"id": "overview",
"title": "GCP Kubernetes Engine Overview",
"description": "Overview of GCP Kubernetes Engine metrics",
"definition": "file://assets/dashboards/overview.json"
}
]
}
}

View File

@@ -0,0 +1,3 @@
### Monitor GCP Kubernetes Engine with SigNoz
Collect key GCP Kubernetes Engine metrics and view them with an out of the box dashboard.

View File

@@ -9,6 +9,16 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
)
var (
CodeClickHouseSQLParserPanic = errors.MustNewCode("clickhouse_sql_parser_panic")
CodeClickHouseSQLUnparseable = errors.MustNewCode("clickhouse_sql_unparseable")
CodeClickHouseSQLNotSingleStatement = errors.MustNewCode("clickhouse_sql_not_single_statement")
CodeClickHouseSQLNotSelect = errors.MustNewCode("clickhouse_sql_not_select")
CodeClickHouseSQLTableFunction = errors.MustNewCode("clickhouse_sql_table_function")
CodeClickHouseSQLInternalDatabase = errors.MustNewCode("clickhouse_sql_internal_database")
CodeClickHouseSQLReadonlyOverride = errors.MustNewCode("clickhouse_sql_readonly_override")
)
// internalDatabases hold server metadata, credentials and grants rather than telemetry.
var internalDatabases = map[string]struct{}{
"system": {},
@@ -20,30 +30,30 @@ func ErrIfStatementIsNotValid(query string) (err error) {
defer func() {
// The parser has a history of panicking on malformed input rather than returning an error.
if recovered := recover(); recovered != nil {
err = errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid ClickHouse SQL (recovered): %v", recovered)
err = errors.NewInvalidInputf(CodeClickHouseSQLParserPanic, "invalid ClickHouse SQL (recovered): %v", recovered)
}
}()
stmts, parseErr := chparser.NewParser(query).ParseStmts()
if parseErr != nil {
// Wrapped rather than formatted in, so that callers can recover the parser's *ParseError and read the position off it.
return errors.WrapInvalidInputf(parseErr, errors.CodeInvalidInput, "invalid ClickHouse SQL: %s", parseErr.Error())
return errors.WrapInvalidInputf(parseErr, CodeClickHouseSQLUnparseable, "invalid ClickHouse SQL: %s", parseErr.Error())
}
if len(stmts) != 1 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse SQL must contain exactly one statement, found %d statements", len(stmts))
return errors.NewInvalidInputf(CodeClickHouseSQLNotSingleStatement, "ClickHouse SQL must contain exactly one statement, found %d statements", len(stmts))
}
selectQuery, ok := stmts[0].(*chparser.SelectQuery)
if !ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "only SELECT statements are allowed in ClickHouse SQL queries")
return errors.NewInvalidInputf(CodeClickHouseSQLNotSelect, "only SELECT statements are allowed in ClickHouse SQL queries")
}
visitor := &chparser.DefaultASTVisitor{Visit: func(node chparser.Expr) error {
switch expr := node.(type) {
case *chparser.TableFunctionExpr:
// Source table functions remain usable in ClickHouse read-only mode.
return errors.NewInvalidInputf(errors.CodeInvalidInput, "ClickHouse table functions are not allowed in SQL queries: %s", chparser.Format(expr.Name))
return errors.NewInvalidInputf(CodeClickHouseSQLTableFunction, "ClickHouse table functions are not allowed in SQL queries: %s", chparser.Format(expr.Name))
case *chparser.TableIdentifier:
// Reading these is unaffected by ClickHouse read-only mode.
@@ -52,13 +62,13 @@ func ErrIfStatementIsNotValid(query string) (err error) {
}
if _, ok := internalDatabases[strings.ToLower(expr.Database.Name)]; ok {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse %s database is not allowed in SQL queries", expr.Database.Name)
return errors.NewInvalidInputf(CodeClickHouseSQLInternalDatabase, "the ClickHouse %s database is not allowed in SQL queries", expr.Database.Name)
}
case *chparser.SettingExpr:
// A query-level setting takes precedence over the context setting.
if strings.EqualFold(expr.Name.Name, "readonly") {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "the ClickHouse readonly setting cannot be overridden")
return errors.NewInvalidInputf(CodeClickHouseSQLReadonlyOverride, "the ClickHouse readonly setting cannot be overridden")
}
}

View File

@@ -3,6 +3,8 @@ package querybuilder
import (
"testing"
"github.com/SigNoz/signoz/pkg/errors"
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -30,10 +32,15 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
{"TrailingUnterminatedBlockComment", "SELECT count() FROM t /* unterminated"},
// The rule keys on the database, not on the table name.
{"TableNamedSystemInTelemetryDatabase", "SELECT * FROM signoz_logs.system"},
{"SignedLiteralAfterClosingParen", "SELECT (toUnixTimestamp(now()) - 3600)*1000000000"},
{"SignedLiteralAfterClosingParenSpaced", "SELECT (toUnixTimestamp(now()) - 3600)*1000000000"},
// order by interval
{"OrderByInterval", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS interval ORDER BY interval"},
{"OrderByIntervalAndDirection", "SELECT toStartOfInterval(timestamp, INTERVAL 1 MINUTE) AS `interval` ORDER BY `interval` ASC"},
// Unspaced, so rejected until the parser stopped lexing a signed literal after a
// closing bracket. The spaced form above no longer needs to be spaced.
// https://github.com/AfterShip/clickhouse-sql-parser/issues/286
{"SignedLiteralAfterClosingParenUnspaced", "SELECT now() AS ts, toFloat64(count()) AS value FROM ( SELECT attributes_string['TableName'] AS T, attributes_string['MissingId'] AS M, max(fromUnixTimestamp64Nano(timestamp)) AS last_seen, dateDiff('minute', min(fromUnixTimestamp64Nano(timestamp)), max(fromUnixTimestamp64Nano(timestamp))) AS age_min FROM signoz_logs.distributed_logs_v2 WHERE body='missing_map_record' AND timestamp >= (toUnixTimestamp(now())-3600)*1000000000 GROUP BY T, M ) WHERE age_min >= 20 AND last_seen >= now() - toIntervalMinute(8)"},
{"SignedLiteralAfterClosingParenMinimal", "SELECT (1)-1"},
{"TrimFunction", "SELECT trimBoth('/api/endpoint/', '/');"},
}
@@ -47,49 +54,52 @@ func TestErrIfStatementIsNotValid_Pass(t *testing.T) {
func TestErrIfStatementIsNotValid_Fail(t *testing.T) {
testCases := []struct {
name string
query string
name string
query string
expectedCode errors.Code
}{
// Not a single statement, or not a statement at all.
{"Empty", ""},
{"UnterminatedBlockCommentOnly", "/* x"},
{"Unparseable", "SELECT FROM WHERE"},
{"MultipleStatements", "SELECT 1; DROP TABLE signoz_logs.logs_v2"},
{"Empty", "", CodeClickHouseSQLNotSingleStatement},
{"UnterminatedBlockCommentOnly", "/* x", CodeClickHouseSQLUnparseable},
{"Unparseable", "SELECT FROM WHERE", CodeClickHouseSQLUnparseable},
{"MultipleStatements", "SELECT 1; DROP TABLE signoz_logs.logs_v2", CodeClickHouseSQLNotSingleStatement},
// Parses, but is not a SELECT.
{"Drop", "DROP TABLE signoz_logs.logs_v2"},
{"Insert", "INSERT INTO signoz_logs.logs_v2 SELECT * FROM signoz_logs.logs_v2"},
{"AlterDelete", "ALTER TABLE signoz_logs.logs_v2 DELETE WHERE 1 = 1"},
{"CreateTable", "CREATE TABLE evil (a Int) ENGINE = Memory"},
{"Grant", "GRANT ALL ON *.* TO admin"},
{"Set", "SET readonly = 0"},
{"Drop", "DROP TABLE signoz_logs.logs_v2", CodeClickHouseSQLNotSelect},
{"Insert", "INSERT INTO signoz_logs.logs_v2 SELECT * FROM signoz_logs.logs_v2", CodeClickHouseSQLNotSelect},
{"AlterDelete", "ALTER TABLE signoz_logs.logs_v2 DELETE WHERE 1 = 1", CodeClickHouseSQLNotSelect},
{"CreateTable", "CREATE TABLE evil (a Int) ENGINE = Memory", CodeClickHouseSQLNotSelect},
{"Grant", "GRANT ALL ON *.* TO admin", CodeClickHouseSQLNotSelect},
{"Set", "SET readonly = 0", CodeClickHouseSQLNotSelect},
// These the parser rejects outright rather than classifying.
{"ShowGrants", "SHOW GRANTS"},
{"IntoOutfile", "SELECT * FROM t INTO OUTFILE '/tmp/x.csv'"},
{"ShowGrants", "SHOW GRANTS", CodeClickHouseSQLUnparseable},
{"IntoOutfile", "SELECT * FROM t INTO OUTFILE '/tmp/x.csv'", CodeClickHouseSQLUnparseable},
// Table functions, which read through something other than a telemetry table.
{"UrlTableFunction", "SELECT * FROM url('http://attacker.example/x', CSV, 'a String')"},
{"FileTableFunction", "SELECT * FROM file('/etc/passwd', CSV, 'a String')"},
{"ExecutableTableFunction", "SELECT * FROM executable('script.sh', CSV, 'a String')"},
{"TableFunctionInJoin", "SELECT * FROM t1 JOIN url('http://x', CSV, 'a String') u ON 1 = 1"},
{"TableFunctionInCommonTableExpression", "WITH c AS (SELECT * FROM url('http://x', CSV, 'a String')) SELECT * FROM c"},
{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM file('/etc/passwd', CSV, 'a String'))"},
{"TableFunctionInUnion", "SELECT * FROM t UNION ALL SELECT * FROM url('http://x', CSV, 'a String')"},
{"UrlTableFunction", "SELECT * FROM url('http://attacker.example/x', CSV, 'a String')", CodeClickHouseSQLTableFunction},
{"FileTableFunction", "SELECT * FROM file('/etc/passwd', CSV, 'a String')", CodeClickHouseSQLTableFunction},
{"ExecutableTableFunction", "SELECT * FROM executable('script.sh', CSV, 'a String')", CodeClickHouseSQLTableFunction},
{"TableFunctionInJoin", "SELECT * FROM t1 JOIN url('http://x', CSV, 'a String') u ON 1 = 1", CodeClickHouseSQLTableFunction},
{"TableFunctionInCommonTableExpression", "WITH c AS (SELECT * FROM url('http://x', CSV, 'a String')) SELECT * FROM c", CodeClickHouseSQLTableFunction},
{"TableFunctionInWhereSubquery", "SELECT * FROM t WHERE a IN (SELECT * FROM file('/etc/passwd', CSV, 'a String'))", CodeClickHouseSQLTableFunction},
{"TableFunctionInUnion", "SELECT * FROM t UNION ALL SELECT * FROM url('http://x', CSV, 'a String')", CodeClickHouseSQLTableFunction},
// Internal databases, which hold grants and server metadata rather than telemetry.
{"SystemUsers", "SELECT * FROM system.users"},
{"SystemUppercase", "SELECT * FROM SYSTEM.USERS"},
{"SystemQuoted", "SELECT count() FROM `system`.`tables`"},
{"SystemInSubquery", "SELECT * FROM (SELECT name FROM system.parts)"},
{"SystemInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l JOIN system.users AS u ON 1 = 1"},
{"SystemInIntersect", "SELECT * FROM t INTERSECT SELECT * FROM system.users"},
{"InformationSchema", "SELECT * FROM information_schema.tables"},
{"SystemUsers", "SELECT * FROM system.users", CodeClickHouseSQLInternalDatabase},
{"SystemUppercase", "SELECT * FROM SYSTEM.USERS", CodeClickHouseSQLInternalDatabase},
{"SystemQuoted", "SELECT count() FROM `system`.`tables`", CodeClickHouseSQLInternalDatabase},
{"SystemInSubquery", "SELECT * FROM (SELECT name FROM system.parts)", CodeClickHouseSQLInternalDatabase},
{"SystemInJoin", "SELECT * FROM signoz_logs.distributed_logs_v2 AS l JOIN system.users AS u ON 1 = 1", CodeClickHouseSQLInternalDatabase},
{"SystemInIntersect", "SELECT * FROM t INTERSECT SELECT * FROM system.users", CodeClickHouseSQLInternalDatabase},
{"InformationSchema", "SELECT * FROM information_schema.tables", CodeClickHouseSQLInternalDatabase},
// A query-level setting takes precedence over the one the caller applies.
{"ReadonlySettingOverride", "SELECT * FROM t SETTINGS readonly = 0"},
{"ReadonlySettingOverrideAmongOthers", "SELECT * FROM t SETTINGS max_threads = 4, readonly = 0"},
{"ReadonlySettingOverride", "SELECT * FROM t SETTINGS readonly = 0", CodeClickHouseSQLReadonlyOverride},
{"ReadonlySettingOverrideAmongOthers", "SELECT * FROM t SETTINGS max_threads = 4, readonly = 0", CodeClickHouseSQLReadonlyOverride},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := ErrIfStatementIsNotValid(testCase.query)
assert.Error(t, err)
assert.True(t, errors.Asc(err, testCase.expectedCode), "expected code %s, got %v", testCase.expectedCode, err)
})
}
}
@@ -122,18 +132,6 @@ func TestErrIfStatementIsNotValid_ShouldPassButFails(t *testing.T) {
expectedStopsAfter: "trim(BOTH '",
fix: "SELECT trimBoth(replaceOne( JSONExtractString(body, 'Action'), 'health_status: ', '' ), ' ')",
},
{
name: "SignedLiteralAfterClosingParen",
query: "SELECT now() AS ts, toFloat64(count()) AS value FROM ( SELECT attributes_string['TableName'] AS T, attributes_string['MissingId'] AS M, max(fromUnixTimestamp64Nano(timestamp)) AS last_seen, dateDiff('minute', min(fromUnixTimestamp64Nano(timestamp)), max(fromUnixTimestamp64Nano(timestamp))) AS age_min FROM signoz_logs.distributed_logs_v2 WHERE body='missing_map_record' AND timestamp >= (toUnixTimestamp(now())-3600)*1000000000 GROUP BY T, M ) WHERE age_min >= 20 AND last_seen >= now() - toIntervalMinute(8)",
expectedStopsAfter: "(toUnixTimestamp(now())",
fix: "SELECT (toUnixTimestamp(now()) - 3600) * 1000000000",
},
{
name: "SignedLiteralAfterClosingParenMinimal",
query: "SELECT (1)-1",
expectedStopsAfter: "(1)",
fix: "SELECT (1) - 1",
},
}
for _, testCase := range testCases {

View File

@@ -44,6 +44,8 @@ var (
GCPServiceCloudSQLPostgres = ServiceID{valuer.NewString("cloudsql_postgres")}
GCPServiceMemorystoreRedis = ServiceID{valuer.NewString("memorystore_redis")}
GCPServiceComputeEngine = ServiceID{valuer.NewString("computeengine")}
GCPServiceGKE = ServiceID{valuer.NewString("gke")}
GCPServiceCloudStorage = ServiceID{valuer.NewString("cloudstorage")}
)
func (ServiceID) Enum() []any {
@@ -78,6 +80,8 @@ func (ServiceID) Enum() []any {
GCPServiceCloudSQLPostgres,
GCPServiceMemorystoreRedis,
GCPServiceComputeEngine,
GCPServiceGKE,
GCPServiceCloudStorage,
}
}
@@ -118,6 +122,8 @@ var SupportedServices = map[CloudProviderType][]ServiceID{
GCPServiceCloudSQLPostgres,
GCPServiceMemorystoreRedis,
GCPServiceComputeEngine,
GCPServiceGKE,
GCPServiceCloudStorage,
},
}