Compare commits

...

5 Commits

Author SHA1 Message Date
Vikrant Gupta
faaed20dbd fix(user): revoke sessions on password reset (#12531)
Some checks are pending
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
cacheci / tests (push) Waiting to run
Release Drafter / update_release_draft (push) Waiting to run
#### Description

- Resetting a password via a reset token left existing sessions valid.
Changing a password voluntarily already revoked them; the reset path now
does the same.

#### Issues closed by this PR

closes: SigNoz/platform-pod#2667
2026-08-13 13:08:30 +00:00
Vinicius Lourenço
b5851ce388 feat(infrastructure-monitoring-details): add chart description (#12430)
## Pull Request

---

### 📄 Summary
> Why does this change exist?  
> What problem does it solve, and why is this the right approach?

Instead of generic phrase, we created a single line summary for each
chart title

#### Screenshots / Screen Recordings (if applicable)
> Include screenshots or screen recordings that clearly show the
behavior before the change and the result after the change. This helps
reviewers quickly understand the impact and verify the update.

Before:

<img width="1755" height="1269" alt="image"
src="https://github.com/user-attachments/assets/5d68bdde-1dd8-4405-ba68-90c89e685d50"
/>

After:

<img width="1756" height="1269" alt="image"
src="https://github.com/user-attachments/assets/003e2740-e25c-4cd0-8b3c-4e28959e194a"
/>

#### Issues closed by this PR
> Reference issues using `Closes #issue-number` to enable automatic
closure on merge.

Closes https://github.com/SigNoz/pulse-pod/issues/196

---

###  Change Type
_Select all that apply_

- [x]  Feature
- [ ] 🐛 Bug fix
- [ ] ♻️ Refactor
- [ ] 🛠️ Infra / Tooling
- [ ] 🧪 Test-only

---

### 🧪 Testing Strategy
> How was this change validated?

- Tests added/updated: No
- Manual verification: Yes
- Edge cases covered: -

---

### ⚠️ Risk & Impact Assessment
> What could break? How do we recover?

- Blast radius: Infrastructure Monitoring - Details
- Potential regressions: None
- Rollback plan: Revert this commit

---

### 📝 Changelog
> Fill only if this affects users, APIs, UI, or documented behavior  
> Use **N/A** for internal or non-user-facing changes

| Field | Value |
|------|-------|
| Deployment Type | Cloud / OSS / Enterprise |
| Change Type | Feature |
| Description | We updated the documentation for each chart title to
include a single line summary to give you a brief explanation about each
chart. |

---

### 📋 Checklist
- [x] Tests added or explicitly not required
- [x] Manually tested
- [ ] Breaking changes documented
- [ ] Backward compatibility considered
2026-08-13 12:33:22 +00:00
Gaurav Tewari
35d1869314 feat: add e2e for llm o11y (#12235)
## Pull Request

---

### 📄 Summary

This is PR ads basic happy flow E2E for our LLM observability flow.
Since most of the cases are covered by the integration test itself, we
have added happy path here.


We have also added test for Test tab here. 


So we have added an integration test for a test endpoint here. 


#### Issues closed by this PR
Closes - https://github.com/SigNoz/engineering-pod/issues/5764
---

###  Change Type
_Select all that apply_

- [x] 🧪 Test-only — E2E + integration coverage for LLM Observability
- [x] 🛠️ Infra / Tooling — E2E helpers + e2e-scoped feature-flag
conftest

---

### 🧪 Testing Strategy

- **Tests added:**
- **E2E (Playwright):**
`tests/e2e/tests/llm-o11y/attribute-mapping.spec.ts`,
`tests/e2e/tests/llm-o11y/llm-pricing.spec.ts` +
`tests/e2e/helpers/{attribute-mapping,llm-pricing}.ts`. Both specs are
`test.describe.configure({ mode: 'serial' })` and tear down what they
create via the API.
- **Integration (Jest/RTL/MSW):** `TestTab/__tests__/TestTab.test.tsx` —
4 cases:

    | Case | Time |
    |---|---|
| runs the sample span through the mappers and renders the populated
result | 447 ms |
    | surfaces a backend error and renders no results | 122 ms |
| persists an edited span to local storage and restores it on remount |
657 ms |
    | resets to the sample span and clears the persisted input | 70 ms |

- **Manual verification:** full LLM Observability Jest suite is green
locally — **7 suites / 70 tests passed in 24.3 s** (`npx jest --verbose
src/container/LLMObservability`), including the 4 new Test-tab cases.
- **Edge cases covered:** Test-tab backend 500 → error surfaced and no
results rendered; span input persisted across remount and cleared on
reset; E2E fixture cleanup is failure-tolerant (`.catch()` on
list/delete so a broken run doesn't mask the real assertion failure).
- **Not yet captured:** a fully-green E2E run in CI — the e2e workflow
only fires on PRs carrying the `safe-to-e2e` label (see Notes).

---

### ⚠️ Risk & Impact Assessment

- **Blast radius:** none in product code. No `frontend/src` runtime
file, no Go package, and no shared pytest fixture is modified — the only
non-test-file change is the new `tests/e2e/conftest.py`, scoped to the
`e2e` package.
- **Potential regressions:** limited to CI surface.
`tests/e2e/conftest.py` overrides the package-scoped `signoz` fixture
with a distinct `cache_key`, so it brings up **one additional container
set** for the e2e package rather than mutating the shared one —
integration suites keep the stock feature set.
- **Rollback plan:** revert the PR. No schema/migration changes, no
runtime behaviour to unwind.

---

### 📝 Changelog

| Field | Value |
|------|-------|
| Deployment Type | N/A |
| Change Type | Maintenance |
| Description | N/A — test-only, no user-facing change. |

---

### 📋 Checklist
- [x] Tests added or explicitly not required
- [x] Manually tested (LLM Observability Jest suite green: 7 suites / 70
tests; E2E specs exercised against a local flag-enabled stack)
- [x] Breaking changes documented (none)
- [x] Backward compatibility considered

---

## Notes for Reviewers

---------

Co-authored-by: Gaurav Tewari <tewarig@users.noreply.github.com>
2026-08-13 12:31:03 +00:00
Nikhil Mantri
fe2200e887 feat(infra-monitoring): remove v1 infra-monitoring endpoints and any dangling code (#12538)
#### Description

- Remove all 34 v1 infra-monitoring routes:
`/api/v1/{hosts,processes,pods,nodes,namespaces,clusters,deployments,daemonsets,statefulsets,jobs,pvcs}/{list,attribute_keys,attribute_values}`
and `/api/v1/infra_onboarding/k8s/status`. The frontend is fully on
`/api/v2/infra_monitoring/*` (checks supersedes the onboarding-status
endpoint).
- Delete the now-dead implementation: `pkg/query-service/app/infra.go`,
the whole `pkg/query-service/app/inframetrics` package, and
`pkg/query-service/model/infra.go` (~8.2k lines).
- Drop `Reader` methods only used by the removed code:
`GetCountOfThings`, `GetActiveHostsFromMetricMetadata`,
`GetMetricsExistenceAndEarliestTime`.
- Delete the last dead frontend caller
`src/api/infraMonitoring/getHostLists.ts` (missed by #12415).

#### Issues closed by this PR

Part of SigNoz/pulse-pod#246

#### Additional Information

- `/api/v1/processes/list` is removed without a v2 equivalent — it was
never shipped in the UI.
- The per-entity `attribute_keys`/`attribute_values` endpoints are
superseded by the generic fields/autocomplete APIs, which the infra
pages already use.
- Verified locally: full build, and the `inframonitoring` integration
suite passes 580/580.
2026-08-13 10:40:14 +00:00
Vikrant Gupta
52a9a893c2 chore(user): remove deprecated user endpoints (#12530)
#### Description

- Removes the deprecated user endpoints that now have v2 replacements:
- `POST /api/v1/invite`, `GET /api/v1/user`, `GET
/api/v1/getResetPasswordToken/{id}`, `POST /api/v1/resetPassword`
- `POST /api/v2/users/{id}/roles` and `DELETE
/api/v2/users/{id}/roles/{roleId}`, superseded by `/api/v2/user_roles`
- `GET /api/v1/user/me` stays registered but returns 501 pointing at
`GET /api/v2/users/me`, following the v1 dashboard endpoints.
- Drops the handlers, module methods and types that only existed to
serve them (`DeprecatedUser`, the `user_invite` types, `PostableRole`).
- Moves the four remaining `*_cleanup` teardown tests off `DELETE
/api/v2/users/{id}/roles/{roleId}` onto `DELETE
/api/v2/user_roles/{id}`. Removal is keyed by the `user_role` entry id,
so they read it from `GET /api/v2/users/{id}` — that endpoint is not
deprecated and its other uses are untouched.
- Regenerates `docs/api/openapi.yml` and the frontend client. No
hand-written frontend code referenced the removed operations.

#### Issues closed by this PR

Closes: https://github.com/SigNoz/platform-pod/issues/2667

#### Additional Information

- `/api/v1/user/me` is a stub rather than a deletion because an
unregistered `/api/*` path falls through to the SPA catch-all and
answers 200 with `index.html`, which older mcp reads as a successful
response — a 501 fails loudly instead.
2026-08-13 10:37:28 +00:00
68 changed files with 681 additions and 9977 deletions

View File

@@ -8866,31 +8866,6 @@ components:
oldPassword:
type: string
type: object
TypesDeprecatedUser:
properties:
createdAt:
format: date-time
type: string
displayName:
type: string
email:
type: string
id:
type: string
isRoot:
type: boolean
orgId:
type: string
role:
type: string
status:
type: string
updatedAt:
format: date-time
type: string
required:
- id
type: object
TypesIdentifiable:
properties:
id:
@@ -8898,31 +8873,6 @@ components:
required:
- id
type: object
TypesInvite:
properties:
createdAt:
format: date-time
type: string
email:
type: string
id:
type: string
inviteLink:
type: string
name:
type: string
orgId:
type: string
role:
type: string
token:
type: string
updatedAt:
format: date-time
type: string
required:
- id
type: object
TypesOrganization:
properties:
alias:
@@ -8957,17 +8907,6 @@ components:
- orgId
- email
type: object
TypesPostableInvite:
properties:
email:
type: string
frontendBaseUrl:
type: string
name:
type: string
role:
type: string
type: object
TypesPostableResetPassword:
properties:
password:
@@ -8975,13 +8914,6 @@ components:
token:
type: string
type: object
TypesPostableRole:
properties:
name:
type: string
required:
- name
type: object
TypesPostableVerifyResetPasswordToken:
properties:
token:
@@ -11396,70 +11328,6 @@ paths:
summary: Get field values
tags:
- fields
/api/v1/getResetPasswordToken/{id}:
get:
deprecated: true
description: This endpoint returns the reset password token by id
operationId: GetResetPasswordTokenDeprecated
parameters:
- in: path
name: id
required: true
schema:
type: string
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/TypesResetPasswordToken'
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Get reset password token
tags:
- users
/api/v1/global/config:
get:
deprecated: false
@@ -11489,69 +11357,6 @@ paths:
summary: Get global config
tags:
- global
/api/v1/invite:
post:
deprecated: true
description: This endpoint creates an invite for a user
operationId: CreateInvite
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/TypesPostableInvite'
responses:
"201":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/TypesInvite'
status:
type: string
required:
- status
- data
type: object
description: Created
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Conflict
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Create invite
tags:
- users
/api/v1/llm_pricing_rules:
get:
deprecated: false
@@ -12203,40 +12008,6 @@ paths:
summary: Get query range result
tags:
- dashboard
/api/v1/resetPassword:
post:
deprecated: true
description: This endpoint resets the password by token
operationId: ResetPasswordDeprecated
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/TypesPostableResetPassword'
responses:
"204":
description: No Content
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Conflict
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
summary: Reset password
tags:
- users
/api/v1/roles:
get:
deprecated: false
@@ -14427,73 +14198,14 @@ paths:
summary: Get aggregations for a trace
tags:
- tracedetail
/api/v1/user:
get:
deprecated: true
description: This endpoint lists all users
operationId: ListUsersDeprecated
responses:
"200":
content:
application/json:
schema:
properties:
data:
items:
$ref: '#/components/schemas/TypesDeprecatedUser'
type: array
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: List users
tags:
- users
/api/v1/user/me:
get:
deprecated: true
description: This endpoint returns the user I belong to
description: This endpoint is deprecated and always fails. Use GET /api/v2/users/me
instead.
operationId: GetMyUserDeprecated
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/TypesDeprecatedUser'
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
@@ -14513,6 +14225,12 @@ paths:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
"501":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Implemented
security:
- tokenizer: []
summary: Get my user
@@ -23762,108 +23480,6 @@ paths:
summary: Get user roles
tags:
- users
post:
deprecated: true
description: This endpoint assigns the role to the user roles by user id
operationId: SetRoleByUserID
parameters:
- in: path
name: id
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/TypesPostableRole'
responses:
"200":
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Set user roles
tags:
- users
/api/v2/users/{id}/roles/{roleId}:
delete:
deprecated: true
description: This endpoint removes a role from the user by user id and role
id
operationId: RemoveUserRoleByUserIDAndRoleID
parameters:
- in: path
name: id
required: true
schema:
type: string
- in: path
name: roleId
required: true
schema:
type: string
responses:
"204":
description: No Content
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"404":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Not Found
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- ADMIN
- tokenizer:
- ADMIN
summary: Remove a role from user
tags:
- users
/api/v2/users/me:
get:
deprecated: false

View File

@@ -183,7 +183,6 @@ func (s *Server) createPublicServer(apiHandler *api.APIHandler, web web.Web) (*h
apiHandler.RegisterLogsRoutes(r, am)
apiHandler.RegisterIntegrationRoutes(r, am)
apiHandler.RegisterQueryRangeV3Routes(r, am)
apiHandler.RegisterInfraMetricsRoutes(r, am)
apiHandler.RegisterQueryRangeV4Routes(r, am)
apiHandler.RegisterWebSocketPaths(r, am)
apiHandler.RegisterMessagingQueuesRoutes(r, am)

View File

@@ -9934,47 +9934,6 @@ export interface TypesChangePasswordRequestDTO {
oldPassword?: string;
}
export interface TypesDeprecatedUserDTO {
/**
* @type string
* @format date-time
*/
createdAt?: string;
/**
* @type string
*/
displayName?: string;
/**
* @type string
*/
email?: string;
/**
* @type string
*/
id: string;
/**
* @type boolean
*/
isRoot?: boolean;
/**
* @type string
*/
orgId?: string;
/**
* @type string
*/
role?: string;
/**
* @type string
*/
status?: string;
/**
* @type string
* @format date-time
*/
updatedAt?: string;
}
export interface TypesIdentifiableDTO {
/**
* @type string
@@ -9982,47 +9941,6 @@ export interface TypesIdentifiableDTO {
id: string;
}
export interface TypesInviteDTO {
/**
* @type string
* @format date-time
*/
createdAt?: string;
/**
* @type string
*/
email?: string;
/**
* @type string
*/
id: string;
/**
* @type string
*/
inviteLink?: string;
/**
* @type string
*/
name?: string;
/**
* @type string
*/
orgId?: string;
/**
* @type string
*/
role?: string;
/**
* @type string
*/
token?: string;
/**
* @type string
* @format date-time
*/
updatedAt?: string;
}
export interface TypesOrganizationDTO {
/**
* @type string
@@ -10072,25 +9990,6 @@ export interface TypesPostableForgotPasswordDTO {
orgId: string;
}
export interface TypesPostableInviteDTO {
/**
* @type string
*/
email?: string;
/**
* @type string
*/
frontendBaseUrl?: string;
/**
* @type string
*/
name?: string;
/**
* @type string
*/
role?: string;
}
export interface TypesPostableResetPasswordDTO {
/**
* @type string
@@ -10102,13 +10001,6 @@ export interface TypesPostableResetPasswordDTO {
token?: string;
}
export interface TypesPostableRoleDTO {
/**
* @type string
*/
name: string;
}
export interface TypesPostableVerifyResetPasswordTokenDTO {
/**
* @type string
@@ -10751,17 +10643,6 @@ export type GetFieldsValues200 = {
status: string;
};
export type GetResetPasswordTokenDeprecatedPathParameters = {
id: string;
};
export type GetResetPasswordTokenDeprecated200 = {
data: TypesResetPasswordTokenDTO;
/**
* @type string
*/
status: string;
};
export type GetGlobalConfig200 = {
data: GlobaltypesConfigDTO;
/**
@@ -10770,14 +10651,6 @@ export type GetGlobalConfig200 = {
status: string;
};
export type CreateInvite201 = {
data: TypesInviteDTO;
/**
* @type string
*/
status: string;
};
export type ListLLMPricingRulesParams = {
/**
* @type integer
@@ -11190,25 +11063,6 @@ export type GetTraceAggregations200 = {
status: string;
};
export type ListUsersDeprecated200 = {
/**
* @type array
*/
data: TypesDeprecatedUserDTO[];
/**
* @type string
*/
status: string;
};
export type GetMyUserDeprecated200 = {
data: TypesDeprecatedUserDTO;
/**
* @type string
*/
status: string;
};
export type ListUserPreferences200 = {
/**
* @type array
@@ -12403,13 +12257,6 @@ export type GetRolesByUserID200 = {
status: string;
};
export type SetRoleByUserIDPathParameters = {
id: string;
};
export type RemoveUserRoleByUserIDAndRoleIDPathParameters = {
id: string;
roleId: string;
};
export type GetMyUser200 = {
data: AuthtypesUserWithRolesDTO;
/**

View File

@@ -20,7 +20,6 @@ import type {
import type {
AuthtypesPostableUserDTO,
AuthtypesPostableUserRoleDTO,
CreateInvite201,
CreateResetPasswordToken201,
CreateResetPasswordTokenPathParameters,
CreateUser201,
@@ -28,10 +27,7 @@ import type {
DeleteUserPathParameters,
DeleteUserRolePathParameters,
GetMyUser200,
GetMyUserDeprecated200,
GetResetPasswordToken200,
GetResetPasswordTokenDeprecated200,
GetResetPasswordTokenDeprecatedPathParameters,
GetResetPasswordTokenPathParameters,
GetRolesByUserID200,
GetRolesByUserIDPathParameters,
@@ -42,15 +38,10 @@ import type {
GetUsersByRoleID200,
GetUsersByRoleIDPathParameters,
ListUsers200,
ListUsersDeprecated200,
RemoveUserRoleByUserIDAndRoleIDPathParameters,
RenderErrorResponseDTO,
SetRoleByUserIDPathParameters,
TypesChangePasswordRequestDTO,
TypesPostableForgotPasswordDTO,
TypesPostableInviteDTO,
TypesPostableResetPasswordDTO,
TypesPostableRoleDTO,
TypesPostableVerifyResetPasswordTokenDTO,
TypesUpdatableUserDTO,
UpdateUserPathParameters,
@@ -60,379 +51,12 @@ import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
/**
* This endpoint returns the reset password token by id
* @deprecated
* @summary Get reset password token
*/
export const getResetPasswordTokenDeprecated = (
{ id }: GetResetPasswordTokenDeprecatedPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<GetResetPasswordTokenDeprecated200>({
url: `/api/v1/getResetPasswordToken/${id}`,
method: 'GET',
signal,
});
};
export const getGetResetPasswordTokenDeprecatedQueryKey = ({
id,
}: GetResetPasswordTokenDeprecatedPathParameters) => {
return [`/api/v1/getResetPasswordToken/${id}`] as const;
};
export const getGetResetPasswordTokenDeprecatedQueryOptions = <
TData = Awaited<ReturnType<typeof getResetPasswordTokenDeprecated>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ id }: GetResetPasswordTokenDeprecatedPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getResetPasswordTokenDeprecated>>,
TError,
TData
>;
},
) => {
const { query: queryOptions } = options ?? {};
const queryKey =
queryOptions?.queryKey ?? getGetResetPasswordTokenDeprecatedQueryKey({ id });
const queryFn: QueryFunction<
Awaited<ReturnType<typeof getResetPasswordTokenDeprecated>>
> = ({ signal }) => getResetPasswordTokenDeprecated({ id }, signal);
return {
queryKey,
queryFn,
enabled: !!id,
...queryOptions,
} as UseQueryOptions<
Awaited<ReturnType<typeof getResetPasswordTokenDeprecated>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type GetResetPasswordTokenDeprecatedQueryResult = NonNullable<
Awaited<ReturnType<typeof getResetPasswordTokenDeprecated>>
>;
export type GetResetPasswordTokenDeprecatedQueryError =
ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Get reset password token
*/
export function useGetResetPasswordTokenDeprecated<
TData = Awaited<ReturnType<typeof getResetPasswordTokenDeprecated>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(
{ id }: GetResetPasswordTokenDeprecatedPathParameters,
options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof getResetPasswordTokenDeprecated>>,
TError,
TData
>;
},
): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getGetResetPasswordTokenDeprecatedQueryOptions(
{ id },
options,
);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @deprecated
* @summary Get reset password token
*/
export const invalidateGetResetPasswordTokenDeprecated = async (
queryClient: QueryClient,
{ id }: GetResetPasswordTokenDeprecatedPathParameters,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getGetResetPasswordTokenDeprecatedQueryKey({ id }) },
options,
);
return queryClient;
};
/**
* This endpoint creates an invite for a user
* @deprecated
* @summary Create invite
*/
export const createInvite = (
typesPostableInviteDTO?: BodyType<TypesPostableInviteDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<CreateInvite201>({
url: `/api/v1/invite`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: typesPostableInviteDTO,
signal,
});
};
export const getCreateInviteMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof createInvite>>,
TError,
{ data?: BodyType<TypesPostableInviteDTO> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof createInvite>>,
TError,
{ data?: BodyType<TypesPostableInviteDTO> },
TContext
> => {
const mutationKey = ['createInvite'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof createInvite>>,
{ data?: BodyType<TypesPostableInviteDTO> }
> = (props) => {
const { data } = props ?? {};
return createInvite(data);
};
return { mutationFn, ...mutationOptions };
};
export type CreateInviteMutationResult = NonNullable<
Awaited<ReturnType<typeof createInvite>>
>;
export type CreateInviteMutationBody =
| BodyType<TypesPostableInviteDTO>
| undefined;
export type CreateInviteMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Create invite
*/
export const useCreateInvite = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof createInvite>>,
TError,
{ data?: BodyType<TypesPostableInviteDTO> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof createInvite>>,
TError,
{ data?: BodyType<TypesPostableInviteDTO> },
TContext
> => {
return useMutation(getCreateInviteMutationOptions(options));
};
/**
* This endpoint resets the password by token
* @deprecated
* @summary Reset password
*/
export const resetPasswordDeprecated = (
typesPostableResetPasswordDTO?: BodyType<TypesPostableResetPasswordDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v1/resetPassword`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: typesPostableResetPasswordDTO,
signal,
});
};
export const getResetPasswordDeprecatedMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof resetPasswordDeprecated>>,
TError,
{ data?: BodyType<TypesPostableResetPasswordDTO> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof resetPasswordDeprecated>>,
TError,
{ data?: BodyType<TypesPostableResetPasswordDTO> },
TContext
> => {
const mutationKey = ['resetPasswordDeprecated'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof resetPasswordDeprecated>>,
{ data?: BodyType<TypesPostableResetPasswordDTO> }
> = (props) => {
const { data } = props ?? {};
return resetPasswordDeprecated(data);
};
return { mutationFn, ...mutationOptions };
};
export type ResetPasswordDeprecatedMutationResult = NonNullable<
Awaited<ReturnType<typeof resetPasswordDeprecated>>
>;
export type ResetPasswordDeprecatedMutationBody =
| BodyType<TypesPostableResetPasswordDTO>
| undefined;
export type ResetPasswordDeprecatedMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Reset password
*/
export const useResetPasswordDeprecated = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof resetPasswordDeprecated>>,
TError,
{ data?: BodyType<TypesPostableResetPasswordDTO> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof resetPasswordDeprecated>>,
TError,
{ data?: BodyType<TypesPostableResetPasswordDTO> },
TContext
> => {
return useMutation(getResetPasswordDeprecatedMutationOptions(options));
};
/**
* This endpoint lists all users
* @deprecated
* @summary List users
*/
export const listUsersDeprecated = (signal?: AbortSignal) => {
return GeneratedAPIInstance<ListUsersDeprecated200>({
url: `/api/v1/user`,
method: 'GET',
signal,
});
};
export const getListUsersDeprecatedQueryKey = () => {
return [`/api/v1/user`] as const;
};
export const getListUsersDeprecatedQueryOptions = <
TData = Awaited<ReturnType<typeof listUsersDeprecated>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listUsersDeprecated>>,
TError,
TData
>;
}) => {
const { query: queryOptions } = options ?? {};
const queryKey = queryOptions?.queryKey ?? getListUsersDeprecatedQueryKey();
const queryFn: QueryFunction<
Awaited<ReturnType<typeof listUsersDeprecated>>
> = ({ signal }) => listUsersDeprecated(signal);
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
Awaited<ReturnType<typeof listUsersDeprecated>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type ListUsersDeprecatedQueryResult = NonNullable<
Awaited<ReturnType<typeof listUsersDeprecated>>
>;
export type ListUsersDeprecatedQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary List users
*/
export function useListUsersDeprecated<
TData = Awaited<ReturnType<typeof listUsersDeprecated>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<
Awaited<ReturnType<typeof listUsersDeprecated>>,
TError,
TData
>;
}): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getListUsersDeprecatedQueryOptions(options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @deprecated
* @summary List users
*/
export const invalidateListUsersDeprecated = async (
queryClient: QueryClient,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getListUsersDeprecatedQueryKey() },
options,
);
return queryClient;
};
/**
* This endpoint returns the user I belong to
* This endpoint is deprecated and always fails. Use GET /api/v2/users/me instead.
* @deprecated
* @summary Get my user
*/
export const getMyUserDeprecated = (signal?: AbortSignal) => {
return GeneratedAPIInstance<GetMyUserDeprecated200>({
return GeneratedAPIInstance<void>({
url: `/api/v1/user/me`,
method: 'GET',
signal,
@@ -1834,189 +1458,6 @@ export const invalidateGetRolesByUserID = async (
return queryClient;
};
/**
* This endpoint assigns the role to the user roles by user id
* @deprecated
* @summary Set user roles
*/
export const setRoleByUserID = (
{ id }: SetRoleByUserIDPathParameters,
typesPostableRoleDTO?: BodyType<TypesPostableRoleDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v2/users/${id}/roles`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: typesPostableRoleDTO,
signal,
});
};
export const getSetRoleByUserIDMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof setRoleByUserID>>,
TError,
{
pathParams: SetRoleByUserIDPathParameters;
data?: BodyType<TypesPostableRoleDTO>;
},
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof setRoleByUserID>>,
TError,
{
pathParams: SetRoleByUserIDPathParameters;
data?: BodyType<TypesPostableRoleDTO>;
},
TContext
> => {
const mutationKey = ['setRoleByUserID'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof setRoleByUserID>>,
{
pathParams: SetRoleByUserIDPathParameters;
data?: BodyType<TypesPostableRoleDTO>;
}
> = (props) => {
const { pathParams, data } = props ?? {};
return setRoleByUserID(pathParams, data);
};
return { mutationFn, ...mutationOptions };
};
export type SetRoleByUserIDMutationResult = NonNullable<
Awaited<ReturnType<typeof setRoleByUserID>>
>;
export type SetRoleByUserIDMutationBody =
| BodyType<TypesPostableRoleDTO>
| undefined;
export type SetRoleByUserIDMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Set user roles
*/
export const useSetRoleByUserID = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof setRoleByUserID>>,
TError,
{
pathParams: SetRoleByUserIDPathParameters;
data?: BodyType<TypesPostableRoleDTO>;
},
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof setRoleByUserID>>,
TError,
{
pathParams: SetRoleByUserIDPathParameters;
data?: BodyType<TypesPostableRoleDTO>;
},
TContext
> => {
return useMutation(getSetRoleByUserIDMutationOptions(options));
};
/**
* This endpoint removes a role from the user by user id and role id
* @deprecated
* @summary Remove a role from user
*/
export const removeUserRoleByUserIDAndRoleID = (
{ id, roleId }: RemoveUserRoleByUserIDAndRoleIDPathParameters,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<void>({
url: `/api/v2/users/${id}/roles/${roleId}`,
method: 'DELETE',
signal,
});
};
export const getRemoveUserRoleByUserIDAndRoleIDMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof removeUserRoleByUserIDAndRoleID>>,
TError,
{ pathParams: RemoveUserRoleByUserIDAndRoleIDPathParameters },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof removeUserRoleByUserIDAndRoleID>>,
TError,
{ pathParams: RemoveUserRoleByUserIDAndRoleIDPathParameters },
TContext
> => {
const mutationKey = ['removeUserRoleByUserIDAndRoleID'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof removeUserRoleByUserIDAndRoleID>>,
{ pathParams: RemoveUserRoleByUserIDAndRoleIDPathParameters }
> = (props) => {
const { pathParams } = props ?? {};
return removeUserRoleByUserIDAndRoleID(pathParams);
};
return { mutationFn, ...mutationOptions };
};
export type RemoveUserRoleByUserIDAndRoleIDMutationResult = NonNullable<
Awaited<ReturnType<typeof removeUserRoleByUserIDAndRoleID>>
>;
export type RemoveUserRoleByUserIDAndRoleIDMutationError =
ErrorType<RenderErrorResponseDTO>;
/**
* @deprecated
* @summary Remove a role from user
*/
export const useRemoveUserRoleByUserIDAndRoleID = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof removeUserRoleByUserIDAndRoleID>>,
TError,
{ pathParams: RemoveUserRoleByUserIDAndRoleIDPathParameters },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof removeUserRoleByUserIDAndRoleID>>,
TError,
{ pathParams: RemoveUserRoleByUserIDAndRoleIDPathParameters },
TContext
> => {
return useMutation(getRemoveUserRoleByUserIDAndRoleIDMutationOptions(options));
};
/**
* This endpoint returns the user I belong to
* @summary Get my user v2

View File

@@ -1,82 +0,0 @@
import axios from 'api';
import { ErrorResponseHandler } from 'api/ErrorResponseHandler';
import { AxiosError } from 'axios';
import { ErrorResponse, SuccessResponse } from 'types/api';
import { BaseAutocompleteData } from 'types/api/queryBuilder/queryAutocompleteResponse';
import { TagFilter } from 'types/api/queryBuilder/queryBuilderData';
export interface HostListPayload {
filters: TagFilter;
groupBy: BaseAutocompleteData[];
offset?: number;
limit?: number;
orderBy?: {
columnName: string;
order: 'asc' | 'desc';
} | null;
start?: number;
end?: number;
}
export interface TimeSeriesValue {
timestamp: number;
value: string;
}
export interface TimeSeries {
labels: Record<string, string>;
labelsArray: Array<Record<string, string>>;
values: TimeSeriesValue[];
}
export interface HostData {
hostName: string;
active: boolean;
os: string;
/** Present when the list API returns grouped rows or extra resource attributes. */
meta?: Record<string, string>;
cpu: number;
cpuTimeSeries: TimeSeries;
memory: number;
memoryTimeSeries: TimeSeries;
wait: number;
waitTimeSeries: TimeSeries;
load15: number;
load15TimeSeries: TimeSeries;
}
export interface HostListResponse {
status: string;
data: {
type: string;
records: HostData[];
groups: null;
total: number;
sentAnyHostMetricsData: boolean;
isSendingK8SAgentMetrics: boolean;
endTimeBeforeRetention: boolean;
};
}
export const getHostLists = async (
props: HostListPayload,
signal?: AbortSignal,
headers?: Record<string, string>,
): Promise<SuccessResponse<HostListResponse> | ErrorResponse> => {
try {
const response = await axios.post('/hosts/list', props, {
signal,
headers,
});
return {
statusCode: 200,
error: null,
message: 'Success',
payload: response.data,
params: props,
};
} catch (error) {
return ErrorResponseHandler(error as AxiosError);
}
};

View File

@@ -64,6 +64,8 @@ export interface K8sDetailsFilters {
export interface K8sDetailsWidgetInfo {
title: string;
yAxisUnit: string;
docPath?: string;
description?: string;
}
export type GetEntityQueryPayload<T> = (

View File

@@ -94,43 +94,59 @@ export const clusterWidgetInfo = [
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/clusters/#cpu-usage-allocatable',
description:
'Avg, max and min pod CPU usage across the cluster against total allocatable CPU.',
},
{
title: 'Memory Usage, allocatable',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/clusters/#memory-usage-allocatable',
description:
'Avg, max and min pod memory usage against allocatable memory; usage closing in on it risks evictions.',
},
{
title: 'Ready Nodes',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#ready-nodes',
description:
'Nodes currently reporting Ready; a line dropping out means that node stopped accepting pods.',
},
{
title: 'NotReady Nodes',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#notready-nodes',
description:
'Nodes whose kubelet reports unhealthy; their pods are evicted after the toleration window.',
},
{
title: 'Deployments available and desired',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/clusters/#deployments-available-and-desired',
description:
'Desired replicas versus pods available past minReadySeconds; a persistent gap means a stuck rollout.',
},
{
title: 'Statefulset pods',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#statefulset-pods',
description:
'Desired, current, ready and updated pod counts per StatefulSet; ready below desired means readiness failures.',
},
{
title: 'Daemonset nodes',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#daemonset-nodes',
description:
'Desired, current and ready node counts per DaemonSet; gaps mean node agents are missing.',
},
{
title: 'Jobs',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/clusters/#jobs',
description:
'Active, succeeded, failed and desired successful pod counts per Job; non-zero failed needs triage.',
},
];

View File

@@ -76,23 +76,31 @@ export const daemonSetWidgetInfo = [
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/daemonsets/#cpu-usage-request-limits',
description:
'Total CPU usage of the DaemonSet pods against their aggregate CPU requests and limits.',
},
{
title: 'Memory usage, request, limits',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/daemonsets/#memory-usage-request-limits',
description:
'Total memory usage of the DaemonSet pods against their aggregate memory requests and limits.',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/daemonsets/#network-io',
description:
'Transmit and receive throughput per interface across all pods of the DaemonSet.',
},
{
title: 'Network errors count',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/daemonsets/#network-errors-count',
description:
'Per-pod-interface network error counts by direction and interface, reported by the kubelet.',
},
];

View File

@@ -76,23 +76,31 @@ export const deploymentWidgetInfo = [
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/deployments/#cpu-usage-request-limits',
description:
'Total CPU usage of the Deployment pods against their aggregate CPU requests and limits.',
},
{
title: 'Memory usage, request, limits',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/deployments/#memory-usage-request-limits',
description:
'Total memory usage of the Deployment pods against their aggregate memory requests and limits.',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/deployments/#network-io',
description:
'Transmit and receive throughput per interface across all pods of the Deployment.',
},
{
title: 'Network error count',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/deployments/#network-error-count',
description:
'Per-pod-interface network error counts by direction and interface, reported by the kubelet.',
},
];

View File

@@ -24,6 +24,7 @@ import { MetricRangePayloadProps } from 'types/api/metrics/getQueryRange';
import { getMetricsExplorerUrl } from 'utils/explorerUtils';
import { getDrawerDurationMs } from 'container/InfraMonitoringK8sV2/Base/useDrawerLifecycleStore';
import { K8sDetailsWidgetInfo } from 'container/InfraMonitoringK8sV2/Base/types';
import { buildEntityMetricsChartConfig } from './configBuilder';
import ChartHeader from './ChartHeader';
@@ -41,11 +42,7 @@ import ChartTooltipFooter from './ChartTooltipFooter';
interface EntityMetricsProps<T> {
entity: T;
eventEntity: string;
entityWidgetInfo: {
title: string;
yAxisUnit: string;
docPath?: string;
}[];
entityWidgetInfo: K8sDetailsWidgetInfo[];
getEntityQueryPayload: (
node: T,
start: number,
@@ -219,6 +216,7 @@ function EntityMetrics<T>({
<ChartHeader
title={entityWidgetInfo[idx].title}
docPath={entityWidgetInfo[idx].docPath}
tooltip={entityWidgetInfo[idx].description}
metricsExplorerUrl={
queryPayloads[idx] && queryPayloads[idx].graphType !== PANEL_TYPES.TABLE
? getMetricsExplorerUrl({

View File

@@ -74,21 +74,27 @@ export const jobWidgetInfo = [
title: 'CPU usage',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#cpu-usage',
description: 'CPU consumption in cores summed across the pods of the Job.',
},
{
title: 'Memory Usage',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#memory-usage',
description: 'Memory consumption in bytes summed across the pods of the Job.',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#network-io',
description:
'Transmit and receive throughput per interface across all pods of the Job.',
},
{
title: 'Network errors count',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/jobs/#network-errors-count',
description:
'Per-pod-interface network error counts by direction and interface, reported by the kubelet.',
},
];

View File

@@ -113,53 +113,73 @@ export const namespaceWidgetInfo = [
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/namespaces/#cpu-usage-cores-1',
description:
'Avg, max and min pod CPU usage in the namespace against the sum of container CPU requests.',
},
{
title: 'Memory Usage (bytes)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/namespaces/#memory-usage-bytes',
description:
'Pod memory usage, working set and RSS in the namespace against the sum of container memory requests.',
},
{
title: 'Pods CPU (top 10)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#pods-cpu-top-10',
description:
'CPU consumption in cores for the ten highest-consuming pods in the namespace.',
},
{
title: 'Pods Memory (top 10)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/namespaces/#pods-memory-top-10',
description:
'Memory consumption in bytes for the ten highest-consuming pods in the namespace.',
},
{
title: 'Network rate',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#network-rate',
description:
'Transmit and receive throughput per interface across the pods of the namespace.',
},
{
title: 'Network errors',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#network-errors',
description:
'Per-pod-interface network error counts by direction and interface, reported by the kubelet.',
},
{
title: 'StatefulSets (pods)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#statefulsets',
description:
'Desired, current and updated pod counts per StatefulSet in the namespace, revealing stalled rollouts.',
},
{
title: 'ReplicaSets (pods)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#replicasets',
description:
'Desired versus available replicas per ReplicaSet in the namespace, revealing pods stuck pending.',
},
{
title: 'DaemonSets (nodes)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#daemonsets',
description:
'Desired, current, ready and misscheduled node counts per DaemonSet in the namespace.',
},
{
title: 'Deployments (pods)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/namespaces/#deployments',
description:
'Desired and available replicas with utilization percentage per Deployment in the namespace.',
},
];

View File

@@ -58,52 +58,71 @@ export const nodeWidgetInfo = [
title: 'CPU Usage (cores)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#cpu-usage-cores-1',
description:
'Avg, max and min node CPU usage against allocatable capacity and the CPU requests scheduled on the node.',
},
{
title: 'Memory Usage (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#memory-usage-bytes',
description:
'Node memory usage, working set and RSS against allocatable memory and scheduled memory requests.',
},
{
title: 'CPU Usage (%)',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#cpu-usage-',
description:
'Node CPU usage as a percentage of allocatable capacity and of the CPU requests scheduled on the node.',
},
{
title: 'Memory Usage (%)',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#memory-usage-',
description:
'Node memory usage as a percentage of allocatable memory and of the memory requests scheduled on the node.',
},
{
title: 'Pods by CPU (top 10)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#pods-by-cpu-top-10',
description:
'CPU consumption in cores for the ten highest-consuming pods on this node.',
},
{
title: 'Pods by Memory (top 10)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#pods-by-memory-top-10',
description:
'Memory consumption in bytes for the ten highest-consuming pods on this node.',
},
{
title: 'Network error count',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#network-error-count',
description:
'Per-interface network error counts by direction, from the kubelet error counters.',
},
{
title: 'Network IO rate',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#network-io-rate',
description:
'Transmit and receive throughput per network interface on the node.',
},
{
title: 'Filesystem usage (bytes)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/nodes/#filesystem-usage-bytes',
description:
'Capacity, available and used bytes for the primary filesystem of the node.',
},
{
title: 'Filesystem usage (%)',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/kubernetes/nodes/#filesystem-usage-',
description: 'Percentage of the nodefs filesystem currently consumed.',
},
];

View File

@@ -68,73 +68,97 @@ export const podWidgetInfo = [
title: 'CPU Usage (cores)',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/pods/#cpu-usage-cores-1',
description:
'Avg, max and min CPU consumption of the pod in cores, showing how volatile it is.',
},
{
title: 'CPU Request, Limit Utilization',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#cpu-request-limit-utilization',
description:
'Pod CPU usage as a fraction of its total container CPU requests and limits, to spot throttling.',
},
{
title: 'Memory Usage (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/pods/#memory-usage-bytes',
description:
'Avg, max and min memory consumption of the pod, including reclaimable page cache.',
},
{
title: 'Memory Request, Limit Utilization',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-request-limit-utilization',
description:
'Pod memory usage as a fraction of its total container memory requests and limits.',
},
{
title: 'Memory by State',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/pods/#memory-by-state',
description:
'RSS, working set and cache memory of the pod, separating heap growth from file cache.',
},
{
title: 'Memory Major Page Faults',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-major-page-faults',
description:
'Major page fault rate of the pod; sustained values mean the working set is paging to disk.',
},
{
title: 'CPU Usage by Container (cores)',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#cpu-usage-by-container-cores',
description:
'CPU consumption in cores per container, showing which container drives the pod CPU.',
},
{
title: 'CPU Request, Limit Utilization by Container',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#cpu-request-limit-utilization-by-container',
description:
'Each container CPU usage as a fraction of its own request and limit, to find the throttled one.',
},
{
title: 'Memory Usage by Container (bytes)',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-usage-by-container-bytes',
description: 'Usage, working set and RSS memory per container of the pod.',
},
{
title: 'Memory Request, Limit Utilization by Container',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/pods/#memory-request-limit-utilization-by-container',
description:
'Each container memory usage as a fraction of its own request and limit; near 100% risks an OOMKill.',
},
{
title: 'Network rate',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/pods/#network-rate',
description: 'Pod network throughput in bytes/s by direction and interface.',
},
{
title: 'Network errors',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/kubernetes/pods/#network-errors',
description:
'Network error counts on the pod interfaces; sustained non-zero values point to CNI or MTU issues.',
},
{
title: 'File system (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/pods/#file-system-bytes',
description:
'Capacity, available and used bytes of the local filesystem of the pod.',
},
];

View File

@@ -77,35 +77,47 @@ export const statefulSetWidgetInfo = [
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#cpu-usage-request-limits',
description:
'Total CPU usage of the StatefulSet pods against their aggregate CPU requests and limits.',
},
{
title: 'CPU request, limit util (%)',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#cpu-request-limit-utilization-',
description:
'Average CPU usage of the StatefulSet as a percentage of its requests and of its limits.',
},
{
title: 'Memory usage, request, limits',
yAxisUnit: 'bytes',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#memory-usage-request-limits',
description:
'Total memory usage of the StatefulSet pods against their aggregate memory requests and limits.',
},
{
title: 'Memory request, limit util (%)',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#memory-request-limit-utilization-',
description:
'Average memory usage as a percentage of requests and limits; above 100% of request means it exceeds its reservation.',
},
{
title: 'Network IO',
yAxisUnit: 'binBps',
docPath: '/infrastructure-monitoring/kubernetes/statefulsets/#network-io',
description:
'Transmit and receive throughput per interface across all pods of the StatefulSet.',
},
{
title: 'Network errors count',
yAxisUnit: '',
docPath:
'/infrastructure-monitoring/kubernetes/statefulsets/#network-errors-count',
description:
'Per-pod-interface network error counts by direction and interface, reported by the kubelet.',
},
];

View File

@@ -70,28 +70,38 @@ export const volumeWidgetInfo = [
title: 'Volume available',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-available-1',
description:
'Free bytes on the volume over time; a steady decline forecasts when the volume fills up.',
},
{
title: 'Volume capacity',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-capacity-1',
description:
'Total provisioned capacity of the volume in bytes, which steps up only when the PVC is resized.',
},
{
title: 'Volume inodes used',
yAxisUnit: 'short',
docPath:
'/infrastructure-monitoring/kubernetes/volumes/#volume-inodes-used-1',
description:
'Inodes consumed on the volume filesystem; a rising line means many small files are being created.',
},
{
title: 'Volume inodes',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/kubernetes/volumes/#volume-inodes-1',
description:
'Total inodes available on the volume filesystem, the reference for spotting inode exhaustion.',
},
{
title: 'Volume inodes free',
yAxisUnit: 'short',
docPath:
'/infrastructure-monitoring/kubernetes/volumes/#volume-inodes-free-1',
description:
'Unallocated inodes on the volume; near zero, file creation fails with ENOSPC even with free bytes.',
},
];

View File

@@ -800,26 +800,36 @@ export const podUtilizationByPodWidgetInfo = [
title: 'CPU Limit Utilization By Pod Name',
yAxisUnit: 'percentunit',
docPath: '#cpu-limit-utilization-by-pod-name',
description:
'CPU usage against the CPU limit for each pod; near 100% means the kernel is throttling that pod.',
},
{
title: 'CPU Request Utilization By Pod Name',
yAxisUnit: 'percentunit',
docPath: '#cpu-request-utilization-by-pod-name',
description:
'CPU usage against the CPU request for each pod; above 100% means the pod uses more than it reserved.',
},
{
title: 'Memory Limit Utilization By Pod Name',
yAxisUnit: 'percentunit',
docPath: '#memory-limit-utilization-by-pod-name',
description:
'Memory usage against the memory limit for each pod; near 100% means that pod is close to an OOMKill.',
},
{
title: 'Memory Request Utilization By Pod Name',
yAxisUnit: 'percentunit',
docPath: '#memory-request-utilization-by-pod-name',
description:
'Memory usage against the memory request for each pod; above 100% means the pod exceeds its reservation.',
},
{
title: 'FileSystem Usage Percentage By Pod Name',
yAxisUnit: 'percentunit',
docPath: '#filesystem-usage-percentage-by-pod-name',
description:
'Local and ephemeral filesystem fill level as a percentage of capacity for each pod.',
},
];

View File

@@ -0,0 +1,171 @@
import { rest, server } from 'mocks-server/server';
import get from 'api/browser/localstorage/get';
import remove from 'api/browser/localstorage/remove';
import set from 'api/browser/localstorage/set';
import { LOCALSTORAGE } from 'constants/localStorage';
import { useAuthZ } from 'lib/authz/hooks/useAuthZ/useAuthZ';
import { mockUseAuthZGrantAll } from 'lib/authz/utils/authz-test-utils';
import { render, screen, userEvent, waitFor } from 'tests/test-utils';
jest.mock('@monaco-editor/react', () => ({
__esModule: true,
default: ({
value,
onChange,
}: {
value: string;
onChange: (next?: string) => void;
}): JSX.Element => (
<textarea
aria-label="json-editor"
data-testid="monaco"
value={value}
onChange={(e): void => onChange(e.target.value)}
/>
),
}));
jest.mock('lib/authz/hooks/useAuthZ/useAuthZ');
const mockedUseAuthZ = useAuthZ as jest.MockedFunction<typeof useAuthZ>;
import LLMObservabilityAttributeMapping from '../../LLMObservabilityAttributeMapping';
import { SAMPLE_SPAN_JSON } from '../spanInputStorage';
import {
GROUPS_ENDPOINT,
makeGroupsResponse,
makeTestResponse,
mockGroups,
TEST_ENDPOINT,
} from '../../__tests__/fixtures';
const RESULT_SPAN = {
attributes: {
'my_company.llm.input': 'What is quantum computing?',
'llm.input_messages': 'What is quantum computing?',
'gen_ai.request.model': 'gpt-4',
'gen_ai.usage.total_tokens': 1250,
'gen_ai.content.completion': 'Quantum computing leverages...',
'gen_ai.content.prompt': 'What is quantum computing?',
},
resource: {
'service.name': 'llm-gateway',
'deployment.environment': 'production',
},
};
const EDITED_SPAN_JSON = `{
"attributes": {
"gen_ai.request.model": "claude-opus-5"
},
"resource": {
"service.name": "my-edited-gateway"
}
}`;
const SPAN_INPUT_KEY = LOCALSTORAGE.LLM_ATTRIBUTE_MAPPING_TEST_SPAN;
describe('TestTab — sample-span flow', () => {
beforeEach(() => {
window.history.pushState(null, '', '/');
remove(SPAN_INPUT_KEY);
server.use(
rest.get(GROUPS_ENDPOINT, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(makeGroupsResponse(mockGroups))),
),
);
mockedUseAuthZ.mockImplementation(mockUseAuthZGrantAll);
});
it('runs the sample span through the mappers and renders the populated result', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
server.use(
rest.post(TEST_ENDPOINT, (_req, res, ctx) =>
res(ctx.status(200), ctx.json(makeTestResponse([RESULT_SPAN]))),
),
);
render(<LLMObservabilityAttributeMapping />);
await user.click(screen.getByRole('tab', { name: 'Test' }));
const runBtn = await screen.findByTestId('run-test-button');
expect(screen.getByTestId('test-results-placeholder')).toBeInTheDocument();
await user.click(runBtn);
await expect(
screen.findByTestId('test-results'),
).resolves.toBeInTheDocument();
expect(screen.getByTestId('test-result-0')).toBeInTheDocument();
expect(screen.getByTestId('test-result-0-attributes')).toHaveTextContent(
'gen_ai.content.prompt',
);
expect(screen.getByText('populated')).toBeInTheDocument();
expect(screen.queryByTestId('test-error')).not.toBeInTheDocument();
});
it('surfaces a backend error and renders no results', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
server.use(
rest.post(TEST_ENDPOINT, (_req, res, ctx) =>
res(
ctx.status(500),
ctx.json({ error: { message: 'span mapper test failed' } }),
),
),
);
render(<LLMObservabilityAttributeMapping />);
await user.click(screen.getByRole('tab', { name: 'Test' }));
await user.click(await screen.findByTestId('run-test-button'));
await expect(screen.findByTestId('test-error')).resolves.toHaveTextContent(
'span mapper test failed',
);
expect(screen.queryByTestId('test-results')).not.toBeInTheDocument();
});
it('persists an edited span to local storage and restores it on remount', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
const { unmount } = render(<LLMObservabilityAttributeMapping />);
await user.click(screen.getByRole('tab', { name: 'Test' }));
await screen.findByTestId('run-test-button');
const editor = screen.getByTestId('monaco');
expect(editor).toHaveValue(SAMPLE_SPAN_JSON);
await user.clear(editor);
await user.paste(EDITED_SPAN_JSON);
await waitFor(() => expect(get(SPAN_INPUT_KEY)).toBe(EDITED_SPAN_JSON), {
timeout: 2000,
});
unmount();
render(<LLMObservabilityAttributeMapping />);
await user.click(screen.getByRole('tab', { name: 'Test' }));
await screen.findByTestId('run-test-button');
expect(screen.getByTestId('monaco')).toHaveValue(EDITED_SPAN_JSON);
});
it('resets to the sample span and clears the persisted input', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
set(SPAN_INPUT_KEY, EDITED_SPAN_JSON);
render(<LLMObservabilityAttributeMapping />);
await user.click(screen.getByRole('tab', { name: 'Test' }));
const resetBtn = await screen.findByTestId('reset-template-button');
expect(screen.getByTestId('monaco')).toHaveValue(EDITED_SPAN_JSON);
expect(resetBtn).toBeEnabled();
await user.click(resetBtn);
expect(screen.getByTestId('monaco')).toHaveValue(SAMPLE_SPAN_JSON);
expect(get(SPAN_INPUT_KEY)).toBeFalsy();
expect(resetBtn).toBeDisabled();
});
});

View File

@@ -3,6 +3,7 @@ import {
SpantypesSpanMapperDTO as Mapper,
SpantypesSpanMapperGroupDTO as MapperGroup,
SpantypesSpanMapperOperationDTO as MapperOperation,
SpantypesSpanMapperTestSpanDTO as TestSpan,
} from 'api/generated/services/sigNoz.schemas';
// Endpoint globs used by MSW handlers. The generated client hits relative
@@ -12,6 +13,7 @@ export const GROUPS_ENDPOINT = '*/api/v1/span_mapper_groups';
export function mappersEndpoint(groupId: string): string {
return `*/api/v1/span_mapper_groups/${groupId}/span_mappers`;
}
export const TEST_ENDPOINT = '*/api/v1/span_mapper_groups/test';
export function makeGroup(overrides: Partial<MapperGroup> = {}): MapperGroup {
return {
@@ -71,6 +73,13 @@ export function makeMappersResponse(mappers: Mapper[]): {
return { status: 'ok', data: { items: mappers } };
}
export function makeTestResponse(spans: TestSpan[]): {
status: string;
data: { spans: TestSpan[] };
} {
return { status: 'ok', data: { spans } };
}
export const mockGroups: MapperGroup[] = [
makeGroup({
id: 'group-1',

View File

@@ -2780,68 +2780,94 @@ export const hostWidgetInfo = [
title: 'CPU Usage',
yAxisUnit: 'percentunit',
docPath: '/infrastructure-monitoring/host-monitoring/#cpu-usage-1',
description:
'CPU time share per state (user, system, wait, steal, idle); sustained wait points to disk I/O blocking.',
},
{
title: 'Memory Usage',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/host-monitoring/#memory-usage-1',
description:
'Physical memory bytes per state (used, cached, buffers, free); a climbing used line suggests a leak.',
},
{
title: 'Disk Usage (%) by mountpoint',
yAxisUnit: 'percentunit',
docPath:
'/infrastructure-monitoring/host-monitoring/#disk-usage--by-mountpoint',
description:
'Used space as a percentage of capacity for each mountpoint, one line per mountpoint.',
},
{
title: 'System Load Average',
yAxisUnit: '',
docPath: '/infrastructure-monitoring/host-monitoring/#system-load-average',
description:
'The 1m, 5m and 15m load averages together; 1m above 15m means load is building.',
},
{
title: 'Network usage (bytes)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/host-monitoring/#network-usage-bytes',
description:
'Throughput in bytes/s per interface and direction, to spot NICs nearing rated bandwidth.',
},
{
title: 'Network usage (packet/s)',
yAxisUnit: 'pps',
docPath: '/infrastructure-monitoring/host-monitoring/#network-usage-packetss',
description:
'Packets per second per interface and direction; a NIC can saturate on packet rate before bytes.',
},
{
title: 'Network errors',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#network-errors',
description:
'Rate of interface-level network errors per interface and direction; any sustained value needs attention.',
},
{
title: 'Network drops',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#network-drops',
description:
'Rate of dropped packets per interface and direction, usually buffer overflow rather than link errors.',
},
{
title: 'Network connections',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#network-connections',
description:
'Active connection counts per protocol and state (ESTABLISHED, TIME_WAIT, SYN_RECV) to spot leaks and churn.',
},
{
title: 'System disk io (bytes transferred)',
yAxisUnit: 'bytes',
docPath: '/infrastructure-monitoring/host-monitoring/#system-disk-io-bytes',
description:
'Disk throughput in bytes/s per device and direction, tracking heavy file I/O or database flushes.',
},
{
title: 'System disk operations/s',
yAxisUnit: 'short',
docPath:
'/infrastructure-monitoring/host-monitoring/#system-disk-operationss',
description:
'Rate of completed read and write operations per device; pair with disk io bytes to size each operation.',
},
{
title: 'Queue size',
yAxisUnit: 'short',
docPath: '/infrastructure-monitoring/host-monitoring/#queue-size',
description:
'Maximum disk request-queue depth per device; sustained high depth means the storage layer is saturated.',
},
{
title: 'System disk operation time/s',
yAxisUnit: 's',
docPath:
'/infrastructure-monitoring/host-monitoring/#system-disk-operation-times',
description:
'Rate of cumulative disk-busy time per device and direction; values near 1s/s mean the device is saturated.',
},
];

View File

@@ -10,40 +10,6 @@ import (
)
func (provider *provider) addUserRoutes(router *mux.Router) error {
if err := router.Handle("/api/v1/invite", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.CreateInvite), handler.OpenAPIDef{
ID: "CreateInvite",
Tags: []string{"users"},
Summary: "Create invite",
Description: "This endpoint creates an invite for a user",
Request: new(types.PostableInvite),
RequestContentType: "application/json",
Response: new(types.Invite),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusConflict},
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
})).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v1/user", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.ListUsersDeprecated), handler.OpenAPIDef{
ID: "ListUsersDeprecated",
Tags: []string{"users"},
Summary: "List users",
Description: "This endpoint lists all users",
Request: nil,
RequestContentType: "",
Response: make([]*types.DeprecatedUser, 0),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{},
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/users", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.ListUsers), handler.OpenAPIDef{
ID: "ListUsers",
Tags: []string{"users"},
@@ -65,13 +31,13 @@ func (provider *provider) addUserRoutes(router *mux.Router) error {
ID: "GetMyUserDeprecated",
Tags: []string{"users"},
Summary: "Get my user",
Description: "This endpoint returns the user I belong to",
Description: "This endpoint is deprecated and always fails. Use GET /api/v2/users/me instead.",
Request: nil,
RequestContentType: "",
Response: new(types.DeprecatedUser),
ResponseContentType: "application/json",
Response: nil,
ResponseContentType: "",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{},
ErrorStatusCodes: []int{http.StatusNotImplemented},
Deprecated: true,
SecuritySchemes: []handler.OpenAPISecurityScheme{{Name: authtypes.IdentNProviderTokenizer.StringValue()}},
})).Methods(http.MethodGet).GetError(); err != nil {
@@ -180,23 +146,6 @@ func (provider *provider) addUserRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/getResetPasswordToken/{id}", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.GetResetPasswordTokenDeprecated), handler.OpenAPIDef{
ID: "GetResetPasswordTokenDeprecated",
Tags: []string{"users"},
Summary: "Get reset password token",
Description: "This endpoint returns the reset password token by id",
Request: nil,
RequestContentType: "",
Response: new(types.ResetPasswordToken),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusNotFound},
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
})).Methods(http.MethodGet).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/users/{id}/reset_password_tokens", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.GetResetPasswordToken), handler.OpenAPIDef{
ID: "GetResetPasswordToken",
Tags: []string{"users"},
@@ -248,23 +197,6 @@ func (provider *provider) addUserRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v1/resetPassword", handler.New(provider.authzMiddleware.OpenAccess(provider.userHandler.ResetPassword), handler.OpenAPIDef{
ID: "ResetPasswordDeprecated",
Tags: []string{"users"},
Summary: "Reset password",
Description: "This endpoint resets the password by token",
Request: new(types.PostableResetPassword),
RequestContentType: "application/json",
Response: nil,
ResponseContentType: "",
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusConflict},
Deprecated: true,
SecuritySchemes: []handler.OpenAPISecurityScheme{},
})).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/users/me/factor_password", handler.New(provider.authzMiddleware.OpenAccess(provider.userHandler.ChangePassword), handler.OpenAPIDef{
ID: "UpdateMyPassword",
Tags: []string{"users"},
@@ -333,40 +265,6 @@ func (provider *provider) addUserRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v2/users/{id}/roles", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.SetRoleByUserID), handler.OpenAPIDef{
ID: "SetRoleByUserID",
Tags: []string{"users"},
Summary: "Set user roles",
Description: "This endpoint assigns the role to the user roles by user id",
Request: new(types.PostableRole),
RequestContentType: "application/json",
Response: nil,
ResponseContentType: "",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusNotFound},
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
})).Methods(http.MethodPost).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/users/{id}/roles/{roleId}", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.RemoveUserRoleByRoleID), handler.OpenAPIDef{
ID: "RemoveUserRoleByUserIDAndRoleID",
Tags: []string{"users"},
Summary: "Remove a role from user",
Description: "This endpoint removes a role from the user by user id and role id",
Request: nil,
RequestContentType: "",
Response: nil,
ResponseContentType: "",
SuccessStatusCode: http.StatusNoContent,
ErrorStatusCodes: []int{http.StatusNotFound},
Deprecated: true,
SecuritySchemes: newSecuritySchemes(types.RoleAdmin),
})).Methods(http.MethodDelete).GetError(); err != nil {
return err
}
if err := router.Handle("/api/v2/roles/{id}/users", handler.New(provider.authzMiddleware.AdminAccess(provider.userHandler.GetUsersByRoleID), handler.OpenAPIDef{
ID: "GetUsersByRoleID",
Tags: []string{"users"},

View File

@@ -37,8 +37,7 @@ var daemonSetsTableMetricNamesList = []string{
"k8s.daemonset.misscheduled_nodes",
}
// Carried forward from v1 daemonSetAttrsToEnrich
// (pkg/query-service/app/inframetrics/daemonsets.go:29-33).
// Carried forward from v1 daemonSetAttrsToEnrich (removed).
var daemonSetAttrKeysForMetadata = []string{
"k8s.daemonset.name",
"k8s.namespace.name",

View File

@@ -35,8 +35,7 @@ var deploymentsTableMetricNamesList = []string{
"k8s.deployment.available",
}
// Carried forward from v1 deploymentAttrsToEnrich
// (pkg/query-service/app/inframetrics/deployments.go:29-33).
// Carried forward from v1 deploymentAttrsToEnrich (removed).
var deploymentAttrKeysForMetadata = []string{
"k8s.deployment.name",
"k8s.namespace.name",

View File

@@ -37,8 +37,7 @@ var jobsTableMetricNamesList = []string{
"k8s.job.desired_successful_pods",
}
// Carried forward from v1 jobAttrsToEnrich
// (pkg/query-service/app/inframetrics/jobs.go:31-35).
// Carried forward from v1 jobAttrsToEnrich (removed).
var jobAttrKeysForMetadata = []string{
"k8s.job.name",
"k8s.namespace.name",

View File

@@ -35,8 +35,7 @@ var statefulSetsTableMetricNamesList = []string{
"k8s.statefulset.current_pods",
}
// Carried forward from v1 statefulSetAttrsToEnrich
// (pkg/query-service/app/inframetrics/statefulsets.go:29-33).
// Carried forward from v1 statefulSetAttrsToEnrich (removed).
var statefulSetAttrKeysForMetadata = []string{
"k8s.statefulset.name",
"k8s.namespace.name",

View File

@@ -26,8 +26,7 @@ var volumesTableMetricNamesList = []string{
"k8s.volume.inodes.used",
}
// Carried forward from v1 volumeAttrsToEnrich
// (pkg/query-service/app/inframetrics/pvcs.go:23-31).
// Carried forward from v1 volumeAttrsToEnrich (removed).
var volumeAttrKeysForMetadata = []string{
"k8s.persistentvolumeclaim.name",
"k8s.pod.uid",

View File

@@ -37,53 +37,6 @@ func (module *getter) GetRootUserByOrgID(ctx context.Context, orgID valuer.UUID)
return rootUser, userRoles, nil
}
func (module *getter) ListDeprecatedUsersByOrgID(ctx context.Context, orgID valuer.UUID) ([]*types.DeprecatedUser, error) {
users, err := module.store.ListUsersByOrgID(ctx, orgID)
if err != nil {
return nil, err
}
// filter root users if feature flag `hide_root_users` is true
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)
hideRootUsers := module.flagger.BooleanOrEmpty(ctx, flagger.FeatureHideRootUser, evalCtx)
if hideRootUsers {
users = slices.DeleteFunc(users, func(user *types.User) bool { return user.IsRoot })
}
userIDs := make([]valuer.UUID, len(users))
for idx, user := range users {
userIDs[idx] = user.ID
}
userRoles, err := module.userRoleStore.ListUserRolesByOrgIDAndUserIDs(ctx, orgID, userIDs)
if err != nil {
return nil, err
}
// Build userID → role name mapping directly from the joined Role
userIDToRoleNames := make(map[valuer.UUID][]string)
for _, ur := range userRoles {
if ur.Role != nil {
userIDToRoleNames[ur.UserID] = append(userIDToRoleNames[ur.UserID], ur.Role.Name)
}
}
deprecatedUsers := make([]*types.DeprecatedUser, 0, len(users))
for _, user := range users {
roleNames := userIDToRoleNames[user.ID]
if len(roleNames) == 0 {
return nil, errors.Newf(errors.TypeInternal, authtypes.ErrCodeUserRolesNotFound, "no user roles entries found for user: %s", user.ID.String())
}
role := authtypes.SigNozManagedRoleToExistingLegacyRole[roleNames[0]]
deprecatedUsers = append(deprecatedUsers, types.NewDeprecatedUserFromUserAndRole(user, role))
}
return deprecatedUsers, nil
}
func (module *getter) ListUsersByOrgID(ctx context.Context, orgID valuer.UUID) ([]*types.User, error) {
users, err := module.store.ListUsersByOrgID(ctx, orgID)
if err != nil {
@@ -101,58 +54,10 @@ func (module *getter) ListUsersByOrgID(ctx context.Context, orgID valuer.UUID) (
return users, nil
}
func (module *getter) GetDeprecatedUserByOrgIDAndID(ctx context.Context, orgID valuer.UUID, id valuer.UUID) (*types.DeprecatedUser, error) {
user, err := module.store.GetByOrgIDAndID(ctx, orgID, id)
if err != nil {
return nil, err
}
userRoles, err := module.GetRolesByUserID(ctx, id)
if err != nil {
return nil, err
}
if len(userRoles) == 0 {
return nil, errors.New(errors.TypeInternal, authtypes.ErrCodeUserRolesNotFound, "no user roles entries found")
}
if userRoles[0].Role == nil {
return nil, errors.New(errors.TypeInternal, authtypes.ErrCodeRoleNotFound, "role not found for user role entry")
}
role := authtypes.SigNozManagedRoleToExistingLegacyRole[userRoles[0].Role.Name]
return types.NewDeprecatedUserFromUserAndRole(user, role), nil
}
func (module *getter) GetUserByOrgIDAndID(ctx context.Context, orgID valuer.UUID, userID valuer.UUID) (*types.User, error) {
return module.store.GetByOrgIDAndID(ctx, orgID, userID)
}
func (module *getter) Get(ctx context.Context, id valuer.UUID) (*types.DeprecatedUser, error) {
user, err := module.store.GetUser(ctx, id)
if err != nil {
return nil, err
}
userRoles, err := module.GetRolesByUserID(ctx, id)
if err != nil {
return nil, err
}
if len(userRoles) == 0 {
return nil, errors.New(errors.TypeInternal, authtypes.ErrCodeUserRolesNotFound, "no user roles entries found")
}
if userRoles[0].Role == nil {
return nil, errors.New(errors.TypeInternal, authtypes.ErrCodeRoleNotFound, "role not found for user role entry")
}
role := authtypes.SigNozManagedRoleToExistingLegacyRole[userRoles[0].Role.Name]
return types.NewDeprecatedUserFromUserAndRole(user, role), nil
}
func (module *getter) ListUsersByEmailAndOrgIDs(ctx context.Context, email valuer.Email, orgIDs []valuer.UUID) ([]*types.User, error) {
return module.store.ListUsersByEmailAndOrgIDs(ctx, email, orgIDs)
}

View File

@@ -61,33 +61,6 @@ func (handler *handler) CreateUser(rw http.ResponseWriter, r *http.Request) {
render.Success(rw, http.StatusCreated, types.Identifiable{ID: user.ID})
}
func (handler *handler) CreateInvite(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
var req types.PostableInvite
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
render.Error(rw, err)
return
}
invites, err := handler.setter.CreateBulkInvite(ctx, valuer.MustNewUUID(claims.OrgID), valuer.MustNewUUID(claims.IdentityID()), valuer.MustNewEmail(claims.Email), &types.PostableBulkInviteRequest{
Invites: []types.PostableInvite{req},
})
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusCreated, invites[0])
}
func (handler *handler) GetUser(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
@@ -121,22 +94,8 @@ func (handler *handler) GetUser(w http.ResponseWriter, r *http.Request) {
}
func (handler *handler) GetMyUserDeprecated(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(w, err)
return
}
user, err := handler.getter.GetDeprecatedUserByOrgIDAndID(ctx, valuer.MustNewUUID(claims.OrgID), valuer.MustNewUUID(claims.UserID))
if err != nil {
render.Error(w, err)
return
}
render.Success(w, http.StatusOK, user)
render.Error(w, errors.New(errors.TypeUnsupported, types.ErrCodeUserDeprecated,
"the v1 user API is deprecated; instead, get your user with GET /api/v2/users/me"))
}
func (handler *handler) GetMyUser(w http.ResponseWriter, r *http.Request) {
@@ -194,25 +153,6 @@ func (handler *handler) UpdateMyUser(w http.ResponseWriter, r *http.Request) {
render.Success(w, http.StatusNoContent, nil)
}
func (handler *handler) ListUsersDeprecated(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(w, err)
return
}
users, err := handler.getter.ListDeprecatedUsersByOrgID(ctx, valuer.MustNewUUID(claims.OrgID))
if err != nil {
render.Error(w, err)
return
}
render.Success(w, http.StatusOK, users)
}
func (handler *handler) ListUsers(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
@@ -284,33 +224,6 @@ func (handler *handler) DeleteUser(w http.ResponseWriter, r *http.Request) {
render.Success(w, http.StatusNoContent, nil)
}
func (handler *handler) GetResetPasswordTokenDeprecated(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
id := mux.Vars(r)["id"]
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(w, err)
return
}
user, err := handler.getter.GetDeprecatedUserByOrgIDAndID(ctx, valuer.MustNewUUID(claims.OrgID), valuer.MustNewUUID(id))
if err != nil {
render.Error(w, err)
return
}
token, err := handler.setter.GetOrCreateResetPasswordToken(ctx, user.ID)
if err != nil {
render.Error(w, err)
return
}
render.Success(w, http.StatusOK, token)
}
func (handler *handler) GetResetPasswordToken(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
@@ -481,68 +394,6 @@ func (handler *handler) GetRolesByUserID(w http.ResponseWriter, r *http.Request)
render.Success(w, http.StatusOK, roles)
}
func (handler *handler) SetRoleByUserID(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
userID := mux.Vars(r)["id"]
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(w, err)
return
}
if userID == claims.UserID {
render.Error(w, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "users cannot call this api on self"))
return
}
postableRole := new(types.PostableRole)
if err := json.NewDecoder(r.Body).Decode(postableRole); err != nil {
render.Error(w, err)
return
}
if postableRole.Name == "" {
render.Error(w, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "role name is required"))
return
}
if _, err := handler.setter.AddUserRole(ctx, valuer.MustNewUUID(claims.OrgID), valuer.MustNewUUID(userID), postableRole.Name); err != nil {
render.Error(w, err)
return
}
render.Success(w, http.StatusOK, nil)
}
func (handler *handler) RemoveUserRoleByRoleID(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
userID := mux.Vars(r)["id"]
roleID := mux.Vars(r)["roleId"]
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(w, err)
return
}
if userID == claims.UserID {
render.Error(w, errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "users cannot call this api on self"))
return
}
if err := handler.setter.RemoveUserRole(ctx, valuer.MustNewUUID(claims.OrgID), valuer.MustNewUUID(userID), valuer.MustNewUUID(roleID)); err != nil {
render.Error(w, err)
return
}
render.Success(w, http.StatusNoContent, nil)
}
func (handler *handler) GetUsersByRoleID(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()

View File

@@ -55,119 +55,6 @@ func NewSetter(store types.UserStore, tokenizer tokenizer.Tokenizer, emailing em
}
}
// CreateBulk implements invite.Module.
func (module *setter) CreateBulkInvite(ctx context.Context, orgID valuer.UUID, identityID valuer.UUID, identityEmail valuer.Email, bulkInvites *types.PostableBulkInviteRequest) ([]*types.Invite, error) {
// validate all emails to be invited
emails := make([]string, len(bulkInvites.Invites))
for idx, invite := range bulkInvites.Invites {
emails[idx] = invite.Email.StringValue()
}
users, err := module.store.GetUsersByEmailsOrgIDAndStatuses(ctx, orgID, emails, []string{types.UserStatusActive.StringValue(), types.UserStatusPendingInvite.StringValue()})
if err != nil {
return nil, err
}
if len(users) > 0 {
if err := users[0].ErrIfRoot(); err != nil {
return nil, errors.WithAdditionalf(err, "Cannot send invite to root user")
}
if users[0].Status == types.UserStatusPendingInvite {
return nil, errors.Newf(errors.TypeAlreadyExists, errors.CodeAlreadyExists, "An invite already exists for this email: %s", users[0].Email.StringValue())
}
return nil, errors.Newf(errors.TypeAlreadyExists, errors.CodeAlreadyExists, "User already exists with this email: %s", users[0].Email.StringValue())
}
type userWithResetToken struct {
User *types.User
ResetPasswordToken *types.ResetPasswordToken
Role types.Role
}
newUsersWithResetToken := make([]*userWithResetToken, len(bulkInvites.Invites))
if err := module.store.RunInTx(ctx, func(ctx context.Context) error {
for idx, invite := range bulkInvites.Invites {
// create a new user with pending invite status
newUser, err := types.NewUser(invite.Name, invite.Email, orgID, types.UserStatusPendingInvite)
if err != nil {
return err
}
// store the user and password in db
err = module.createUserWithoutGrant(ctx, newUser, root.WithRoleNames([]string{authtypes.MustGetSigNozManagedRoleFromExistingRole(invite.Role)}))
if err != nil {
return err
}
// generate reset password token
resetPasswordToken, err := module.GetOrCreateResetPasswordToken(ctx, newUser.ID)
if err != nil {
module.settings.Logger().ErrorContext(ctx, "failed to create reset password token for invited user", errors.Attr(err))
return err
}
newUsersWithResetToken[idx] = &userWithResetToken{
User: newUser,
ResetPasswordToken: resetPasswordToken,
Role: invite.Role,
}
}
return nil
}); err != nil {
return nil, err
}
invites := make([]*types.Invite, len(bulkInvites.Invites))
// send password reset emails to all the invited users
for idx, userWithToken := range newUsersWithResetToken {
module.analytics.TrackUser(ctx, orgID.String(), identityID.String(), "Invite Sent", map[string]any{
"invitee_email": userWithToken.User.Email,
"invitee_role": userWithToken.Role,
})
invite := &types.Invite{
Identifiable: types.Identifiable{
ID: userWithToken.User.ID,
},
Name: userWithToken.User.DisplayName,
Email: userWithToken.User.Email,
Token: userWithToken.ResetPasswordToken.Token,
Role: userWithToken.Role,
OrgID: userWithToken.User.OrgID,
TimeAuditable: types.TimeAuditable{
CreatedAt: userWithToken.User.CreatedAt,
UpdatedAt: userWithToken.User.UpdatedAt,
},
}
invites[idx] = invite
frontendBaseUrl := bulkInvites.Invites[idx].FrontendBaseUrl
if frontendBaseUrl == "" {
module.settings.Logger().InfoContext(ctx, "frontend base url is not provided, skipping email", slog.Any("invitee_email", userWithToken.User.Email))
continue
}
resetLink := userWithToken.ResetPasswordToken.FactorPasswordResetLink(frontendBaseUrl)
tokenLifetime := module.config.Password.Invite.MaxTokenLifetime
humanizedTokenLifetime := strings.TrimSpace(humanize.RelTime(time.Now(), time.Now().Add(tokenLifetime), "", ""))
if err := module.emailing.SendHTML(ctx, userWithToken.User.Email.String(), "You're Invited to Join SigNoz", emailtypes.TemplateNameInvitationEmail, map[string]any{
"inviter_email": identityEmail.StringValue(),
"link": resetLink,
"Expiry": humanizedTokenLifetime,
}); err != nil {
module.settings.Logger().ErrorContext(ctx, "failed to send invite email", errors.Attr(err))
}
}
return invites, nil
}
func (module *setter) CreateUser(ctx context.Context, user *types.User, opts ...root.CreateUserOption) error {
createUserOpts := root.NewCreateUserOptions(opts...)
@@ -557,7 +444,7 @@ func (module *setter) UpdatePasswordByResetPasswordToken(ctx context.Context, to
module.analytics.TrackUser(ctx, user.OrgID.String(), user.ID.String(), "User Activated", traitsOrProperties)
}
return module.store.RunInTx(ctx, func(ctx context.Context) error {
if err := module.store.RunInTx(ctx, func(ctx context.Context) error {
if isPendingInviteUser {
err := module.store.UpdateUser(ctx, user.OrgID, user)
if err != nil {
@@ -574,7 +461,11 @@ func (module *setter) UpdatePasswordByResetPasswordToken(ctx context.Context, to
}
return nil
})
}); err != nil {
return err
}
return module.tokenizer.DeleteTokensByUserID(ctx, user.ID)
}
func (module *setter) UpdatePassword(ctx context.Context, userID valuer.UUID, oldpasswd string, passwd string) error {

View File

@@ -7,7 +7,6 @@ import (
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
)
type userRoleStore struct {
@@ -19,26 +18,6 @@ func NewUserRoleStore(sqlstore sqlstore.SQLStore, settings factory.ProviderSetti
return &userRoleStore{sqlstore: sqlstore, settings: settings}
}
func (store *userRoleStore) ListUserRolesByOrgIDAndUserIDs(ctx context.Context, orgID valuer.UUID, userIDs []valuer.UUID) ([]*authtypes.UserRole, error) {
userRoles := make([]*authtypes.UserRole, 0)
err := store.sqlstore.
BunDBCtx(ctx).
NewSelect().
Model(&userRoles).
Join("JOIN users").
JoinOn("users.id = user_role.user_id").
Where("users.org_id = ?", orgID).
Where("users.id IN (?)", bun.In(userIDs)).
Relation("Role").
Scan(ctx)
if err != nil {
return nil, err
}
return userRoles, nil
}
func (store *userRoleStore) GetUserRoleByOrgIDAndID(ctx context.Context, orgID valuer.UUID, id valuer.UUID) (*authtypes.UserRole, error) {
userRole := new(authtypes.UserRole)

View File

@@ -40,9 +40,6 @@ type Setter interface {
UpdateAnyUser(ctx context.Context, orgID valuer.UUID, user *types.User) error
DeleteUser(ctx context.Context, orgID valuer.UUID, id string, deletedBy string) error
// invite
CreateBulkInvite(ctx context.Context, orgID valuer.UUID, identityID valuer.UUID, identityEmail valuer.Email, bulkInvites *types.PostableBulkInviteRequest) ([]*types.Invite, error)
// Creates a pending invite user with the roles given via opts and emails them the invite link.
CreatePendingInviteUser(ctx context.Context, identityID valuer.UUID, identityEmail valuer.Email, frontendBaseURL string, user *types.User, opts ...CreateUserOption) (*types.User, error)
@@ -60,16 +57,11 @@ type Getter interface {
GetRootUserByOrgID(context.Context, valuer.UUID) (*types.User, []*authtypes.UserRole, error)
// Get gets the users based on the given org id
ListDeprecatedUsersByOrgID(context.Context, valuer.UUID) ([]*types.DeprecatedUser, error)
ListUsersByOrgID(ctx context.Context, orgID valuer.UUID) ([]*types.User, error)
// Get deprecated user object by orgID and id.
GetDeprecatedUserByOrgIDAndID(context.Context, valuer.UUID, valuer.UUID) (*types.DeprecatedUser, error)
// Get user by orgID and id.
GetUserByOrgIDAndID(ctx context.Context, orgID valuer.UUID, userID valuer.UUID) (*types.User, error)
// Get user by id.
Get(context.Context, valuer.UUID) (*types.DeprecatedUser, error)
// List users by email and org ids.
ListUsersByEmailAndOrgIDs(context.Context, valuer.Email, []valuer.UUID) ([]*types.User, error)
@@ -105,11 +97,7 @@ type Getter interface {
}
type Handler interface {
// invite
CreateInvite(http.ResponseWriter, *http.Request)
// users
ListUsersDeprecated(http.ResponseWriter, *http.Request)
ListUsers(http.ResponseWriter, *http.Request)
CreateUser(http.ResponseWriter, *http.Request)
UpdateUser(http.ResponseWriter, *http.Request)
@@ -119,8 +107,6 @@ type Handler interface {
GetMyUser(http.ResponseWriter, *http.Request)
UpdateMyUser(http.ResponseWriter, *http.Request)
GetRolesByUserID(http.ResponseWriter, *http.Request)
SetRoleByUserID(http.ResponseWriter, *http.Request)
RemoveUserRoleByRoleID(http.ResponseWriter, *http.Request)
GetUsersByRoleID(http.ResponseWriter, *http.Request)
// user roles
@@ -129,7 +115,6 @@ type Handler interface {
DeleteUserRole(http.ResponseWriter, *http.Request)
// Reset Password
GetResetPasswordTokenDeprecated(http.ResponseWriter, *http.Request)
GetResetPasswordToken(http.ResponseWriter, *http.Request)
CreateResetPasswordToken(http.ResponseWriter, *http.Request)
VerifyResetPasswordToken(http.ResponseWriter, *http.Request)

View File

@@ -3334,60 +3334,6 @@ func (r *ClickHouseReader) GetMetricMetadata(ctx context.Context, orgID valuer.U
}, nil
}
// GetCountOfThings returns the count of things in the query
// This is a generic function that can be used to check if any data exists for a given query
func (r *ClickHouseReader) GetCountOfThings(ctx context.Context, query string) (uint64, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "clickhouse-reader",
instrumentationtypes.CodeFunctionName: "GetCountOfThings",
})
var count uint64
err := r.db.QueryRow(ctx, query).Scan(&count)
if err != nil {
return 0, err
}
return count, nil
}
func (r *ClickHouseReader) GetActiveHostsFromMetricMetadata(ctx context.Context, metricNames []string, hostNameAttr string, sinceUnixMilli int64) (map[string]bool, error) {
activeHosts := map[string]bool{}
query := fmt.Sprintf(
`SELECT DISTINCT attr_string_value
FROM %s.%s
WHERE metric_name IN @metricNames
AND attr_name = @attrName
AND last_reported_unix_milli >= @sinceUnixMilli`,
signozMetricDBName,
constants.SIGNOZ_METADATA_TABLENAME,
)
rows, err := r.db.Query(ctx, query,
clickhouse.Named("metricNames", metricNames),
clickhouse.Named("attrName", hostNameAttr),
clickhouse.Named("sinceUnixMilli", sinceUnixMilli),
)
if err != nil {
return nil, errorsV2.WrapInternalf(err, errorsV2.CodeInternal, "error querying active hosts")
}
defer rows.Close()
for rows.Next() {
var hostName string
if err := rows.Scan(&hostName); err != nil {
return nil, errorsV2.WrapInternalf(err, errorsV2.CodeInternal, "error scanning active host row")
}
if hostName != "" {
activeHosts[hostName] = true
}
}
if err := rows.Err(); err != nil {
return nil, errorsV2.WrapInternalf(err, errorsV2.CodeInternal, "error iterating active host rows")
}
return activeHosts, nil
}
func (r *ClickHouseReader) GetLatestReceivedMetric(
ctx context.Context, orgID valuer.UUID, metricNames []string, labelValues map[string]string,
) (*model.MetricStatus, *model.ApiError) {
@@ -4200,33 +4146,6 @@ func (r *ClickHouseReader) GetListResultV3(ctx context.Context, query string) ([
}
// GetHostMetricsExistenceAndEarliestTime returns (count, minFirstReportedUnixMilli, error) for the given host metric names
// from distributed_metadata. When count is 0, minFirstReportedUnixMilli is 0.
func (r *ClickHouseReader) GetMetricsExistenceAndEarliestTime(ctx context.Context, metricNames []string) (uint64, uint64, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.TelemetrySignal: telemetrytypes.SignalMetrics.StringValue(),
instrumentationtypes.CodeNamespace: "clickhouse-reader",
instrumentationtypes.CodeFunctionName: "GetMetricsExistenceAndEarliestTime",
})
if len(metricNames) == 0 {
return 0, 0, nil
}
query := fmt.Sprintf(
`SELECT count(*) AS cnt, min(first_reported_unix_milli) AS min_first_reported
FROM %s.%s
WHERE metric_name IN @metric_names`,
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_METADATA_TABLENAME)
var count, minFirstReported uint64
err := r.db.QueryRow(ctx, query, clickhouse.Named("metric_names", metricNames)).Scan(&count, &minFirstReported)
if err != nil {
r.logger.Error("error getting host metrics existence and earliest time", errorsV2.Attr(err))
return 0, 0, err
}
return count, minFirstReported, nil
}
func getPersonalisedError(err error) error {
if err == nil {
return nil

View File

@@ -44,7 +44,6 @@ import (
"github.com/SigNoz/signoz/pkg/contextlinks"
traceFunnelsModule "github.com/SigNoz/signoz/pkg/modules/tracefunnel"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
"github.com/SigNoz/signoz/pkg/query-service/app/inframetrics"
queues2 "github.com/SigNoz/signoz/pkg/query-service/app/integrations/messagingQueues/queues"
"github.com/SigNoz/signoz/pkg/query-service/app/logs"
logsv3 "github.com/SigNoz/signoz/pkg/query-service/app/logs/v3"
@@ -120,20 +119,6 @@ type APIHandler struct {
// Websocket connection upgrader
Upgrader *websocket.Upgrader
hostsRepo *inframetrics.HostsRepo
processesRepo *inframetrics.ProcessesRepo
podsRepo *inframetrics.PodsRepo
nodesRepo *inframetrics.NodesRepo
namespacesRepo *inframetrics.NamespacesRepo
clustersRepo *inframetrics.ClustersRepo
// workloads
deploymentsRepo *inframetrics.DeploymentsRepo
daemonsetsRepo *inframetrics.DaemonSetsRepo
statefulsetsRepo *inframetrics.StatefulSetsRepo
jobsRepo *inframetrics.JobsRepo
pvcsRepo *inframetrics.PvcsRepo
LicensingAPI licensing.API
QueryParserAPI *queryparser.API
@@ -180,17 +165,6 @@ func NewAPIHandler(opts APIHandlerOpts, config signoz.Config) (*APIHandler, erro
querier := querier.NewQuerier(querierOpts)
querierv2 := querierV2.NewQuerier(querierOptsV2)
hostsRepo := inframetrics.NewHostsRepo(opts.Reader, querierv2)
processesRepo := inframetrics.NewProcessesRepo(opts.Reader, querierv2)
podsRepo := inframetrics.NewPodsRepo(opts.Reader, querierv2)
nodesRepo := inframetrics.NewNodesRepo(opts.Reader, querierv2)
namespacesRepo := inframetrics.NewNamespacesRepo(opts.Reader, querierv2)
clustersRepo := inframetrics.NewClustersRepo(opts.Reader, querierv2)
deploymentsRepo := inframetrics.NewDeploymentsRepo(opts.Reader, querierv2)
daemonsetsRepo := inframetrics.NewDaemonSetsRepo(opts.Reader, querierv2)
statefulsetsRepo := inframetrics.NewStatefulSetsRepo(opts.Reader, querierv2)
jobsRepo := inframetrics.NewJobsRepo(opts.Reader, querierv2)
pvcsRepo := inframetrics.NewPvcsRepo(opts.Reader, querierv2)
//quickFilterModule := quickfilter.NewAPI(opts.QuickFilterModule)
aH := &APIHandler{
@@ -202,17 +176,6 @@ func NewAPIHandler(opts APIHandlerOpts, config signoz.Config) (*APIHandler, erro
LogsParsingPipelineController: opts.LogsParsingPipelineController,
querier: querier,
querierV2: querierv2,
hostsRepo: hostsRepo,
processesRepo: processesRepo,
podsRepo: podsRepo,
nodesRepo: nodesRepo,
namespacesRepo: namespacesRepo,
clustersRepo: clustersRepo,
deploymentsRepo: deploymentsRepo,
daemonsetsRepo: daemonsetsRepo,
statefulsetsRepo: statefulsetsRepo,
jobsRepo: jobsRepo,
pvcsRepo: pvcsRepo,
LicensingAPI: opts.LicensingAPI,
Signoz: opts.Signoz,
QueryParserAPI: opts.QueryParserAPI,
@@ -404,66 +367,6 @@ func (aH *APIHandler) RegisterQueryRangeV3Routes(router *mux.Router, am *middlew
subRouter.HandleFunc("/logs/livetail", am.ViewAccess(aH.Signoz.Handlers.QuerierHandler.QueryRawStream)).Methods(http.MethodGet)
}
func (aH *APIHandler) RegisterInfraMetricsRoutes(router *mux.Router, am *middleware.AuthZ) {
hostsSubRouter := router.PathPrefix("/api/v1/hosts").Subrouter()
hostsSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getHostAttributeKeys)).Methods(http.MethodGet)
hostsSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getHostAttributeValues)).Methods(http.MethodGet)
hostsSubRouter.HandleFunc("/list", am.ViewAccess(aH.getHostList)).Methods(http.MethodPost)
processesSubRouter := router.PathPrefix("/api/v1/processes").Subrouter()
processesSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getProcessAttributeKeys)).Methods(http.MethodGet)
processesSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getProcessAttributeValues)).Methods(http.MethodGet)
processesSubRouter.HandleFunc("/list", am.ViewAccess(aH.getProcessList)).Methods(http.MethodPost)
podsSubRouter := router.PathPrefix("/api/v1/pods").Subrouter()
podsSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getPodAttributeKeys)).Methods(http.MethodGet)
podsSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getPodAttributeValues)).Methods(http.MethodGet)
podsSubRouter.HandleFunc("/list", am.ViewAccess(aH.getPodList)).Methods(http.MethodPost)
pvcsSubRouter := router.PathPrefix("/api/v1/pvcs").Subrouter()
pvcsSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getPvcAttributeKeys)).Methods(http.MethodGet)
pvcsSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getPvcAttributeValues)).Methods(http.MethodGet)
pvcsSubRouter.HandleFunc("/list", am.ViewAccess(aH.getPvcList)).Methods(http.MethodPost)
nodesSubRouter := router.PathPrefix("/api/v1/nodes").Subrouter()
nodesSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getNodeAttributeKeys)).Methods(http.MethodGet)
nodesSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getNodeAttributeValues)).Methods(http.MethodGet)
nodesSubRouter.HandleFunc("/list", am.ViewAccess(aH.getNodeList)).Methods(http.MethodPost)
namespacesSubRouter := router.PathPrefix("/api/v1/namespaces").Subrouter()
namespacesSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getNamespaceAttributeKeys)).Methods(http.MethodGet)
namespacesSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getNamespaceAttributeValues)).Methods(http.MethodGet)
namespacesSubRouter.HandleFunc("/list", am.ViewAccess(aH.getNamespaceList)).Methods(http.MethodPost)
clustersSubRouter := router.PathPrefix("/api/v1/clusters").Subrouter()
clustersSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getClusterAttributeKeys)).Methods(http.MethodGet)
clustersSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getClusterAttributeValues)).Methods(http.MethodGet)
clustersSubRouter.HandleFunc("/list", am.ViewAccess(aH.getClusterList)).Methods(http.MethodPost)
deploymentsSubRouter := router.PathPrefix("/api/v1/deployments").Subrouter()
deploymentsSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getDeploymentAttributeKeys)).Methods(http.MethodGet)
deploymentsSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getDeploymentAttributeValues)).Methods(http.MethodGet)
deploymentsSubRouter.HandleFunc("/list", am.ViewAccess(aH.getDeploymentList)).Methods(http.MethodPost)
daemonsetsSubRouter := router.PathPrefix("/api/v1/daemonsets").Subrouter()
daemonsetsSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getDaemonSetAttributeKeys)).Methods(http.MethodGet)
daemonsetsSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getDaemonSetAttributeValues)).Methods(http.MethodGet)
daemonsetsSubRouter.HandleFunc("/list", am.ViewAccess(aH.getDaemonSetList)).Methods(http.MethodPost)
statefulsetsSubRouter := router.PathPrefix("/api/v1/statefulsets").Subrouter()
statefulsetsSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getStatefulSetAttributeKeys)).Methods(http.MethodGet)
statefulsetsSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getStatefulSetAttributeValues)).Methods(http.MethodGet)
statefulsetsSubRouter.HandleFunc("/list", am.ViewAccess(aH.getStatefulSetList)).Methods(http.MethodPost)
jobsSubRouter := router.PathPrefix("/api/v1/jobs").Subrouter()
jobsSubRouter.HandleFunc("/attribute_keys", am.ViewAccess(aH.getJobAttributeKeys)).Methods(http.MethodGet)
jobsSubRouter.HandleFunc("/attribute_values", am.ViewAccess(aH.getJobAttributeValues)).Methods(http.MethodGet)
jobsSubRouter.HandleFunc("/list", am.ViewAccess(aH.getJobList)).Methods(http.MethodPost)
infraOnboardingSubRouter := router.PathPrefix("/api/v1/infra_onboarding").Subrouter()
infraOnboardingSubRouter.HandleFunc("/k8s/status", am.ViewAccess(aH.getK8sInfraOnboardingStatus)).Methods(http.MethodGet)
}
func (aH *APIHandler) RegisterWebSocketPaths(router *mux.Router, am *middleware.AuthZ) {
subRouter := router.PathPrefix("/ws").Subrouter()
subRouter.HandleFunc("/query_progress", am.ViewAccess(aH.GetQueryProgressUpdates)).Methods(http.MethodGet)

View File

@@ -1,908 +0,0 @@
package app
import (
"encoding/json"
"net/http"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/query-service/model"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
func (aH *APIHandler) getHostAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
// get attribute keys
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.hostsRepo.GetHostAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
// write response
aH.Respond(w, keys)
}
func (aH *APIHandler) getHostAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
// parse request
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
// get attribute values
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.hostsRepo.GetHostAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
// write response
aH.Respond(w, values)
}
func (aH *APIHandler) getHostList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.HostListRequest{}
// parse request
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
// get host list
hostList, err := aH.hostsRepo.GetHostList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
// write response
aH.Respond(w, hostList)
}
func (aH *APIHandler) getProcessAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.processesRepo.GetProcessAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getProcessAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.processesRepo.GetProcessAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getProcessList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.ProcessListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
hostList, err := aH.processesRepo.GetProcessList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, hostList)
}
func (aH *APIHandler) getPodAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.podsRepo.GetPodAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getPodAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.podsRepo.GetPodAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getPodList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.PodListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
podList, err := aH.podsRepo.GetPodList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, podList)
}
func (aH *APIHandler) getNodeAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.nodesRepo.GetNodeAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getNodeAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.nodesRepo.GetNodeAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getNodeList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.NodeListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
nodeList, err := aH.nodesRepo.GetNodeList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, nodeList)
}
func (aH *APIHandler) getNamespaceAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
keys, err := aH.namespacesRepo.GetNamespaceAttributeKeys(ctx, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getNamespaceAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.namespacesRepo.GetNamespaceAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getNamespaceList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.NamespaceListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
namespaceList, err := aH.namespacesRepo.GetNamespaceList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, namespaceList)
}
func (aH *APIHandler) getClusterAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.clustersRepo.GetClusterAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getClusterAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.clustersRepo.GetClusterAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getClusterList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.ClusterListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
clusterList, err := aH.clustersRepo.GetClusterList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, clusterList)
}
func (aH *APIHandler) getDeploymentAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.deploymentsRepo.GetDeploymentAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getDeploymentAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.deploymentsRepo.GetDeploymentAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getDeploymentList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.DeploymentListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
deploymentList, err := aH.deploymentsRepo.GetDeploymentList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, deploymentList)
}
func (aH *APIHandler) getDaemonSetAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.daemonsetsRepo.GetDaemonSetAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getDaemonSetAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.daemonsetsRepo.GetDaemonSetAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getDaemonSetList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.DaemonSetListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
daemonSetList, err := aH.daemonsetsRepo.GetDaemonSetList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, daemonSetList)
}
func (aH *APIHandler) getStatefulSetAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.statefulsetsRepo.GetStatefulSetAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getStatefulSetAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.statefulsetsRepo.GetStatefulSetAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getStatefulSetList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.StatefulSetListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
statefulSetList, err := aH.statefulsetsRepo.GetStatefulSetList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, statefulSetList)
}
func (aH *APIHandler) getJobAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.jobsRepo.GetJobAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getJobAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.jobsRepo.GetJobAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getJobList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.JobListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
jobList, err := aH.jobsRepo.GetJobList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, jobList)
}
func (aH *APIHandler) getPvcList(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(w, err)
return
}
req := model.VolumeListRequest{}
err = json.NewDecoder(r.Body).Decode(&req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
pvcList, err := aH.pvcsRepo.GetPvcList(ctx, orgID, req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, pvcList)
}
func (aH *APIHandler) getPvcAttributeKeys(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeKeyRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
keys, err := aH.pvcsRepo.GetPvcAttributeKeys(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, keys)
}
func (aH *APIHandler) getPvcAttributeValues(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
req, err := parseFilterAttributeValueRequest(r)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
claims, err := authtypes.ClaimsFromContext(r.Context())
if err != nil {
render.Error(w, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
values, err := aH.pvcsRepo.GetPvcAttributeValues(ctx, orgID, *req)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
aH.Respond(w, values)
}
func (aH *APIHandler) getK8sInfraOnboardingStatus(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
status := model.OnboardingStatus{}
didSendPodMetrics, err := aH.podsRepo.DidSendPodMetrics(ctx)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
if !didSendPodMetrics {
aH.Respond(w, status)
return
}
didSendClusterMetrics, err := aH.podsRepo.DidSendClusterMetrics(ctx)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
didSendNodeMetrics, err := aH.nodesRepo.DidSendNodeMetrics(ctx)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
didSendOptionalPodMetrics, err := aH.podsRepo.IsSendingOptionalPodMetrics(ctx)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
requiredMetadata, err := aH.podsRepo.SendingRequiredMetadata(ctx)
if err != nil {
RespondError(w, &model.ApiError{Typ: model.ErrorInternal, Err: err}, nil)
return
}
status.DidSendPodMetrics = didSendPodMetrics
status.DidSendClusterMetrics = didSendClusterMetrics
status.DidSendNodeMetrics = didSendNodeMetrics
status.IsSendingOptionalPodMetrics = didSendOptionalPodMetrics
status.IsSendingRequiredMetadata = requiredMetadata
aH.Respond(w, status)
}

View File

@@ -1,356 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForClusters = "k8s.node.cpu.usage"
clusterAttrsToEnrich = []string{"k8s.cluster.name"}
// TODO(srikanthccv): change this to k8s_cluster_uid after showing the missing data banner
k8sClusterUIDAttrKey = "k8s.cluster.name"
queryNamesForClusters = map[string][]string{
"cpu": {"A"},
"cpu_allocatable": {"B"},
"memory": {"C"},
"memory_allocatable": {"D"},
}
clusterQueryNames = []string{"A", "B", "C", "D"}
)
type ClustersRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewClustersRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *ClustersRepo {
return &ClustersRepo{reader: reader, querierV2: querierV2}
}
func (n *ClustersRepo) GetClusterAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForClusters
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := n.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeKeysResponse, nil
}
func (n *ClustersRepo) GetClusterAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForClusters
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := n.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (p *ClustersRepo) getMetadataAttributes(ctx context.Context, req model.ClusterListRequest) (map[string]map[string]string, error) {
clusterAttrs := map[string]map[string]string{}
for _, key := range clusterAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForClusters,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := p.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
clusterUID := stringData[k8sClusterUIDAttrKey]
if _, ok := clusterAttrs[clusterUID]; !ok {
clusterAttrs[clusterUID] = map[string]string{}
}
for _, key := range req.GroupBy {
clusterAttrs[clusterUID][key.Key] = stringData[key.Key]
}
}
return clusterAttrs, nil
}
func (p *ClustersRepo) getTopClusterGroups(ctx context.Context, orgID valuer.UUID, req model.ClusterListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopClusters(req)
queryNames := queryNamesForClusters[req.OrderBy.ColumnName]
topClusterGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topClusterGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopClusterGroups",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, topClusterGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topClusterGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
max := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopClusterGroupsSeries := formattedResponse[0].Series[req.Offset:int(max)]
topClusterGroups := []map[string]string{}
for _, series := range paginatedTopClusterGroupsSeries {
topClusterGroups = append(topClusterGroups, series.Labels)
}
allClusterGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allClusterGroups = append(allClusterGroups, series.Labels)
}
return topClusterGroups, allClusterGroups, nil
}
func (p *ClustersRepo) GetClusterList(ctx context.Context, orgID valuer.UUID, req model.ClusterListRequest) (model.ClusterListResponse, error) {
resp := model.ClusterListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sClusterUIDAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := NodesTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
}
clusterAttrs, err := p.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topClusterGroups, allClusterGroups, err := p.getTopClusterGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topClusterGroup := range topClusterGroups {
for k, v := range topClusterGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetClusterList",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.ClusterListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.ClusterListRecord{
CPUUsage: -1,
CPUAllocatable: -1,
MemoryUsage: -1,
MemoryAllocatable: -1,
}
if clusterUID, ok := row.Data[k8sClusterUIDAttrKey].(string); ok {
record.ClusterUID = clusterUID
}
if cpu, ok := row.Data["A"].(float64); ok {
record.CPUUsage = cpu
}
if cpuAllocatable, ok := row.Data["B"].(float64); ok {
record.CPUAllocatable = cpuAllocatable
}
if mem, ok := row.Data["C"].(float64); ok {
record.MemoryUsage = mem
}
if memoryAllocatable, ok := row.Data["D"].(float64); ok {
record.MemoryAllocatable = memoryAllocatable
}
record.Meta = map[string]string{}
if _, ok := clusterAttrs[record.ClusterUID]; ok && record.ClusterUID != "" {
record.Meta = clusterAttrs[record.ClusterUID]
}
for k, v := range row.Data {
if slices.Contains(clusterQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allClusterGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,208 +0,0 @@
package inframetrics
import (
"fmt"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/model"
)
const fromWhereQuery = `
FROM %s.%s
WHERE metric_name IN (%s)
AND unix_milli >= toUnixTimestamp(now() - toIntervalMinute(60)) * 1000
`
var (
// TODO(srikanthccv): import metadata yaml from receivers and use generated files to check the metrics
podMetricNamesToCheck = []string{
"k8s.pod.cpu.usage",
"k8s.pod.memory.working_set",
"k8s.pod.cpu_request_utilization",
"k8s.pod.memory_request_utilization",
"k8s.pod.cpu_limit_utilization",
"k8s.pod.memory_limit_utilization",
"k8s.container.restarts",
"k8s.pod.phase",
}
nodeMetricNamesToCheck = []string{
"k8s.node.cpu.usage",
"k8s.node.allocatable_cpu",
"k8s.node.memory.working_set",
"k8s.node.allocatable_memory",
"k8s.node.condition_ready",
}
clusterMetricNamesToCheck = []string{
"k8s.daemonset.desired_scheduled_nodes",
"k8s.daemonset.current_scheduled_nodes",
"k8s.deployment.desired",
"k8s.deployment.available",
"k8s.job.desired_successful_pods",
"k8s.job.active_pods",
"k8s.job.failed_pods",
"k8s.job.successful_pods",
"k8s.statefulset.desired_pods",
"k8s.statefulset.current_pods",
}
optionalPodMetricNamesToCheck = []string{
"k8s.pod.cpu_request_utilization",
"k8s.pod.memory_request_utilization",
"k8s.pod.cpu_limit_utilization",
"k8s.pod.memory_limit_utilization",
}
// did they ever send _any_ pod metrics?
didSendPodMetricsQuery = `
SELECT count() FROM %s.%s WHERE metric_name IN (%s)
`
// did they ever send any node metrics?
didSendNodeMetricsQuery = `
SELECT count() FROM %s.%s WHERE metric_name IN (%s)
`
// did they ever send any cluster metrics?
didSendClusterMetricsQuery = `
SELECT count() FROM %s.%s WHERE metric_name IN (%s)
`
// if they ever sent _any_ pod metrics, we assume they know how to send pod metrics
// now, are they sending optional pod metrics such request/limit metrics?
isSendingOptionalPodMetricsQuery = `
SELECT count() FROM %s.%s WHERE metric_name IN (%s)
`
// there should be [cluster, node, namespace, one of (deployment, statefulset, daemonset, cronjob, job)] for each pod
selectQuery = fmt.Sprintf(`
SELECT
any(JSONExtractString(labels, '%s')) as k8s_cluster_name,
any(JSONExtractString(labels, '%s')) as k8s_node_name,
any(JSONExtractString(labels, '%s')) as k8s_namespace_name,
any(JSONExtractString(labels, '%s')) as k8s_deployment_name,
any(JSONExtractString(labels, '%s')) as k8s_statefulset_name,
any(JSONExtractString(labels, '%s')) as k8s_daemonset_name,
any(JSONExtractString(labels, '%s')) as k8s_cronjob_name,
any(JSONExtractString(labels, '%s')) as k8s_job_name,
JSONExtractString(labels, '%s') as k8s_pod_name
`,
"k8s.cluster.name",
"k8s.node.name",
"k8s.namespace.name",
"k8s.deployment.name",
"k8s.statefulset.name",
"k8s.daemonset.name",
"k8s.cronjob.name",
"k8s.job.name",
"k8s.pod.name",
)
filterGroupQuery = fmt.Sprintf(`
AND JSONExtractString(labels, '%s')
NOT IN ('kube-system','kube-public','kube-node-lease','metallb-system')
GROUP BY k8s_pod_name
LIMIT 1 BY k8s_cluster_name, k8s_node_name, k8s_namespace_name
`,
"k8s.namespace.name",
)
isSendingRequiredMetadataQuery = selectQuery + fromWhereQuery + filterGroupQuery
)
// getParamsForTopItems returns the step, time series table name and samples table name
// for the top items query. what are we doing here?
// we want to identify the top hosts/pods/nodes quickly, so we use pre-aggregated data
// for samples and time series tables to speed up the query
// the speed of the query depends on the number of values in group by clause, the higher
// the step interval, the faster the query will be as number of rows to group by is reduced
// here we are using the averaged value of the time series data to get the top items
func getParamsForTopItems(start, end int64) (int64, string, string) {
var step int64
var timeSeriesTableName string
var samplesTableName string
if end-start < time.Hour.Milliseconds() {
// 5 minute aggregation for any query less than 1 hour
step = 5 * 60
timeSeriesTableName = constants.SIGNOZ_TIMESERIES_v4_LOCAL_TABLENAME
samplesTableName = constants.SIGNOZ_SAMPLES_V4_AGG_5M_TABLENAME
} else if end-start < time.Hour.Milliseconds()*6 {
// 15 minute aggregation for any query less than 6 hours
step = 15 * 60
timeSeriesTableName = constants.SIGNOZ_TIMESERIES_v4_6HRS_LOCAL_TABLENAME
samplesTableName = constants.SIGNOZ_SAMPLES_V4_AGG_5M_TABLENAME
} else if end-start < time.Hour.Milliseconds()*24 {
// 1 hour aggregation for any query less than 1 day
step = 60 * 60
timeSeriesTableName = constants.SIGNOZ_TIMESERIES_v4_1DAY_LOCAL_TABLENAME
samplesTableName = constants.SIGNOZ_SAMPLES_V4_AGG_30M_TABLENAME
} else if end-start < time.Hour.Milliseconds()*7 {
// 6 hours aggregation for any query less than 1 week
step = 6 * 60 * 60
timeSeriesTableName = constants.SIGNOZ_TIMESERIES_v4_1WEEK_LOCAL_TABLENAME
samplesTableName = constants.SIGNOZ_SAMPLES_V4_AGG_30M_TABLENAME
} else {
// 12 hours aggregation for any query greater than 1 week
step = 12 * 60 * 60
timeSeriesTableName = constants.SIGNOZ_TIMESERIES_v4_1WEEK_LOCAL_TABLENAME
samplesTableName = constants.SIGNOZ_SAMPLES_V4_AGG_30M_TABLENAME
}
return step, timeSeriesTableName, samplesTableName
}
func getParamsForTopHosts(req model.HostListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopProcesses(req model.ProcessListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopPods(req model.PodListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopNodes(req model.NodeListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopNamespaces(req model.NamespaceListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopClusters(req model.ClusterListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopDeployments(req model.DeploymentListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopDaemonSets(req model.DaemonSetListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopStatefulSets(req model.StatefulSetListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopJobs(req model.JobListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
func getParamsForTopVolumes(req model.VolumeListRequest) (int64, string, string) {
return getParamsForTopItems(req.Start, req.End)
}
// TODO(srikanthccv): remove this
// What is happening here?
// The `PrepareTimeseriesFilterQuery` uses the local time series table for sub-query because each fingerprint
// goes to same shard.
// However, in this case, we are interested in the attributes values across all the shards.
// So, we replace the local time series table with the distributed time series table.
// See `PrepareTimeseriesFilterQuery` for more details.
func localQueryToDistributedQuery(query string) string {
return strings.Replace(query, ".time_series_v4", ".distributed_time_series_v4", 1)
}

View File

@@ -1,457 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForDaemonSets = "k8s.pod.cpu.usage"
k8sDaemonSetNameAttrKey = "k8s.daemonset.name"
metricNamesForDaemonSets = map[string]string{
"desired_nodes": "k8s.daemonset.desired_scheduled_nodes",
"available_nodes": "k8s.daemonset.current_scheduled_nodes",
}
daemonSetAttrsToEnrich = []string{
"k8s.daemonset.name",
"k8s.namespace.name",
"k8s.cluster.name",
}
queryNamesForDaemonSets = map[string][]string{
"cpu": {"A"},
"cpu_request": {"B", "A"},
"cpu_limit": {"C", "A"},
"memory": {"D"},
"memory_request": {"E", "D"},
"memory_limit": {"F", "D"},
"restarts": {"G", "A"},
"desired_nodes": {"H"},
"available_nodes": {"I"},
}
builderQueriesForDaemonSets = map[string]*v3.BuilderQuery{
// desired nodes
"H": {
QueryName: "H",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForDaemonSets["desired_nodes"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "H",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// available nodes
"I": {
QueryName: "I",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForDaemonSets["available_nodes"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "I",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
}
daemonSetQueryNames = []string{"A", "B", "C", "D", "E", "F", "G", "H", "I"}
)
type DaemonSetsRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewDaemonSetsRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *DaemonSetsRepo {
return &DaemonSetsRepo{reader: reader, querierV2: querierV2}
}
func (d *DaemonSetsRepo) GetDaemonSetAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
// TODO(srikanthccv): remove hardcoded metric name and support keys from any pod metric
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForDaemonSets
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := d.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (d *DaemonSetsRepo) GetDaemonSetAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForDaemonSets
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := d.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (d *DaemonSetsRepo) getMetadataAttributes(ctx context.Context, req model.DaemonSetListRequest) (map[string]map[string]string, error) {
daemonSetAttrs := map[string]map[string]string{}
for _, key := range daemonSetAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForDaemonSets,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := d.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
daemonSetName := stringData[k8sDaemonSetNameAttrKey]
if _, ok := daemonSetAttrs[daemonSetName]; !ok {
daemonSetAttrs[daemonSetName] = map[string]string{}
}
for _, key := range req.GroupBy {
daemonSetAttrs[daemonSetName][key.Key] = stringData[key.Key]
}
}
return daemonSetAttrs, nil
}
func (d *DaemonSetsRepo) getTopDaemonSetGroups(ctx context.Context, orgID valuer.UUID, req model.DaemonSetListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopDaemonSets(req)
queryNames := queryNamesForDaemonSets[req.OrderBy.ColumnName]
topDaemonSetGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topDaemonSetGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopDaemonSetGroups",
})
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, topDaemonSetGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topDaemonSetGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopDaemonSetGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topDaemonSetGroups := []map[string]string{}
for _, series := range paginatedTopDaemonSetGroupsSeries {
topDaemonSetGroups = append(topDaemonSetGroups, series.Labels)
}
allDaemonSetGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allDaemonSetGroups = append(allDaemonSetGroups, series.Labels)
}
return topDaemonSetGroups, allDaemonSetGroups, nil
}
func (d *DaemonSetsRepo) GetDaemonSetList(ctx context.Context, orgID valuer.UUID, req model.DaemonSetListRequest) (model.DaemonSetListResponse, error) {
resp := model.DaemonSetListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sDaemonSetNameAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := WorkloadTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
// add additional queries for daemon sets
for _, daemonSetQuery := range builderQueriesForDaemonSets {
query.CompositeQuery.BuilderQueries[daemonSetQuery.QueryName] = daemonSetQuery.Clone()
}
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
// make sure we only get records for daemon sets
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: k8sDaemonSetNameAttrKey},
Operator: v3.FilterOperatorExists,
})
}
daemonSetAttrs, err := d.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topDaemonSetGroups, allDaemonSetGroups, err := d.getTopDaemonSetGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topDaemonSetGroup := range topDaemonSetGroups {
for k, v := range topDaemonSetGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetDaemonSetList",
})
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.DaemonSetListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.DaemonSetListRecord{
DaemonSetName: "",
CPUUsage: -1,
CPURequest: -1,
CPULimit: -1,
MemoryUsage: -1,
MemoryRequest: -1,
MemoryLimit: -1,
DesiredNodes: -1,
AvailableNodes: -1,
}
if daemonSetName, ok := row.Data[k8sDaemonSetNameAttrKey].(string); ok {
record.DaemonSetName = daemonSetName
}
if cpu, ok := row.Data["A"].(float64); ok {
record.CPUUsage = cpu
}
if cpuRequest, ok := row.Data["B"].(float64); ok {
record.CPURequest = cpuRequest
}
if cpuLimit, ok := row.Data["C"].(float64); ok {
record.CPULimit = cpuLimit
}
if memory, ok := row.Data["D"].(float64); ok {
record.MemoryUsage = memory
}
if memoryRequest, ok := row.Data["E"].(float64); ok {
record.MemoryRequest = memoryRequest
}
if memoryLimit, ok := row.Data["F"].(float64); ok {
record.MemoryLimit = memoryLimit
}
if restarts, ok := row.Data["G"].(float64); ok {
record.Restarts = int(restarts)
}
if desiredNodes, ok := row.Data["H"].(float64); ok {
record.DesiredNodes = int(desiredNodes)
}
if availableNodes, ok := row.Data["I"].(float64); ok {
record.AvailableNodes = int(availableNodes)
}
record.Meta = map[string]string{}
if _, ok := daemonSetAttrs[record.DaemonSetName]; ok && record.DaemonSetName != "" {
record.Meta = daemonSetAttrs[record.DaemonSetName]
}
for k, v := range row.Data {
if slices.Contains(daemonSetQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allDaemonSetGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,453 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForDeployments = "k8s.pod.cpu.usage"
k8sDeploymentNameAttrKey = "k8s.deployment.name"
metricNamesForDeployments = map[string]string{
"desired_pods": "k8s.deployment.desired",
"available_pods": "k8s.deployment.available",
}
deploymentAttrsToEnrich = []string{
"k8s.deployment.name",
"k8s.namespace.name",
"k8s.cluster.name",
}
queryNamesForDeployments = map[string][]string{
"cpu": {"A"},
"cpu_request": {"B", "A"},
"cpu_limit": {"C", "A"},
"memory": {"D"},
"memory_request": {"E", "D"},
"memory_limit": {"F", "D"},
"restarts": {"G", "A"},
"desired_pods": {"H"},
"available_pods": {"I"},
}
builderQueriesForDeployments = map[string]*v3.BuilderQuery{
// desired pods
"H": {
QueryName: "H",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForDeployments["desired_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "H",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// available pods
"I": {
QueryName: "I",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForDeployments["available_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "I",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
}
deploymentQueryNames = []string{"A", "B", "C", "D", "E", "F", "G", "H", "I"}
)
type DeploymentsRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewDeploymentsRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *DeploymentsRepo {
return &DeploymentsRepo{reader: reader, querierV2: querierV2}
}
func (d *DeploymentsRepo) GetDeploymentAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
// TODO(srikanthccv): remove hardcoded metric name and support keys from any pod metric
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForDeployments
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := d.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (d *DeploymentsRepo) GetDeploymentAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForDeployments
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := d.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (d *DeploymentsRepo) getMetadataAttributes(ctx context.Context, req model.DeploymentListRequest) (map[string]map[string]string, error) {
deploymentAttrs := map[string]map[string]string{}
for _, key := range deploymentAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForDeployments,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := d.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
deploymentName := stringData[k8sDeploymentNameAttrKey]
if _, ok := deploymentAttrs[deploymentName]; !ok {
deploymentAttrs[deploymentName] = map[string]string{}
}
for _, key := range req.GroupBy {
deploymentAttrs[deploymentName][key.Key] = stringData[key.Key]
}
}
return deploymentAttrs, nil
}
func (d *DeploymentsRepo) getTopDeploymentGroups(ctx context.Context, orgID valuer.UUID, req model.DeploymentListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopDeployments(req)
queryNames := queryNamesForDeployments[req.OrderBy.ColumnName]
topDeploymentGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topDeploymentGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopDeploymentGroups",
})
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, topDeploymentGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topDeploymentGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopDeploymentGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topDeploymentGroups := []map[string]string{}
for _, series := range paginatedTopDeploymentGroupsSeries {
topDeploymentGroups = append(topDeploymentGroups, series.Labels)
}
allDeploymentGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allDeploymentGroups = append(allDeploymentGroups, series.Labels)
}
return topDeploymentGroups, allDeploymentGroups, nil
}
func (d *DeploymentsRepo) GetDeploymentList(ctx context.Context, orgID valuer.UUID, req model.DeploymentListRequest) (model.DeploymentListResponse, error) {
resp := model.DeploymentListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sDeploymentNameAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := WorkloadTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
// add additional queries for deployments
for _, deploymentQuery := range builderQueriesForDeployments {
query.CompositeQuery.BuilderQueries[deploymentQuery.QueryName] = deploymentQuery.Clone()
}
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
// make sure we only get records for deployments
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: k8sDeploymentNameAttrKey},
Operator: v3.FilterOperatorExists,
})
}
deploymentAttrs, err := d.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topDeploymentGroups, allDeploymentGroups, err := d.getTopDeploymentGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topDeploymentGroup := range topDeploymentGroups {
for k, v := range topDeploymentGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.DeploymentListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.DeploymentListRecord{
DeploymentName: "",
CPUUsage: -1,
CPURequest: -1,
CPULimit: -1,
MemoryUsage: -1,
MemoryRequest: -1,
MemoryLimit: -1,
DesiredPods: -1,
AvailablePods: -1,
}
if deploymentName, ok := row.Data[k8sDeploymentNameAttrKey].(string); ok {
record.DeploymentName = deploymentName
}
if cpu, ok := row.Data["A"].(float64); ok {
record.CPUUsage = cpu
}
if cpuRequest, ok := row.Data["B"].(float64); ok {
record.CPURequest = cpuRequest
}
if cpuLimit, ok := row.Data["C"].(float64); ok {
record.CPULimit = cpuLimit
}
if memory, ok := row.Data["D"].(float64); ok {
record.MemoryUsage = memory
}
if memoryRequest, ok := row.Data["E"].(float64); ok {
record.MemoryRequest = memoryRequest
}
if memoryLimit, ok := row.Data["F"].(float64); ok {
record.MemoryLimit = memoryLimit
}
if restarts, ok := row.Data["G"].(float64); ok {
record.Restarts = int(restarts)
}
if desiredPods, ok := row.Data["H"].(float64); ok {
record.DesiredPods = int(desiredPods)
}
if availablePods, ok := row.Data["I"].(float64); ok {
record.AvailablePods = int(availablePods)
}
record.Meta = map[string]string{}
if _, ok := deploymentAttrs[record.DeploymentName]; ok && record.DeploymentName != "" {
record.Meta = deploymentAttrs[record.DeploymentName]
}
for k, v := range row.Data {
if slices.Contains(deploymentQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allDeploymentGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,550 +0,0 @@
package inframetrics
import (
"context"
"errors"
"fmt"
"math"
"sort"
"strings"
"time"
"log/slog"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
)
type HostsRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
var (
// we don't have a way to get the resource attributes from the current time series table
// but we only want to suggest resource attributes for system metrics,
// this is a list of attributes that we skip from all labels as they are data point attributes
// TODO(srikanthccv): remove this once we have a way to get resource attributes
pointAttrsToIgnore = []string{
"state",
"cpu",
"device",
"direction",
"mode",
"mountpoint",
"type",
"os.type",
"process.cgroup",
"process.command",
"process.command_line",
"process.executable.name",
"process.executable.path",
"process.owner",
"process.parent_pid",
"process.pid",
}
queryNamesForTopHosts = map[string][]string{
"cpu": {"A", "B", "F1"},
"memory": {"C", "D", "F2"},
"wait": {"E", "F", "F3"},
"load15": {"G"},
}
// TODO(srikanthccv): remove hardcoded metric name and support keys from any system metric
metricToUseForHostAttributes = "system.cpu.load_average.15m"
hostNameAttrKey = "host.name"
agentNameToIgnore = "k8s-infra-otel-agent"
hostAttrsToEnrich = []string{
"os.type",
}
metricNamesForHosts = map[string]string{
"filesystem": "system.filesystem.usage",
"cpu": "system.cpu.time",
"memory": "system.memory.usage",
"load15": "system.cpu.load_average.15m",
"wait": "system.cpu.time",
}
uniqueMetricNamesForHosts = []string{
"system.uptime",
"system.cpu.time",
"system.cpu.load_average.1m",
"system.cpu.load_average.5m",
"system.cpu.load_average.15m",
"system.memory.usage",
"system.paging.usage",
"system.paging.faults",
"system.paging.operations",
"system.disk.io",
"system.disk.operations",
"system.disk.io_time",
"system.disk.operation_time",
"system.disk.merged",
"system.disk.pending_operations",
"system.disk.weighted_io_time",
"system.filesystem.usage",
"system.filesystem.inodes.usage",
"system.network.io",
"system.network.errors",
"system.network.connections",
"system.network.dropped",
"system.network.packets",
"system.processes.count",
"system.processes.created",
"process.cpu.time",
"process.disk.io",
"process.memory.usage",
"process.memory.virtual",
"nfs.client.net.count",
"nfs.client.net.tcp.connection.accepted",
"nfs.client.operation.count",
"nfs.client.procedure.count",
"nfs.client.rpc.authrefresh.count",
"nfs.client.rpc.count",
"nfs.client.rpc.retransmit.count",
"nfs.server.fh.stale.count",
"nfs.server.io",
"nfs.server.net.count",
"nfs.server.net.tcp.connection.accepted",
"nfs.server.operation.count",
"nfs.server.procedure.count",
"nfs.server.repcache.requests",
"nfs.server.rpc.count",
"nfs.server.thread.count",
}
)
func NewHostsRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *HostsRepo {
return &HostsRepo{reader: reader, querierV2: querierV2}
}
func (h *HostsRepo) GetHostAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
// TODO(srikanthccv): remove hardcoded metric name and support keys from any system metric
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForHostAttributes
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := h.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (h *HostsRepo) GetHostAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForHostAttributes
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := h.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
if req.FilterAttributeKey != hostNameAttrKey {
return attributeValuesResponse, nil
}
hostNames := []string{}
for _, attributeValue := range attributeValuesResponse.StringAttributeValues {
if strings.Contains(attributeValue, agentNameToIgnore) {
continue
}
hostNames = append(hostNames, attributeValue)
}
return &v3.FilterAttributeValueResponse{StringAttributeValues: hostNames}, nil
}
func (h *HostsRepo) getActiveHosts(ctx context.Context) (map[string]bool, error) {
tenMinAgo := time.Now().Add(-10 * time.Minute).UTC().UnixMilli()
return h.reader.GetActiveHostsFromMetricMetadata(ctx, uniqueMetricNamesForHosts, hostNameAttrKey, tenMinAgo)
}
func (h *HostsRepo) getMetadataAttributes(ctx context.Context, req model.HostListRequest) (map[string]map[string]string, error) {
hostAttrs := map[string]map[string]string{}
for _, key := range hostAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForHostAttributes,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := h.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
hostName := stringData[hostNameAttrKey]
if _, ok := hostAttrs[hostName]; !ok {
hostAttrs[hostName] = map[string]string{}
}
for _, key := range req.GroupBy {
hostAttrs[hostName][key.Key] = stringData[key.Key]
}
}
return hostAttrs, nil
}
func (h *HostsRepo) getTopHostGroups(ctx context.Context, orgID valuer.UUID, req model.HostListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopHosts(req)
queryNames := queryNamesForTopHosts[req.OrderBy.ColumnName]
topHostGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topHostGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopHostGroups",
})
queryResponse, _, err := h.querierV2.QueryRange(ctx, orgID, topHostGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topHostGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopHostGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topHostGroups := []map[string]string{}
for _, series := range paginatedTopHostGroupsSeries {
topHostGroups = append(topHostGroups, series.Labels)
}
allHostGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allHostGroups = append(allHostGroups, series.Labels)
}
return topHostGroups, allHostGroups, nil
}
// GetHostMetricsExistenceAndEarliestTime returns (count, minFirstReportedUnixMilli, error) for host metrics
// in distributed_metadata. Uses metricNamesForHosts plus system.filesystem.usage.
func (h *HostsRepo) GetHostMetricsExistenceAndEarliestTime(ctx context.Context, req model.HostListRequest) (uint64, uint64, error) {
names := []string{}
for _, metricName := range metricNamesForHosts {
names = append(names, metricName)
}
return h.reader.GetMetricsExistenceAndEarliestTime(ctx, names)
}
func (h *HostsRepo) IsSendingK8SAgentMetrics(ctx context.Context, req model.HostListRequest) ([]string, []string, error) {
names := []string{}
for _, metricName := range metricNamesForHosts {
names = append(names, metricName)
}
namesStr := "'" + strings.Join(names, "','") + "'"
queryForRecentFingerprints := fmt.Sprintf(`
SELECT DISTINCT fingerprint
FROM %s.%s
WHERE metric_name IN (%s)
AND unix_milli >= toUnixTimestamp(now() - INTERVAL 5 MINUTE) * 1000`,
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_SAMPLES_V4_TABLENAME, namesStr)
query := fmt.Sprintf(`
SELECT DISTINCT JSONExtractString(labels, '%s') as k8s_cluster_name, JSONExtractString(labels, '%s') as k8s_node_name
FROM %s.%s
WHERE metric_name IN (%s)
AND unix_milli >= toUnixTimestamp(now() - INTERVAL 60 MINUTE) * 1000
AND JSONExtractString(labels, '%s') LIKE '%%-otel-agent%%'
AND fingerprint GLOBAL IN (%s)`,
"k8s.cluster.name", "k8s.node.name",
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_TIMESERIES_V4_TABLENAME, namesStr, "host.name", queryForRecentFingerprints)
result, err := h.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, nil, err
}
clusterNames := make(map[string]struct{})
nodeNames := make(map[string]struct{})
for _, row := range result {
switch v := row.Data["k8s.cluster.name"].(type) {
case string:
clusterNames[v] = struct{}{}
case *string:
clusterNames[*v] = struct{}{}
}
switch v := row.Data["k8s.node.name"].(type) {
case string:
nodeNames[v] = struct{}{}
case *string:
nodeNames[*v] = struct{}{}
}
}
return maps.Keys(clusterNames), maps.Keys(nodeNames), nil
}
func (h *HostsRepo) GetHostList(ctx context.Context, orgID valuer.UUID, req model.HostListRequest) (model.HostListResponse, error) {
resp := model.HostListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
// default to cpu order by
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
// default to host name group by
if len(req.GroupBy) == 0 {
req.GroupBy = []v3.AttributeKey{{Key: hostNameAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
// don't fail the request if we can't get these values
if clusterNames, nodeNames, err := h.IsSendingK8SAgentMetrics(ctx, req); err == nil {
resp.IsSendingK8SAgentMetrics = len(clusterNames) > 0 || len(nodeNames) > 0
resp.ClusterNames = clusterNames
resp.NodeNames = nodeNames
}
// 1. Check if any host metrics exist and get earliest retention time
// if no hosts metrics exist, that means we should show the onboarding guide on UI, and return early.
// 2. If host metrics exist, but req.End is earlier than the earliest time of host metrics as read from
// metadata table, then we should convey the same to the user and return early
if count, minFirstReportedUnixMilli, err := h.GetHostMetricsExistenceAndEarliestTime(ctx, req); err == nil {
if count == 0 {
resp.SentAnyHostMetricsData = false
resp.Records = []model.HostListRecord{}
resp.Total = 0
return resp, nil
}
resp.SentAnyHostMetricsData = true
if req.End < int64(minFirstReportedUnixMilli) {
resp.EndTimeBeforeRetention = true
resp.Records = []model.HostListRecord{}
resp.Total = 0
return resp, nil
}
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
if step <= 0 {
slog.ErrorContext(ctx, "step is less than or equal to 0", "step", step)
return resp, errors.New("step is less than or equal to 0")
}
query := HostsTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
}
hostAttrs, err := h.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
activeHosts, err := h.getActiveHosts(ctx)
if err != nil {
return resp, err
}
topHostGroups, allHostGroups, err := h.getTopHostGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topHostGroup := range topHostGroups {
for k, v := range topHostGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetHostList",
})
queryResponse, _, err := h.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.HostListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.HostListRecord{
CPU: -1,
Memory: -1,
Wait: -1,
Load15: -1,
}
if hostName, ok := row.Data[hostNameAttrKey].(string); ok {
record.HostName = hostName
}
if cpu, ok := row.Data["F1"].(float64); ok {
record.CPU = cpu
}
if memory, ok := row.Data["F2"].(float64); ok {
record.Memory = memory
}
if wait, ok := row.Data["F3"].(float64); ok {
record.Wait = wait
}
if load15, ok := row.Data["G"].(float64); ok {
record.Load15 = load15
}
record.Meta = map[string]string{}
if _, ok := hostAttrs[record.HostName]; ok {
record.Meta = hostAttrs[record.HostName]
}
if osType, ok := record.Meta["os.type"]; ok {
record.OS = osType
}
record.Active = activeHosts[record.HostName]
records = append(records, record)
}
}
resp.Total = len(allHostGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,314 +0,0 @@
package inframetrics
import v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
var HostsTableListQuery = v3.QueryRangeParamsV3{
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{
"A": {
QueryName: "A",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForHosts["cpu"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "state",
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeTag,
},
Operator: v3.FilterOperatorNotEqual,
Value: "idle",
},
{
Key: v3.AttributeKey{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotContains,
Value: agentNameToIgnore,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "A",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationRate,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: true,
},
"B": {
QueryName: "B",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForHosts["cpu"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotContains,
Value: agentNameToIgnore,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "B",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationRate,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: true,
},
"F1": {
QueryName: "F1",
Expression: "A/B",
Legend: "CPU Usage (%)",
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
},
"C": {
QueryName: "C",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForHosts["memory"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "state",
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeTag,
},
Operator: v3.FilterOperatorEqual,
Value: "used",
},
{
Key: v3.AttributeKey{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotContains,
Value: agentNameToIgnore,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "C",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: true,
},
"D": {
QueryName: "D",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForHosts["memory"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotContains,
Value: agentNameToIgnore,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "D",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: true,
},
"F2": {
QueryName: "F2",
Expression: "C/D",
Legend: "Memory Usage (%)",
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
},
"E": {
QueryName: "E",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForHosts["wait"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "state",
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeTag,
},
Operator: v3.FilterOperatorEqual,
Value: "wait",
},
{
Key: v3.AttributeKey{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotContains,
Value: agentNameToIgnore,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "E",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationRate,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: true,
},
"F": {
QueryName: "F",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForHosts["wait"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotContains,
Value: agentNameToIgnore,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "F",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationRate,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: true,
},
"F3": {
QueryName: "F3",
Expression: "E/F",
Legend: "CPU Wait Time (%)",
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
},
"G": {
QueryName: "G",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForHosts["load15"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotContains,
Value: agentNameToIgnore,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: hostNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "G",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
Legend: "CPU Load Average (15m)",
},
},
PanelType: v3.PanelTypeTable,
QueryType: v3.QueryTypeBuilder,
},
Version: "v4",
FormatForWeb: true,
}

View File

@@ -1,511 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForJobs = "k8s.job.desired_successful_pods"
k8sJobNameAttrKey = "k8s.job.name"
metricNamesForJobs = map[string]string{
"desired_successful_pods": "k8s.job.desired_successful_pods",
"active_pods": "k8s.job.active_pods",
"failed_pods": "k8s.job.failed_pods",
"successful_pods": "k8s.job.successful_pods",
}
jobAttrsToEnrich = []string{
"k8s.job.name",
"k8s.namespace.name",
"k8s.cluster.name",
}
queryNamesForJobs = map[string][]string{
"cpu": {"A"},
"cpu_request": {"B", "A"},
"cpu_limit": {"C", "A"},
"memory": {"D"},
"memory_request": {"E", "D"},
"memory_limit": {"F", "D"},
"restarts": {"G", "A"},
"desired_successful_pods": {"H"},
"active_pods": {"I"},
"failed_pods": {"J"},
"successful_pods": {"K"},
}
builderQueriesForJobs = map[string]*v3.BuilderQuery{
// desired nodes
"H": {
QueryName: "H",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForJobs["desired_successful_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "H",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// available nodes
"I": {
QueryName: "I",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForJobs["active_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "I",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// failed pods
"J": {
QueryName: "J",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForJobs["failed_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "J",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// successful pods
"K": {
QueryName: "K",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForJobs["successful_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "K",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
}
jobQueryNames = []string{"A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K"}
)
type JobsRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewJobsRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *JobsRepo {
return &JobsRepo{reader: reader, querierV2: querierV2}
}
func (d *JobsRepo) GetJobAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
// TODO(srikanthccv): remove hardcoded metric name and support keys from any pod metric
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForJobs
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := d.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (d *JobsRepo) GetJobAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForJobs
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := d.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (d *JobsRepo) getMetadataAttributes(ctx context.Context, req model.JobListRequest) (map[string]map[string]string, error) {
jobAttrs := map[string]map[string]string{}
for _, key := range jobAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForJobs,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := d.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
jobName := stringData[k8sJobNameAttrKey]
if _, ok := jobAttrs[jobName]; !ok {
jobAttrs[jobName] = map[string]string{}
}
for _, key := range req.GroupBy {
jobAttrs[jobName][key.Key] = stringData[key.Key]
}
}
return jobAttrs, nil
}
func (d *JobsRepo) getTopJobGroups(ctx context.Context, orgID valuer.UUID, req model.JobListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopJobs(req)
queryNames := queryNamesForJobs[req.OrderBy.ColumnName]
topJobGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topJobGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopJobGroups",
})
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, topJobGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topJobGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopJobGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topJobGroups := []map[string]string{}
for _, series := range paginatedTopJobGroupsSeries {
topJobGroups = append(topJobGroups, series.Labels)
}
allJobGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allJobGroups = append(allJobGroups, series.Labels)
}
return topJobGroups, allJobGroups, nil
}
func (d *JobsRepo) GetJobList(ctx context.Context, orgID valuer.UUID, req model.JobListRequest) (model.JobListResponse, error) {
resp := model.JobListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "desired_pods", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sJobNameAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := WorkloadTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
// add additional queries for jobs
for _, jobQuery := range builderQueriesForJobs {
query.CompositeQuery.BuilderQueries[jobQuery.QueryName] = jobQuery.Clone()
}
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
// make sure we only get records for jobs
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: k8sJobNameAttrKey},
Operator: v3.FilterOperatorExists,
})
}
jobAttrs, err := d.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topJobGroups, allJobGroups, err := d.getTopJobGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topJobGroup := range topJobGroups {
for k, v := range topJobGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetJobList",
})
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.JobListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.JobListRecord{
JobName: "",
CPUUsage: -1,
CPURequest: -1,
CPULimit: -1,
MemoryUsage: -1,
MemoryRequest: -1,
MemoryLimit: -1,
DesiredSuccessfulPods: -1,
ActivePods: -1,
FailedPods: -1,
SuccessfulPods: -1,
}
if jobName, ok := row.Data[k8sJobNameAttrKey].(string); ok {
record.JobName = jobName
}
if cpu, ok := row.Data["A"].(float64); ok {
record.CPUUsage = cpu
}
if cpuRequest, ok := row.Data["B"].(float64); ok {
record.CPURequest = cpuRequest
}
if cpuLimit, ok := row.Data["C"].(float64); ok {
record.CPULimit = cpuLimit
}
if memory, ok := row.Data["D"].(float64); ok {
record.MemoryUsage = memory
}
if memoryRequest, ok := row.Data["E"].(float64); ok {
record.MemoryRequest = memoryRequest
}
if memoryLimit, ok := row.Data["F"].(float64); ok {
record.MemoryLimit = memoryLimit
}
if restarts, ok := row.Data["G"].(float64); ok {
record.Restarts = int(restarts)
}
if desiredSuccessfulPods, ok := row.Data["H"].(float64); ok {
record.DesiredSuccessfulPods = int(desiredSuccessfulPods)
}
if activePods, ok := row.Data["I"].(float64); ok {
record.ActivePods = int(activePods)
}
if failedPods, ok := row.Data["J"].(float64); ok {
record.FailedPods = int(failedPods)
}
if successfulPods, ok := row.Data["K"].(float64); ok {
record.SuccessfulPods = int(successfulPods)
}
record.Meta = map[string]string{}
if _, ok := jobAttrs[record.JobName]; ok && record.JobName != "" {
record.Meta = jobAttrs[record.JobName]
}
for k, v := range row.Data {
if slices.Contains(jobQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allJobGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,358 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForNamespaces = "k8s.pod.cpu.usage"
namespaceAttrsToEnrich = []string{
"k8s.namespace.name",
"k8s.cluster.name",
}
queryNamesForNamespaces = map[string][]string{
"cpu": {"A"},
"memory": {"D"},
"pod_phase": {"H", "I", "J", "K"},
}
namespaceQueryNames = []string{"A", "D", "H", "I", "J", "K"}
attributesKeysForNamespaces = []v3.AttributeKey{
{Key: "k8s.namespace.name"},
{Key: "k8s.cluster.name"},
}
k8sNamespaceNameAttrKey = "k8s.namespace.name"
)
type NamespacesRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewNamespacesRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *NamespacesRepo {
return &NamespacesRepo{reader: reader, querierV2: querierV2}
}
func (p *NamespacesRepo) GetNamespaceAttributeKeys(ctx context.Context, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
return &v3.FilterAttributeKeyResponse{AttributeKeys: attributesKeysForNamespaces}, nil
}
func (p *NamespacesRepo) GetNamespaceAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForNamespaces
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := p.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (p *NamespacesRepo) getMetadataAttributes(ctx context.Context, req model.NamespaceListRequest) (map[string]map[string]string, error) {
namespaceAttrs := map[string]map[string]string{}
for _, key := range namespaceAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForNamespaces,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := p.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
namespaceName := stringData[k8sNamespaceNameAttrKey]
if _, ok := namespaceAttrs[namespaceName]; !ok {
namespaceAttrs[namespaceName] = map[string]string{}
}
for _, key := range req.GroupBy {
namespaceAttrs[namespaceName][key.Key] = stringData[key.Key]
}
}
return namespaceAttrs, nil
}
func (p *NamespacesRepo) getTopNamespaceGroups(ctx context.Context, orgID valuer.UUID, req model.NamespaceListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopNamespaces(req)
queryNames := queryNamesForNamespaces[req.OrderBy.ColumnName]
topNamespaceGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topNamespaceGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopNamespaceGroups",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, topNamespaceGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topNamespaceGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopNamespaceGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topNamespaceGroups := []map[string]string{}
for _, series := range paginatedTopNamespaceGroupsSeries {
topNamespaceGroups = append(topNamespaceGroups, series.Labels)
}
allNamespaceGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allNamespaceGroups = append(allNamespaceGroups, series.Labels)
}
return topNamespaceGroups, allNamespaceGroups, nil
}
func (p *NamespacesRepo) GetNamespaceList(ctx context.Context, orgID valuer.UUID, req model.NamespaceListRequest) (model.NamespaceListResponse, error) {
resp := model.NamespaceListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sNamespaceNameAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := PodsTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
for _, q := range query.CompositeQuery.BuilderQueries {
if !slices.Contains(namespaceQueryNames, q.QueryName) {
delete(query.CompositeQuery.BuilderQueries, q.QueryName)
}
q.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if q.Filters == nil {
q.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
q.Filters.Items = append(q.Filters.Items, req.Filters.Items...)
}
q.GroupBy = req.GroupBy
}
namespaceAttrs, err := p.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topNamespaceGroups, allNamespaceGroups, err := p.getTopNamespaceGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topNamespaceGroup := range topNamespaceGroups {
for k, v := range topNamespaceGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetNamespaceList",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.NamespaceListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.NamespaceListRecord{
CPUUsage: -1,
MemoryUsage: -1,
}
if name, ok := row.Data[k8sNamespaceNameAttrKey].(string); ok {
record.NamespaceName = name
}
if cpu, ok := row.Data["A"].(float64); ok {
record.CPUUsage = cpu
}
if memory, ok := row.Data["D"].(float64); ok {
record.MemoryUsage = memory
}
if pending, ok := row.Data["H"].(float64); ok {
record.CountByPhase.Pending = int(pending)
}
if running, ok := row.Data["I"].(float64); ok {
record.CountByPhase.Running = int(running)
}
if succeeded, ok := row.Data["J"].(float64); ok {
record.CountByPhase.Succeeded = int(succeeded)
}
if failed, ok := row.Data["K"].(float64); ok {
record.CountByPhase.Failed = int(failed)
}
record.Meta = map[string]string{}
if _, ok := namespaceAttrs[record.NamespaceName]; ok && record.NamespaceName != "" {
record.Meta = namespaceAttrs[record.NamespaceName]
}
for k, v := range row.Data {
if slices.Contains(namespaceQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allNamespaceGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,387 +0,0 @@
package inframetrics
import (
"context"
"fmt"
"math"
"sort"
"strings"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForNodes = "k8s.node.cpu.usage"
nodeAttrsToEnrich = []string{"k8s.node.name", "k8s.node.uid", "k8s.cluster.name"}
k8sNodeGroupAttrKey = "k8s.node.name"
queryNamesForNodes = map[string][]string{
"cpu": {"A"},
"cpu_allocatable": {"B"},
"memory": {"C"},
"memory_allocatable": {"D"},
}
nodeQueryNames = []string{"A", "B", "C", "D", "E", "F"}
metricNamesForNodes = map[string]string{
"cpu": "k8s.node.cpu.usage",
"cpu_allocatable": "k8s.node.allocatable_cpu",
"memory": "k8s.node.memory.working_set",
"memory_allocatable": "k8s.node.allocatable_memory",
"node_condition": "k8s.node.condition_ready",
}
)
type NodesRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewNodesRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *NodesRepo {
return &NodesRepo{reader: reader, querierV2: querierV2}
}
func (n *NodesRepo) GetNodeAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForNodes
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := n.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeKeysResponse, nil
}
func (n *NodesRepo) DidSendNodeMetrics(ctx context.Context) (bool, error) {
namesStr := "'" + strings.Join(nodeMetricNamesToCheck, "','") + "'"
query := fmt.Sprintf(didSendNodeMetricsQuery,
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_TIMESERIES_v4_1DAY_TABLENAME, namesStr)
count, err := n.reader.GetCountOfThings(ctx, query)
if err != nil {
return false, err
}
return count > 0, nil
}
func (n *NodesRepo) GetNodeAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForNodes
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := n.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (p *NodesRepo) getMetadataAttributes(ctx context.Context, req model.NodeListRequest) (map[string]map[string]string, error) {
nodeAttrs := map[string]map[string]string{}
for _, key := range nodeAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForNodes,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := p.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
nodeUID := stringData[k8sNodeGroupAttrKey]
if _, ok := nodeAttrs[nodeUID]; !ok {
nodeAttrs[nodeUID] = map[string]string{}
}
for _, key := range req.GroupBy {
nodeAttrs[nodeUID][key.Key] = stringData[key.Key]
}
}
return nodeAttrs, nil
}
func (p *NodesRepo) getTopNodeGroups(ctx context.Context, orgID valuer.UUID, req model.NodeListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopNodes(req)
queryNames := queryNamesForNodes[req.OrderBy.ColumnName]
topNodeGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topNodeGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopNodeGroups",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, topNodeGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topNodeGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
max := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopNodeGroupsSeries := formattedResponse[0].Series[req.Offset:int(max)]
topNodeGroups := []map[string]string{}
for _, series := range paginatedTopNodeGroupsSeries {
topNodeGroups = append(topNodeGroups, series.Labels)
}
allNodeGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allNodeGroups = append(allNodeGroups, series.Labels)
}
return topNodeGroups, allNodeGroups, nil
}
func (p *NodesRepo) GetNodeList(ctx context.Context, orgID valuer.UUID, req model.NodeListRequest) (model.NodeListResponse, error) {
resp := model.NodeListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sNodeGroupAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := NodesTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
}
nodeAttrs, err := p.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topNodeGroups, allNodeGroups, err := p.getTopNodeGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topNodeGroup := range topNodeGroups {
for k, v := range topNodeGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetNodeList",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.NodeListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.NodeListRecord{
NodeCPUUsage: -1,
NodeCPUAllocatable: -1,
NodeMemoryUsage: -1,
NodeMemoryAllocatable: -1,
}
if nodeUID, ok := row.Data[k8sNodeGroupAttrKey].(string); ok {
record.NodeUID = nodeUID
}
if cpu, ok := row.Data["A"].(float64); ok {
record.NodeCPUUsage = cpu
}
if cpuAllocatable, ok := row.Data["B"].(float64); ok {
record.NodeCPUAllocatable = cpuAllocatable
}
if mem, ok := row.Data["C"].(float64); ok {
record.NodeMemoryUsage = mem
}
if memory, ok := row.Data["D"].(float64); ok {
record.NodeMemoryAllocatable = memory
}
if ready, ok := row.Data["E"].(float64); ok {
record.CountByCondition.Ready = int(ready)
}
if notReady, ok := row.Data["F"].(float64); ok {
record.CountByCondition.NotReady = int(notReady)
}
record.Meta = map[string]string{}
if _, ok := nodeAttrs[record.NodeUID]; ok && record.NodeUID != "" {
record.Meta = nodeAttrs[record.NodeUID]
}
for k, v := range row.Data {
if slices.Contains(nodeQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allNodeGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,186 +0,0 @@
package inframetrics
import v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
var NodesTableListQuery = v3.QueryRangeParamsV3{
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{
// node cpu utilization
"A": {
QueryName: "A",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForNodes["cpu"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sNodeGroupAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "A",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// node cpu allocatable
"B": {
QueryName: "B",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForNodes["cpu_allocatable"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sNodeGroupAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "B",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// node memory utilization
"C": {
QueryName: "C",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForNodes["memory"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sNodeGroupAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "C",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// node memory allocatable
"D": {
QueryName: "D",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForNodes["memory_allocatable"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sNodeGroupAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "D",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// node conditions - Ready
"E": {
QueryName: "E",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForNodes["node_condition"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "__value",
},
Operator: v3.FilterOperatorEqual,
Value: 1,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sNodeGroupAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "E",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// node conditions - NotReady
"F": {
QueryName: "F",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForNodes["node_condition"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "__value",
},
Operator: v3.FilterOperatorEqual,
Value: 0,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sNodeGroupAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "F",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
},
PanelType: v3.PanelTypeTable,
QueryType: v3.QueryTypeBuilder,
},
Version: "v4",
FormatForWeb: true,
}

View File

@@ -1,555 +0,0 @@
package inframetrics
import (
"context"
"fmt"
"math"
"sort"
"strings"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForPods = "k8s.pod.cpu.usage"
podAttrsToEnrich = []string{
"k8s.pod.uid",
"k8s.pod.name",
"k8s.namespace.name",
"k8s.node.name",
"k8s.deployment.name",
"k8s.statefulset.name",
"k8s.daemonset.name",
"k8s.job.name",
"k8s.cronjob.name",
"k8s.cluster.name",
}
k8sPodUIDAttrKey = "k8s.pod.uid"
queryNamesForPods = map[string][]string{
"cpu": {"A"},
"cpu_request": {"B", "A"},
"cpu_limit": {"C", "A"},
"memory": {"D"},
"memory_request": {"E", "D"},
"memory_limit": {"F", "D"},
"restarts": {"G", "A"},
"pod_phase": {"H", "I", "J", "K"},
}
podQueryNames = []string{"A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K"}
metricNamesForPods = map[string]string{
"cpu": "k8s.pod.cpu.usage",
"cpu_request": "k8s.pod.cpu_request_utilization",
"cpu_limit": "k8s.pod.cpu_limit_utilization",
"memory": "k8s.pod.memory.working_set",
"memory_request": "k8s.pod.memory_request_utilization",
"memory_limit": "k8s.pod.memory_limit_utilization",
"restarts": "k8s.container.restarts",
"pod_phase": "k8s.pod.phase",
}
)
type PodsRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewPodsRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *PodsRepo {
return &PodsRepo{reader: reader, querierV2: querierV2}
}
func (p *PodsRepo) GetPodAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
// TODO(srikanthccv): remove hardcoded metric name and support keys from any pod metric
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForPods
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := p.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (p *PodsRepo) GetPodAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForPods
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := p.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (p *PodsRepo) DidSendPodMetrics(ctx context.Context) (bool, error) {
namesStr := "'" + strings.Join(podMetricNamesToCheck, "','") + "'"
query := fmt.Sprintf(didSendPodMetricsQuery,
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_TIMESERIES_v4_1DAY_TABLENAME, namesStr)
count, err := p.reader.GetCountOfThings(ctx, query)
if err != nil {
return false, err
}
return count > 0, nil
}
func (p *PodsRepo) DidSendClusterMetrics(ctx context.Context) (bool, error) {
namesStr := "'" + strings.Join(clusterMetricNamesToCheck, "','") + "'"
query := fmt.Sprintf(didSendClusterMetricsQuery,
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_TIMESERIES_v4_1DAY_TABLENAME, namesStr)
count, err := p.reader.GetCountOfThings(ctx, query)
if err != nil {
return false, err
}
return count > 0, nil
}
func (p *PodsRepo) IsSendingOptionalPodMetrics(ctx context.Context) (bool, error) {
namesStr := "'" + strings.Join(optionalPodMetricNamesToCheck, "','") + "'"
query := fmt.Sprintf(isSendingOptionalPodMetricsQuery,
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_TIMESERIES_v4_1DAY_TABLENAME, namesStr)
count, err := p.reader.GetCountOfThings(ctx, query)
if err != nil {
return false, err
}
return count > 0, nil
}
func (p *PodsRepo) SendingRequiredMetadata(ctx context.Context) ([]model.PodOnboardingStatus, error) {
namesStr := "'" + strings.Join(podMetricNamesToCheck, "','") + "'"
query := fmt.Sprintf(isSendingRequiredMetadataQuery,
constants.SIGNOZ_METRIC_DBNAME, constants.SIGNOZ_TIMESERIES_V4_TABLENAME, namesStr)
result, err := p.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
statuses := []model.PodOnboardingStatus{}
// for each pod, check if we have all the required metadata
for _, row := range result {
status := model.PodOnboardingStatus{}
switch v := row.Data["k8s.cluster.name"].(type) {
case string:
status.HasClusterName = true
status.ClusterName = v
case *string:
status.HasClusterName = *v != ""
status.ClusterName = *v
}
switch v := row.Data["k8s.node.name"].(type) {
case string:
status.HasNodeName = true
status.NodeName = v
case *string:
status.HasNodeName = *v != ""
status.NodeName = *v
}
switch v := row.Data["k8s.namespace.name"].(type) {
case string:
status.HasNamespaceName = true
status.NamespaceName = v
case *string:
status.HasNamespaceName = *v != ""
status.NamespaceName = *v
}
switch v := row.Data["k8s.deployment.name"].(type) {
case string:
status.HasDeploymentName = true
case *string:
status.HasDeploymentName = *v != ""
}
switch v := row.Data["k8s.statefulset.name"].(type) {
case string:
status.HasStatefulsetName = true
case *string:
status.HasStatefulsetName = *v != ""
}
switch v := row.Data["k8s.daemonset.name"].(type) {
case string:
status.HasDaemonsetName = true
case *string:
status.HasDaemonsetName = *v != ""
}
switch v := row.Data["k8s.cronjob.name"].(type) {
case string:
status.HasCronjobName = true
case *string:
status.HasCronjobName = *v != ""
}
switch v := row.Data["k8s.job.name"].(type) {
case string:
status.HasJobName = true
case *string:
status.HasJobName = *v != ""
}
switch v := row.Data["k8s.pod.name"].(type) {
case string:
status.PodName = v
case *string:
status.PodName = *v
}
if !status.HasClusterName ||
!status.HasNodeName ||
!status.HasNamespaceName ||
(!status.HasDeploymentName && !status.HasStatefulsetName && !status.HasDaemonsetName && !status.HasCronjobName && !status.HasJobName) {
statuses = append(statuses, status)
}
}
return statuses, nil
}
func (p *PodsRepo) getMetadataAttributes(ctx context.Context, req model.PodListRequest) (map[string]map[string]string, error) {
podAttrs := map[string]map[string]string{}
for _, key := range podAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForPods,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := p.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
podName := stringData[k8sPodUIDAttrKey]
if _, ok := podAttrs[podName]; !ok {
podAttrs[podName] = map[string]string{}
}
for _, key := range req.GroupBy {
podAttrs[podName][key.Key] = stringData[key.Key]
}
}
return podAttrs, nil
}
func (p *PodsRepo) getTopPodGroups(ctx context.Context, orgID valuer.UUID, req model.PodListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopPods(req)
queryNames := queryNamesForPods[req.OrderBy.ColumnName]
topPodGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topPodGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopPodGroups",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, topPodGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topPodGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopPodGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topPodGroups := []map[string]string{}
for _, series := range paginatedTopPodGroupsSeries {
topPodGroups = append(topPodGroups, series.Labels)
}
allPodGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allPodGroups = append(allPodGroups, series.Labels)
}
return topPodGroups, allPodGroups, nil
}
func (p *PodsRepo) GetPodList(ctx context.Context, orgID valuer.UUID, req model.PodListRequest) (model.PodListResponse, error) {
resp := model.PodListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sPodUIDAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := PodsTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
}
podAttrs, err := p.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topPodGroups, allPodGroups, err := p.getTopPodGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topPodGroup := range topPodGroups {
for k, v := range topPodGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetPodList",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.PodListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.PodListRecord{
PodCPU: -1,
PodCPURequest: -1,
PodCPULimit: -1,
PodMemory: -1,
PodMemoryRequest: -1,
PodMemoryLimit: -1,
RestartCount: -1,
}
if podUID, ok := row.Data[k8sPodUIDAttrKey].(string); ok {
record.PodUID = podUID
}
if cpu, ok := row.Data["A"].(float64); ok {
record.PodCPU = cpu
}
if cpuRequest, ok := row.Data["B"].(float64); ok {
record.PodCPURequest = cpuRequest
}
if cpuLimit, ok := row.Data["C"].(float64); ok {
record.PodCPULimit = cpuLimit
}
if memory, ok := row.Data["D"].(float64); ok {
record.PodMemory = memory
}
if memoryRequest, ok := row.Data["E"].(float64); ok {
record.PodMemoryRequest = memoryRequest
}
if memoryLimit, ok := row.Data["F"].(float64); ok {
record.PodMemoryLimit = memoryLimit
}
if restarts, ok := row.Data["G"].(float64); ok {
record.RestartCount = int(restarts)
}
if pending, ok := row.Data["H"].(float64); ok {
record.CountByPhase.Pending = int(pending)
}
if running, ok := row.Data["I"].(float64); ok {
record.CountByPhase.Running = int(running)
}
if succeeded, ok := row.Data["J"].(float64); ok {
record.CountByPhase.Succeeded = int(succeeded)
}
if failed, ok := row.Data["K"].(float64); ok {
record.CountByPhase.Failed = int(failed)
}
record.Meta = map[string]string{}
if _, ok := podAttrs[record.PodUID]; ok && record.PodUID != "" {
record.Meta = podAttrs[record.PodUID]
}
for k, v := range row.Data {
if slices.Contains(podQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allPodGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,332 +0,0 @@
package inframetrics
import v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
var PodsTableListQuery = v3.QueryRangeParamsV3{
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{
// pod cpu utilization
"A": {
QueryName: "A",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["cpu"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "A",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// pod cpu request utilization
"B": {
QueryName: "B",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["cpu_request"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "B",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
// pod cpu limit utilization
"C": {
QueryName: "C",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["cpu_limit"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "C",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
// pod memory utilization
"D": {
QueryName: "D",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["memory"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "D",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// pod memory request utilization
"E": {
QueryName: "E",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["memory_request"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "E",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
// pod memory limit utilization
"F": {
QueryName: "F",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["memory_limit"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "F",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
"G": {
QueryName: "G",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["restarts"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "G",
ReduceTo: v3.ReduceToOperatorSum,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationMax,
Functions: []v3.Function{{Name: v3.FunctionNameRunningDiff}},
Disabled: false,
},
// pod phase pending
"H": {
QueryName: "H",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["pod_phase"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "__value",
},
Operator: v3.FilterOperatorEqual,
Value: 1,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "H",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationCount,
Disabled: false,
},
// pod phase running
"I": {
QueryName: "I",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["pod_phase"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "__value",
},
Operator: v3.FilterOperatorEqual,
Value: 2,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "I",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationCount,
Disabled: false,
},
// pod phase succeeded
"J": {
QueryName: "J",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["pod_phase"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "__value",
},
Operator: v3.FilterOperatorEqual,
Value: 3,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "J",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationCount,
Disabled: false,
},
// pod phase failed
"K": {
QueryName: "K",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForPods["pod_phase"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: "__value",
},
Operator: v3.FilterOperatorEqual,
Value: 4,
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPodUIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "K",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationCount,
Disabled: false,
},
},
PanelType: v3.PanelTypeTable,
QueryType: v3.QueryTypeBuilder,
},
Version: "v4",
FormatForWeb: true,
}

View File

@@ -1,73 +0,0 @@
package inframetrics
import v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
var ProcessesTableListQuery = v3.QueryRangeParamsV3{
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{
"A": {
QueryName: "A",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForProcesses["cpu"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: processPIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "A",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationRate,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: true,
},
"F1": {
QueryName: "F1",
Expression: "A",
Legend: "Process CPU Usage (%)",
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
},
"C": {
QueryName: "C",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForProcesses["memory"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{
{
Key: processPIDAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "C",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
},
PanelType: v3.PanelTypeTable,
QueryType: v3.QueryTypeBuilder,
},
Version: "v4",
FormatForWeb: true,
}

View File

@@ -1,342 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
queryNamesForTopProcesses = map[string][]string{
"cpu": {"A"},
"memory": {"C"},
}
processPIDAttrKey = "process.pid"
metricNamesForProcesses = map[string]string{
"cpu": "process.cpu.time",
"memory": "process.memory.usage",
}
metricToUseForProcessAttributes = "process.memory.usage"
processNameAttrKey = "process.executable.name"
processCMDAttrKey = "process.command"
processCMDLineAttrKey = "process.command_line"
)
type ProcessesRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewProcessesRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *ProcessesRepo {
return &ProcessesRepo{reader: reader, querierV2: querierV2}
}
func (p *ProcessesRepo) GetProcessAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
// TODO(srikanthccv): remove hardcoded metric name and support keys from any system metric
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = "process.memory.usage"
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := p.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (p *ProcessesRepo) GetProcessAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = "process.memory.usage"
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := p.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (p *ProcessesRepo) getMetadataAttributes(ctx context.Context,
req model.ProcessListRequest) (map[string]map[string]string, error) {
processAttrs := map[string]map[string]string{}
keysToAdd := []string{"process.pid", "process.executable.name", "process.command", "process.command_line"}
for _, key := range keysToAdd {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForProcessAttributes,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Cumulative,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := p.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
processID := stringData[processPIDAttrKey]
if _, ok := processAttrs[processID]; !ok {
processAttrs[processID] = map[string]string{}
}
for _, key := range req.GroupBy {
processAttrs[processID][key.Key] = stringData[key.Key]
}
}
return processAttrs, nil
}
func (p *ProcessesRepo) getTopProcessGroups(ctx context.Context, orgID valuer.UUID, req model.ProcessListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopProcesses(req)
queryNames := queryNamesForTopProcesses[req.OrderBy.ColumnName]
topProcessGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topProcessGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopProcessGroups",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, topProcessGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topProcessGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopProcessGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topProcessGroups := []map[string]string{}
for _, series := range paginatedTopProcessGroupsSeries {
topProcessGroups = append(topProcessGroups, series.Labels)
}
allProcessGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allProcessGroups = append(allProcessGroups, series.Labels)
}
return topProcessGroups, allProcessGroups, nil
}
func (p *ProcessesRepo) GetProcessList(ctx context.Context, orgID valuer.UUID, req model.ProcessListRequest) (model.ProcessListResponse, error) {
resp := model.ProcessListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
// default to cpu order by
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
// default to process pid group by
if len(req.GroupBy) == 0 {
req.GroupBy = []v3.AttributeKey{{Key: processPIDAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := ProcessesTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
}
processAttrs, err := p.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topProcessGroups, allProcessGroups, err := p.getTopProcessGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topProcessGroup := range topProcessGroups {
for k, v := range topProcessGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetProcessList",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.ProcessListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.ProcessListRecord{
ProcessCPU: -1,
ProcessMemory: -1,
}
pid, ok := row.Data[processPIDAttrKey].(string)
if ok {
record.ProcessID = pid
}
processCPU, ok := row.Data["F1"].(float64)
if ok {
record.ProcessCPU = processCPU
}
processMemory, ok := row.Data["C"].(float64)
if ok {
record.ProcessMemory = processMemory
}
record.Meta = processAttrs[record.ProcessID]
record.ProcessName = record.Meta[processNameAttrKey]
record.ProcessCMD = record.Meta[processCMDAttrKey]
record.ProcessCMDLine = record.Meta[processCMDLineAttrKey]
records = append(records, record)
}
}
resp.Total = len(allProcessGroups)
resp.Records = records
return resp, nil
}

View File

@@ -1,391 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForVolumes = "k8s.volume.available"
volumeAttrsToEnrich = []string{
"k8s.pod.uid",
"k8s.pod.name",
"k8s.namespace.name",
"k8s.node.name",
"k8s.statefulset.name",
"k8s.cluster.name",
"k8s.persistentvolumeclaim.name",
}
k8sPersistentVolumeClaimNameAttrKey = "k8s.persistentvolumeclaim.name"
queryNamesForVolumes = map[string][]string{
"available": {"A"},
"capacity": {"B", "A"},
"usage": {"F1", "B", "A"},
"inodes": {"C", "A"},
"inodes_free": {"D", "A"},
"inodes_used": {"E", "A"},
}
volumeQueryNames = []string{"A", "B", "C", "D", "E", "F1"}
metricNamesForVolumes = map[string]string{
"available": "k8s.volume.available",
"capacity": "k8s.volume.capacity",
"inodes": "k8s.volume.inodes",
"inodes_free": "k8s.volume.inodes.free",
"inodes_used": "k8s.volume.inodes.used",
}
)
type PvcsRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewPvcsRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *PvcsRepo {
return &PvcsRepo{reader: reader, querierV2: querierV2}
}
func (p *PvcsRepo) GetPvcAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForVolumes
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := p.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (p *PvcsRepo) GetPvcAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForVolumes
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := p.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (p *PvcsRepo) getMetadataAttributes(ctx context.Context, req model.VolumeListRequest) (map[string]map[string]string, error) {
volumeAttrs := map[string]map[string]string{}
for _, key := range volumeAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForVolumes,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := p.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
volumeName := stringData[k8sPersistentVolumeClaimNameAttrKey]
if _, ok := volumeAttrs[volumeName]; !ok {
volumeAttrs[volumeName] = map[string]string{}
}
for _, key := range req.GroupBy {
volumeAttrs[volumeName][key.Key] = stringData[key.Key]
}
}
return volumeAttrs, nil
}
func (p *PvcsRepo) getTopVolumeGroups(ctx context.Context, orgID valuer.UUID, req model.VolumeListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopVolumes(req)
queryNames := queryNamesForVolumes[req.OrderBy.ColumnName]
topVolumeGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topVolumeGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopVolumeGroups",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, topVolumeGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topVolumeGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopVolumeGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topVolumeGroups := []map[string]string{}
for _, series := range paginatedTopVolumeGroupsSeries {
topVolumeGroups = append(topVolumeGroups, series.Labels)
}
allVolumeGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allVolumeGroups = append(allVolumeGroups, series.Labels)
}
return topVolumeGroups, allVolumeGroups, nil
}
func (p *PvcsRepo) GetPvcList(ctx context.Context, orgID valuer.UUID, req model.VolumeListRequest) (model.VolumeListResponse, error) {
resp := model.VolumeListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "usage", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sPersistentVolumeClaimNameAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := PvcsTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
}
volumeAttrs, err := p.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topVolumeGroups, allVolumeGroups, err := p.getTopVolumeGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topVolumeGroup := range topVolumeGroups {
for k, v := range topVolumeGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetPvcList",
})
queryResponse, _, err := p.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.VolumeListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.VolumeListRecord{
VolumeUsage: -1,
VolumeAvailable: -1,
VolumeCapacity: -1,
VolumeInodes: -1,
VolumeInodesFree: -1,
VolumeInodesUsed: -1,
Meta: map[string]string{},
}
if volumeName, ok := row.Data[k8sPersistentVolumeClaimNameAttrKey].(string); ok {
record.PersistentVolumeClaimName = volumeName
}
if volumeAvailable, ok := row.Data["A"].(float64); ok {
record.VolumeAvailable = volumeAvailable
}
if volumeCapacity, ok := row.Data["B"].(float64); ok {
record.VolumeCapacity = volumeCapacity
}
if volumeInodes, ok := row.Data["C"].(float64); ok {
record.VolumeInodes = volumeInodes
}
if volumeInodesFree, ok := row.Data["D"].(float64); ok {
record.VolumeInodesFree = volumeInodesFree
}
if volumeInodesUsed, ok := row.Data["E"].(float64); ok {
record.VolumeInodesUsed = volumeInodesUsed
}
record.VolumeUsage = record.VolumeCapacity - record.VolumeAvailable
record.Meta = map[string]string{}
if _, ok := volumeAttrs[record.PersistentVolumeClaimName]; ok && record.PersistentVolumeClaimName != "" {
record.Meta = volumeAttrs[record.PersistentVolumeClaimName]
}
for k, v := range row.Data {
if slices.Contains(volumeQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allVolumeGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,204 +0,0 @@
package inframetrics
import v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
var PvcsTableListQuery = v3.QueryRangeParamsV3{
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{
// k8s.volume.available
"A": {
QueryName: "A",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForVolumes["available"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotEqual,
Value: "",
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "A",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// k8s.volume.capacity
"B": {
QueryName: "B",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForVolumes["capacity"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotEqual,
Value: "",
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "B",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
"F1": {
QueryName: "F1",
DataSource: v3.DataSourceMetrics,
Expression: "B - A",
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
ReduceTo: v3.ReduceToOperatorLast,
},
// k8s.volume.inodes
"C": {
QueryName: "C",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForVolumes["inodes"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotEqual,
Value: "",
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "C",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// k8s.volume.inodes_free
"D": {
QueryName: "D",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForVolumes["inodes_free"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotEqual,
Value: "",
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "D",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// k8s.volume.inodes_used
"E": {
QueryName: "E",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForVolumes["inodes_used"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{
{
Key: v3.AttributeKey{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
Operator: v3.FilterOperatorNotEqual,
Value: "",
},
},
},
GroupBy: []v3.AttributeKey{
{
Key: k8sPersistentVolumeClaimNameAttrKey,
DataType: v3.AttributeKeyDataTypeString,
Type: v3.AttributeKeyTypeResource,
},
},
Expression: "E",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
},
PanelType: v3.PanelTypeTable,
QueryType: v3.QueryTypeBuilder,
},
Version: "v4",
FormatForWeb: true,
}

View File

@@ -1,457 +0,0 @@
package inframetrics
import (
"context"
"math"
"sort"
"github.com/SigNoz/signoz/pkg/query-service/app/metrics/v4/helpers"
"github.com/SigNoz/signoz/pkg/query-service/common"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/postprocess"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"golang.org/x/exp/slices"
)
var (
metricToUseForStatefulSets = "k8s.pod.cpu.usage"
k8sStatefulSetNameAttrKey = "k8s.statefulset.name"
metricNamesForStatefulSets = map[string]string{
"desired_pods": "k8s.statefulset.desired_pods",
"available_pods": "k8s.statefulset.current_pods",
}
statefulSetAttrsToEnrich = []string{
"k8s.statefulset.name",
"k8s.namespace.name",
"k8s.cluster.name",
}
queryNamesForStatefulSets = map[string][]string{
"cpu": {"A"},
"cpu_request": {"B", "A"},
"cpu_limit": {"C", "A"},
"memory": {"D"},
"memory_request": {"E", "D"},
"memory_limit": {"F", "D"},
"restarts": {"G", "A"},
"desired_pods": {"H"},
"available_pods": {"I"},
}
builderQueriesForStatefulSets = map[string]*v3.BuilderQuery{
// desired pods
"H": {
QueryName: "H",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForStatefulSets["desired_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "H",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// available pods
"I": {
QueryName: "I",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForStatefulSets["available_pods"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "I",
ReduceTo: v3.ReduceToOperatorLast,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
}
statefulSetQueryNames = []string{"A", "B", "C", "D", "E", "F", "G", "H", "I"}
)
type StatefulSetsRepo struct {
reader interfaces.Reader
querierV2 interfaces.Querier
}
func NewStatefulSetsRepo(reader interfaces.Reader, querierV2 interfaces.Querier) *StatefulSetsRepo {
return &StatefulSetsRepo{reader: reader, querierV2: querierV2}
}
func (d *StatefulSetsRepo) GetStatefulSetAttributeKeys(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeKeyRequest) (*v3.FilterAttributeKeyResponse, error) {
// TODO(srikanthccv): remove hardcoded metric name and support keys from any pod metric
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForStatefulSets
if req.Limit == 0 {
req.Limit = 50
}
attributeKeysResponse, err := d.reader.GetMetricAttributeKeys(ctx, orgID, &req)
if err != nil {
return nil, err
}
// TODO(srikanthccv): only return resource attributes when we have a way to
// distinguish between resource attributes and other attributes.
filteredKeys := []v3.AttributeKey{}
for _, key := range attributeKeysResponse.AttributeKeys {
if slices.Contains(pointAttrsToIgnore, key.Key) {
continue
}
filteredKeys = append(filteredKeys, key)
}
return &v3.FilterAttributeKeyResponse{AttributeKeys: filteredKeys}, nil
}
func (d *StatefulSetsRepo) GetStatefulSetAttributeValues(ctx context.Context, orgID valuer.UUID, req v3.FilterAttributeValueRequest) (*v3.FilterAttributeValueResponse, error) {
req.DataSource = v3.DataSourceMetrics
req.AggregateAttribute = metricToUseForStatefulSets
if req.Limit == 0 {
req.Limit = 50
}
attributeValuesResponse, err := d.reader.GetMetricAttributeValues(ctx, orgID, &req)
if err != nil {
return nil, err
}
return attributeValuesResponse, nil
}
func (d *StatefulSetsRepo) getMetadataAttributes(ctx context.Context, req model.StatefulSetListRequest) (map[string]map[string]string, error) {
statefulSetAttrs := map[string]map[string]string{}
for _, key := range statefulSetAttrsToEnrich {
hasKey := false
for _, groupByKey := range req.GroupBy {
if groupByKey.Key == key {
hasKey = true
break
}
}
if !hasKey {
req.GroupBy = append(req.GroupBy, v3.AttributeKey{Key: key})
}
}
mq := v3.BuilderQuery{
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricToUseForStatefulSets,
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
GroupBy: req.GroupBy,
}
query, err := helpers.PrepareTimeseriesFilterQuery(req.Start, req.End, &mq)
if err != nil {
return nil, err
}
query = localQueryToDistributedQuery(query)
attrsListResponse, err := d.reader.GetListResultV3(ctx, query)
if err != nil {
return nil, err
}
for _, row := range attrsListResponse {
stringData := map[string]string{}
for key, value := range row.Data {
if str, ok := value.(string); ok {
stringData[key] = str
} else if strPtr, ok := value.(*string); ok {
stringData[key] = *strPtr
}
}
statefulSetName := stringData[k8sStatefulSetNameAttrKey]
if _, ok := statefulSetAttrs[statefulSetName]; !ok {
statefulSetAttrs[statefulSetName] = map[string]string{}
}
for _, key := range req.GroupBy {
statefulSetAttrs[statefulSetName][key.Key] = stringData[key.Key]
}
}
return statefulSetAttrs, nil
}
func (d *StatefulSetsRepo) getTopStatefulSetGroups(ctx context.Context, orgID valuer.UUID, req model.StatefulSetListRequest, q *v3.QueryRangeParamsV3) ([]map[string]string, []map[string]string, error) {
step, timeSeriesTableName, samplesTableName := getParamsForTopStatefulSets(req)
queryNames := queryNamesForStatefulSets[req.OrderBy.ColumnName]
topStatefulSetGroupsQueryRangeParams := &v3.QueryRangeParamsV3{
Start: req.Start,
End: req.End,
Step: step,
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{},
QueryType: v3.QueryTypeBuilder,
PanelType: v3.PanelTypeTable,
},
}
for _, queryName := range queryNames {
query := q.CompositeQuery.BuilderQueries[queryName].Clone()
query.StepInterval = step
query.MetricTableHints = &v3.MetricTableHints{
TimeSeriesTableName: timeSeriesTableName,
SamplesTableName: samplesTableName,
}
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
topStatefulSetGroupsQueryRangeParams.CompositeQuery.BuilderQueries[queryName] = query
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "getTopStatefulSetGroups",
})
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, topStatefulSetGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, topStatefulSetGroupsQueryRangeParams)
if err != nil {
return nil, nil, err
}
if len(formattedResponse) == 0 || len(formattedResponse[0].Series) == 0 {
return nil, nil, nil
}
if req.OrderBy.Order == v3.DirectionDesc {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value > formattedResponse[0].Series[j].Points[0].Value
})
} else {
sort.Slice(formattedResponse[0].Series, func(i, j int) bool {
return formattedResponse[0].Series[i].Points[0].Value < formattedResponse[0].Series[j].Points[0].Value
})
}
limit := math.Min(float64(req.Offset+req.Limit), float64(len(formattedResponse[0].Series)))
paginatedTopStatefulSetGroupsSeries := formattedResponse[0].Series[req.Offset:int(limit)]
topStatefulSetGroups := []map[string]string{}
for _, series := range paginatedTopStatefulSetGroupsSeries {
topStatefulSetGroups = append(topStatefulSetGroups, series.Labels)
}
allStatefulSetGroups := []map[string]string{}
for _, series := range formattedResponse[0].Series {
allStatefulSetGroups = append(allStatefulSetGroups, series.Labels)
}
return topStatefulSetGroups, allStatefulSetGroups, nil
}
func (d *StatefulSetsRepo) GetStatefulSetList(ctx context.Context, orgID valuer.UUID, req model.StatefulSetListRequest) (model.StatefulSetListResponse, error) {
resp := model.StatefulSetListResponse{}
if req.Limit == 0 {
req.Limit = 10
}
if req.OrderBy == nil {
req.OrderBy = &v3.OrderBy{ColumnName: "cpu", Order: v3.DirectionDesc}
}
if req.GroupBy == nil {
req.GroupBy = []v3.AttributeKey{{Key: k8sStatefulSetNameAttrKey}}
resp.Type = model.ResponseTypeList
} else {
resp.Type = model.ResponseTypeGroupedList
}
step := int64(math.Max(float64(common.MinAllowedStepInterval(req.Start, req.End)), 60))
query := WorkloadTableListQuery.Clone()
query.Start = req.Start
query.End = req.End
query.Step = step
// add additional queries for stateful sets
for _, statefulSetQuery := range builderQueriesForStatefulSets {
query.CompositeQuery.BuilderQueries[statefulSetQuery.QueryName] = statefulSetQuery.Clone()
}
for _, query := range query.CompositeQuery.BuilderQueries {
query.StepInterval = step
if req.Filters != nil && len(req.Filters.Items) > 0 {
if query.Filters == nil {
query.Filters = &v3.FilterSet{Operator: "AND", Items: []v3.FilterItem{}}
}
query.Filters.Items = append(query.Filters.Items, req.Filters.Items...)
}
query.GroupBy = req.GroupBy
// make sure we only get records for daemon sets
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: k8sStatefulSetNameAttrKey},
Operator: v3.FilterOperatorExists,
})
}
statefulSetAttrs, err := d.getMetadataAttributes(ctx, req)
if err != nil {
return resp, err
}
topStatefulSetGroups, allStatefulSetGroups, err := d.getTopStatefulSetGroups(ctx, orgID, req, query)
if err != nil {
return resp, err
}
groupFilters := map[string][]string{}
for _, topStatefulSetGroup := range topStatefulSetGroups {
for k, v := range topStatefulSetGroup {
groupFilters[k] = append(groupFilters[k], v)
}
}
for groupKey, groupValues := range groupFilters {
hasGroupFilter := false
if req.Filters != nil && len(req.Filters.Items) > 0 {
for _, filter := range req.Filters.Items {
if filter.Key.Key == groupKey {
hasGroupFilter = true
break
}
}
}
if !hasGroupFilter {
for _, query := range query.CompositeQuery.BuilderQueries {
query.Filters.Items = append(query.Filters.Items, v3.FilterItem{
Key: v3.AttributeKey{Key: groupKey},
Value: groupValues,
Operator: v3.FilterOperatorIn,
})
}
}
}
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "inframetrics",
instrumentationtypes.CodeFunctionName: "GetStatefulSetList",
})
queryResponse, _, err := d.querierV2.QueryRange(ctx, orgID, query)
if err != nil {
return resp, err
}
formattedResponse, err := postprocess.PostProcessResult(queryResponse, query)
if err != nil {
return resp, err
}
records := []model.StatefulSetListRecord{}
for _, result := range formattedResponse {
for _, row := range result.Table.Rows {
record := model.StatefulSetListRecord{
StatefulSetName: "",
CPUUsage: -1,
CPURequest: -1,
CPULimit: -1,
MemoryUsage: -1,
MemoryRequest: -1,
MemoryLimit: -1,
DesiredPods: -1,
AvailablePods: -1,
}
if statefulSetName, ok := row.Data[k8sStatefulSetNameAttrKey].(string); ok {
record.StatefulSetName = statefulSetName
}
if cpu, ok := row.Data["A"].(float64); ok {
record.CPUUsage = cpu
}
if cpuRequest, ok := row.Data["B"].(float64); ok {
record.CPURequest = cpuRequest
}
if cpuLimit, ok := row.Data["C"].(float64); ok {
record.CPULimit = cpuLimit
}
if memory, ok := row.Data["D"].(float64); ok {
record.MemoryUsage = memory
}
if memoryRequest, ok := row.Data["E"].(float64); ok {
record.MemoryRequest = memoryRequest
}
if memoryLimit, ok := row.Data["F"].(float64); ok {
record.MemoryLimit = memoryLimit
}
if restarts, ok := row.Data["G"].(float64); ok {
record.Restarts = int(restarts)
}
if desiredPods, ok := row.Data["H"].(float64); ok {
record.DesiredPods = int(desiredPods)
}
if availablePods, ok := row.Data["I"].(float64); ok {
record.AvailablePods = int(availablePods)
}
record.Meta = map[string]string{}
if _, ok := statefulSetAttrs[record.StatefulSetName]; ok && record.StatefulSetName != "" {
record.Meta = statefulSetAttrs[record.StatefulSetName]
}
for k, v := range row.Data {
if slices.Contains(statefulSetQueryNames, k) {
continue
}
if labelValue, ok := v.(string); ok {
record.Meta[k] = labelValue
}
}
records = append(records, record)
}
}
resp.Total = len(allStatefulSetGroups)
resp.Records = records
resp.SortBy(req.OrderBy)
return resp, nil
}

View File

@@ -1,166 +0,0 @@
package inframetrics
import v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
var (
metricNamesForWorkloads = map[string]string{
"cpu": "k8s.pod.cpu.usage",
"cpu_request": "k8s.pod.cpu_request_utilization",
"cpu_limit": "k8s.pod.cpu_limit_utilization",
"memory": "k8s.pod.memory.working_set",
"memory_request": "k8s.pod.memory_request_utilization",
"memory_limit": "k8s.pod.memory_limit_utilization",
"restarts": "k8s.container.restarts",
}
)
var WorkloadTableListQuery = v3.QueryRangeParamsV3{
CompositeQuery: &v3.CompositeQuery{
BuilderQueries: map[string]*v3.BuilderQuery{
// pod cpu utilization
"A": {
QueryName: "A",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForWorkloads["cpu"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "A",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// pod cpu request utilization
"B": {
QueryName: "B",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForWorkloads["cpu_request"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "B",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
// pod cpu limit utilization
"C": {
QueryName: "C",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForWorkloads["cpu_limit"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "C",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
// pod memory utilization
"D": {
QueryName: "D",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForWorkloads["memory"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "D",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationSum,
Disabled: false,
},
// pod memory request utilization
"E": {
QueryName: "E",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForWorkloads["memory_request"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "E",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
// pod memory limit utilization
"F": {
QueryName: "F",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForWorkloads["memory_limit"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "F",
ReduceTo: v3.ReduceToOperatorAvg,
TimeAggregation: v3.TimeAggregationAvg,
SpaceAggregation: v3.SpaceAggregationAvg,
Disabled: false,
},
"G": {
QueryName: "G",
DataSource: v3.DataSourceMetrics,
AggregateAttribute: v3.AttributeKey{
Key: metricNamesForWorkloads["restarts"],
DataType: v3.AttributeKeyDataTypeFloat64,
},
Temporality: v3.Unspecified,
Filters: &v3.FilterSet{
Operator: "AND",
Items: []v3.FilterItem{},
},
GroupBy: []v3.AttributeKey{},
Expression: "G",
ReduceTo: v3.ReduceToOperatorSum,
TimeAggregation: v3.TimeAggregationAnyLast,
SpaceAggregation: v3.SpaceAggregationMax,
Functions: []v3.Function{{Name: v3.FunctionNameRunningDiff}},
Disabled: false,
},
},
PanelType: v3.PanelTypeTable,
QueryType: v3.QueryTypeBuilder,
},
Version: "v4",
FormatForWeb: true,
}

View File

@@ -166,7 +166,6 @@ func (s *Server) createPublicServer(api *APIHandler, web web.Web) (*http.Server,
api.RegisterLogsRoutes(r, am)
api.RegisterIntegrationRoutes(r, am)
api.RegisterQueryRangeV3Routes(r, am)
api.RegisterInfraMetricsRoutes(r, am)
api.RegisterWebSocketPaths(r, am)
api.RegisterQueryRangeV4Routes(r, am)
api.RegisterMessagingQueuesRoutes(r, am)

View File

@@ -95,11 +95,6 @@ type Reader interface {
ReportQueryStartForProgressTracking(queryId string) (reportQueryFinished func(), apiErr *model.ApiError)
SubscribeToQueryProgress(queryId string) (<-chan model.QueryProgress, func(), *model.ApiError)
GetCountOfThings(ctx context.Context, query string) (uint64, error)
GetActiveHostsFromMetricMetadata(ctx context.Context, metricNames []string, hostNameAttr string, sinceUnixMilli int64) (map[string]bool, error)
GetMetricsExistenceAndEarliestTime(ctx context.Context, metricNames []string) (uint64, uint64, error)
//trace
GetTraceFields(ctx context.Context) (*model.GetFieldsResponse, *model.ApiError)
UpdateTraceField(ctx context.Context, field *model.UpdateField) *model.ApiError

View File

@@ -1,757 +0,0 @@
package model
import (
"sort"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
)
type (
ResponseType string
)
const (
ResponseTypeList ResponseType = "list"
ResponseTypeGroupedList ResponseType = "grouped_list"
)
type HostListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type HostListRecord struct {
HostName string `json:"hostName"`
Active bool `json:"active"`
OS string `json:"os"`
CPU float64 `json:"cpu"`
Memory float64 `json:"memory"`
Wait float64 `json:"wait"`
Load15 float64 `json:"load15"`
Meta map[string]string `json:"meta"`
}
type HostListResponse struct {
Type ResponseType `json:"type"`
Records []HostListRecord `json:"records"`
Total int `json:"total"`
SentAnyHostMetricsData bool `json:"sentAnyHostMetricsData"`
IsSendingK8SAgentMetrics bool `json:"isSendingK8SAgentMetrics"`
ClusterNames []string `json:"clusterNames"`
NodeNames []string `json:"nodeNames"`
EndTimeBeforeRetention bool `json:"endTimeBeforeRetention"`
}
func (r *HostListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPU > r.Records[j].CPU
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].Memory > r.Records[j].Memory
})
case "load15":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].Load15 > r.Records[j].Load15
})
case "wait":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].Wait > r.Records[j].Wait
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type ProcessListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type ProcessListResponse struct {
Type ResponseType `json:"type"`
Records []ProcessListRecord `json:"records"`
Total int `json:"total"`
}
type ProcessListRecord struct {
ProcessName string `json:"processName"`
ProcessID string `json:"processID"`
ProcessCMD string `json:"processCMD"`
ProcessCMDLine string `json:"processCMDLine"`
ProcessCPU float64 `json:"processCPU"`
ProcessMemory float64 `json:"processMemory"`
Meta map[string]string `json:"meta"`
}
type PodListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type PodListResponse struct {
Type ResponseType `json:"type"`
Records []PodListRecord `json:"records"`
Total int `json:"total"`
}
func (r *PodListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].PodCPU > r.Records[j].PodCPU
})
case "cpu_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].PodCPURequest > r.Records[j].PodCPURequest
})
case "cpu_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].PodCPULimit > r.Records[j].PodCPULimit
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].PodMemory > r.Records[j].PodMemory
})
case "memory_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].PodMemoryRequest > r.Records[j].PodMemoryRequest
})
case "memory_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].PodMemoryLimit > r.Records[j].PodMemoryLimit
})
case "restarts":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].RestartCount > r.Records[j].RestartCount
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type PodListRecord struct {
PodUID string `json:"podUID,omitempty"`
PodCPU float64 `json:"podCPU"`
PodCPURequest float64 `json:"podCPURequest"`
PodCPULimit float64 `json:"podCPULimit"`
PodMemory float64 `json:"podMemory"`
PodMemoryRequest float64 `json:"podMemoryRequest"`
PodMemoryLimit float64 `json:"podMemoryLimit"`
RestartCount int `json:"restartCount"`
Meta map[string]string `json:"meta"`
CountByPhase PodCountByPhase `json:"countByPhase"`
}
type PodCountByPhase struct {
Pending int `json:"pending"`
Running int `json:"running"`
Succeeded int `json:"succeeded"`
Failed int `json:"failed"`
Unknown int `json:"unknown"`
}
type NodeListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type NodeListResponse struct {
Type ResponseType `json:"type"`
Records []NodeListRecord `json:"records"`
Total int `json:"total"`
}
func (r *NodeListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].NodeCPUUsage > r.Records[j].NodeCPUUsage
})
case "cpu_allocatable":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].NodeCPUAllocatable > r.Records[j].NodeCPUAllocatable
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].NodeMemoryUsage > r.Records[j].NodeMemoryUsage
})
case "memory_allocatable":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].NodeMemoryAllocatable > r.Records[j].NodeMemoryAllocatable
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type NodeCountByCondition struct {
Ready int `json:"ready"`
NotReady int `json:"notReady"`
Unknown int `json:"unknown"`
}
type NodeListRecord struct {
NodeUID string `json:"nodeUID,omitempty"`
NodeCPUUsage float64 `json:"nodeCPUUsage"`
NodeCPUAllocatable float64 `json:"nodeCPUAllocatable"`
NodeMemoryUsage float64 `json:"nodeMemoryUsage"`
NodeMemoryAllocatable float64 `json:"nodeMemoryAllocatable"`
CountByCondition NodeCountByCondition `json:"countByCondition"`
Meta map[string]string `json:"meta"`
}
type NamespaceListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type NamespaceListResponse struct {
Type ResponseType `json:"type"`
Records []NamespaceListRecord `json:"records"`
Total int `json:"total"`
}
func (r *NamespaceListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPUUsage > r.Records[j].CPUUsage
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryUsage > r.Records[j].MemoryUsage
})
case "pod_phase":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CountByPhase.Pending > r.Records[j].CountByPhase.Pending
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type NamespaceListRecord struct {
NamespaceName string `json:"namespaceName"`
CPUUsage float64 `json:"cpuUsage"`
MemoryUsage float64 `json:"memoryUsage"`
CountByPhase PodCountByPhase `json:"countByPhase"`
Meta map[string]string `json:"meta"`
}
type ClusterListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type ClusterListResponse struct {
Type ResponseType `json:"type"`
Records []ClusterListRecord `json:"records"`
Total int `json:"total"`
}
func (r *ClusterListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPUUsage > r.Records[j].CPUUsage
})
case "cpu_allocatable":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPUAllocatable > r.Records[j].CPUAllocatable
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryUsage > r.Records[j].MemoryUsage
})
case "memory_allocatable":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryAllocatable > r.Records[j].MemoryAllocatable
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type ClusterListRecord struct {
ClusterUID string `json:"clusterUID"`
CPUUsage float64 `json:"cpuUsage"`
CPUAllocatable float64 `json:"cpuAllocatable"`
MemoryUsage float64 `json:"memoryUsage"`
MemoryAllocatable float64 `json:"memoryAllocatable"`
Meta map[string]string `json:"meta"`
}
type DeploymentListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type DeploymentListResponse struct {
Type ResponseType `json:"type"`
Records []DeploymentListRecord `json:"records"`
Total int `json:"total"`
}
func (r *DeploymentListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPUUsage > r.Records[j].CPUUsage
})
case "cpu_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPURequest > r.Records[j].CPURequest
})
case "cpu_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPULimit > r.Records[j].CPULimit
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryUsage > r.Records[j].MemoryUsage
})
case "memory_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryRequest > r.Records[j].MemoryRequest
})
case "memory_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryLimit > r.Records[j].MemoryLimit
})
case "desired_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].DesiredPods > r.Records[j].DesiredPods
})
case "available_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].AvailablePods > r.Records[j].AvailablePods
})
case "restarts":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].Restarts > r.Records[j].Restarts
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type DeploymentListRecord struct {
DeploymentName string `json:"deploymentName"`
CPUUsage float64 `json:"cpuUsage"`
MemoryUsage float64 `json:"memoryUsage"`
DesiredPods int `json:"desiredPods"`
AvailablePods int `json:"availablePods"`
CPURequest float64 `json:"cpuRequest"`
MemoryRequest float64 `json:"memoryRequest"`
CPULimit float64 `json:"cpuLimit"`
MemoryLimit float64 `json:"memoryLimit"`
Restarts int `json:"restarts"`
Meta map[string]string `json:"meta"`
}
type DaemonSetListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type DaemonSetListResponse struct {
Type ResponseType `json:"type"`
Records []DaemonSetListRecord `json:"records"`
Total int `json:"total"`
}
func (r *DaemonSetListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPUUsage > r.Records[j].CPUUsage
})
case "cpu_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPURequest > r.Records[j].CPURequest
})
case "cpu_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPULimit > r.Records[j].CPULimit
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryUsage > r.Records[j].MemoryUsage
})
case "memory_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryRequest > r.Records[j].MemoryRequest
})
case "memory_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryLimit > r.Records[j].MemoryLimit
})
case "restarts":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].Restarts > r.Records[j].Restarts
})
case "desired_nodes":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].DesiredNodes > r.Records[j].DesiredNodes
})
case "available_nodes":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].AvailableNodes > r.Records[j].AvailableNodes
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type DaemonSetListRecord struct {
DaemonSetName string `json:"daemonSetName"`
CPUUsage float64 `json:"cpuUsage"`
MemoryUsage float64 `json:"memoryUsage"`
CPURequest float64 `json:"cpuRequest"`
MemoryRequest float64 `json:"memoryRequest"`
CPULimit float64 `json:"cpuLimit"`
MemoryLimit float64 `json:"memoryLimit"`
Restarts int `json:"restarts"`
DesiredNodes int `json:"desiredNodes"`
AvailableNodes int `json:"availableNodes"`
Meta map[string]string `json:"meta"`
}
type StatefulSetListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type StatefulSetListResponse struct {
Type ResponseType `json:"type"`
Records []StatefulSetListRecord `json:"records"`
Total int `json:"total"`
}
func (r *StatefulSetListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPUUsage > r.Records[j].CPUUsage
})
case "cpu_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPURequest > r.Records[j].CPURequest
})
case "cpu_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPULimit > r.Records[j].CPULimit
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryUsage > r.Records[j].MemoryUsage
})
case "memory_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryRequest > r.Records[j].MemoryRequest
})
case "memory_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryLimit > r.Records[j].MemoryLimit
})
case "restarts":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].Restarts > r.Records[j].Restarts
})
case "desired_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].DesiredPods > r.Records[j].DesiredPods
})
case "available_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].AvailablePods > r.Records[j].AvailablePods
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type StatefulSetListRecord struct {
StatefulSetName string `json:"statefulSetName"`
CPUUsage float64 `json:"cpuUsage"`
MemoryUsage float64 `json:"memoryUsage"`
CPURequest float64 `json:"cpuRequest"`
MemoryRequest float64 `json:"memoryRequest"`
CPULimit float64 `json:"cpuLimit"`
MemoryLimit float64 `json:"memoryLimit"`
Restarts int `json:"restarts"`
DesiredPods int `json:"desiredPods"`
AvailablePods int `json:"availablePods"`
Meta map[string]string `json:"meta"`
}
type JobListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type JobListResponse struct {
Type ResponseType `json:"type"`
Records []JobListRecord `json:"records"`
Total int `json:"total"`
}
func (r *JobListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "cpu":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPUUsage > r.Records[j].CPUUsage
})
case "cpu_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPURequest > r.Records[j].CPURequest
})
case "cpu_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].CPULimit > r.Records[j].CPULimit
})
case "memory":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryUsage > r.Records[j].MemoryUsage
})
case "memory_request":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryRequest > r.Records[j].MemoryRequest
})
case "memory_limit":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].MemoryLimit > r.Records[j].MemoryLimit
})
case "restarts":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].Restarts > r.Records[j].Restarts
})
case "desired_successful_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].DesiredSuccessfulPods > r.Records[j].DesiredSuccessfulPods
})
case "active_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].ActivePods > r.Records[j].ActivePods
})
case "failed_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].FailedPods > r.Records[j].FailedPods
})
case "successful_pods":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].SuccessfulPods > r.Records[j].SuccessfulPods
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type JobListRecord struct {
JobName string `json:"jobName"`
CPUUsage float64 `json:"cpuUsage"`
MemoryUsage float64 `json:"memoryUsage"`
CPURequest float64 `json:"cpuRequest"`
MemoryRequest float64 `json:"memoryRequest"`
CPULimit float64 `json:"cpuLimit"`
MemoryLimit float64 `json:"memoryLimit"`
Restarts int `json:"restarts"`
DesiredSuccessfulPods int `json:"desiredSuccessfulPods"`
ActivePods int `json:"activePods"`
FailedPods int `json:"failedPods"`
SuccessfulPods int `json:"successfulPods"`
Meta map[string]string `json:"meta"`
}
type VolumeListRequest struct {
Start int64 `json:"start"` // epoch time in ms
End int64 `json:"end"` // epoch time in ms
Filters *v3.FilterSet `json:"filters"`
GroupBy []v3.AttributeKey `json:"groupBy"`
OrderBy *v3.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
type VolumeListResponse struct {
Type ResponseType `json:"type"`
Records []VolumeListRecord `json:"records"`
Total int `json:"total"`
}
func (r *VolumeListResponse) SortBy(orderBy *v3.OrderBy) {
switch orderBy.ColumnName {
case "available":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].VolumeAvailable > r.Records[j].VolumeAvailable
})
case "capacity":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].VolumeCapacity > r.Records[j].VolumeCapacity
})
case "usage":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].VolumeUsage > r.Records[j].VolumeUsage
})
case "inodes":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].VolumeInodes > r.Records[j].VolumeInodes
})
case "inodes_free":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].VolumeInodesFree > r.Records[j].VolumeInodesFree
})
case "inodes_used":
sort.Slice(r.Records, func(i, j int) bool {
return r.Records[i].VolumeInodesUsed > r.Records[j].VolumeInodesUsed
})
}
// the default is descending
if orderBy.Order == v3.DirectionAsc {
// reverse the list
for i, j := 0, len(r.Records)-1; i < j; i, j = i+1, j-1 {
r.Records[i], r.Records[j] = r.Records[j], r.Records[i]
}
}
}
type VolumeListRecord struct {
PersistentVolumeClaimName string `json:"persistentVolumeClaimName"`
VolumeAvailable float64 `json:"volumeAvailable"`
VolumeCapacity float64 `json:"volumeCapacity"`
VolumeInodes float64 `json:"volumeInodes"`
VolumeInodesFree float64 `json:"volumeInodesFree"`
VolumeInodesUsed float64 `json:"volumeInodesUsed"`
VolumeUsage float64 `json:"volumeUsage"`
Meta map[string]string `json:"meta"`
}
type PodOnboardingStatus struct {
ClusterName string `json:"clusterName"`
NodeName string `json:"nodeName"`
NamespaceName string `json:"namespaceName"`
PodName string `json:"podName"`
HasClusterName bool `json:"hasClusterName"`
HasNodeName bool `json:"hasNodeName"`
HasNamespaceName bool `json:"hasNamespaceName"`
HasDeploymentName bool `json:"hasDeploymentName"`
HasStatefulsetName bool `json:"hasStatefulsetName"`
HasDaemonsetName bool `json:"hasDaemonsetName"`
HasCronjobName bool `json:"hasCronjobName"`
HasJobName bool `json:"hasJobName"`
}
type OnboardingStatus struct {
DidSendPodMetrics bool `json:"didSendPodMetrics"`
DidSendNodeMetrics bool `json:"didSendNodeMetrics"`
DidSendClusterMetrics bool `json:"didSendClusterMetrics"`
IsSendingOptionalPodMetrics bool `json:"isSendingOptionalPodMetrics"`
IsSendingRequiredMetadata []PodOnboardingStatus `json:"isSendingRequiredMetadata"`
}

View File

@@ -53,12 +53,6 @@ var (
types.RoleEditor: SigNozEditorRoleName,
types.RoleViewer: SigNozViewerRoleName,
}
SigNozManagedRoleToExistingLegacyRole = map[string]types.Role{
SigNozAdminRoleName: types.RoleAdmin,
SigNozEditorRoleName: types.RoleEditor,
SigNozViewerRoleName: types.RoleViewer,
}
)
type Role struct {

View File

@@ -119,9 +119,6 @@ type UserRoleStore interface {
// get a single user role entry by org id and its own id
GetUserRoleByOrgIDAndID(ctx context.Context, orgID valuer.UUID, id valuer.UUID) (*UserRole, error)
// list all user_role entries for
ListUserRolesByOrgIDAndUserIDs(ctx context.Context, orgID valuer.UUID, userIDs []valuer.UUID) ([]*UserRole, error)
// delete user role entries by user id
DeleteUserRoles(ctx context.Context, userID valuer.UUID) error

View File

@@ -1,83 +0,0 @@
package types
import (
"encoding/json"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/uptrace/bun"
)
var (
ErrInviteAlreadyExists = errors.MustNewCode("invite_already_exists")
ErrInviteNotFound = errors.MustNewCode("invite_not_found")
)
type GettableInvite = Invite
type Invite struct {
bun.BaseModel `bun:"table:user_invite"`
Identifiable
TimeAuditable
Name string `bun:"name,type:text" json:"name"`
Email valuer.Email `bun:"email,type:text" json:"email"`
Token string `bun:"token,type:text" json:"token"`
Role Role `bun:"role,type:text" json:"role"`
OrgID valuer.UUID `bun:"org_id,type:text" json:"orgId"`
InviteLink string `bun:"-" json:"inviteLink"`
}
type PostableInvite struct {
Name string `json:"name"`
Email valuer.Email `json:"email"`
Role Role `json:"role"`
FrontendBaseUrl string `json:"frontendBaseUrl"`
}
type PostableBulkInviteRequest struct {
Invites []PostableInvite `json:"invites" required:"true" nullable:"false"`
}
func (request *PostableBulkInviteRequest) UnmarshalJSON(data []byte) error {
type Alias PostableBulkInviteRequest
var temp Alias
if err := json.Unmarshal(data, &temp); err != nil {
return err
}
// check for duplicate emails in the same request
seen := make(map[string]struct{}, len(temp.Invites))
for _, invite := range temp.Invites {
email := invite.Email.StringValue()
if _, exists := seen[email]; exists {
return errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "Duplicate email in request: %s", email)
}
seen[email] = struct{}{}
}
*request = PostableBulkInviteRequest(temp)
return nil
}
func NewInvite(name string, role Role, orgID valuer.UUID, email valuer.Email) (*Invite, error) {
invite := &Invite{
Identifiable: Identifiable{
ID: valuer.GenerateUUID(),
},
Name: name,
Email: email,
Token: valuer.GenerateUUID().String(),
Role: role,
OrgID: orgID,
TimeAuditable: TimeAuditable{
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
},
}
return invite, nil
}

View File

@@ -25,6 +25,7 @@ var (
ErrCodeUserStatusDeleted = errors.MustNewCode("user_status_deleted")
ErrCodeUserStatusPendingInvite = errors.MustNewCode("user_status_pending_invite")
ErrCodeUserStatusNotPendingInvite = errors.MustNewCode("user_status_not_pending_invite")
ErrCodeUserDeprecated = errors.MustNewCode("user_deprecated")
)
var (
@@ -46,19 +47,10 @@ type User struct {
TimeAuditable
}
type DeprecatedUser struct {
*User
Role Role `json:"role"`
}
type UpdatableUser struct {
DisplayName string `json:"displayName" required:"true"`
}
type PostableRole struct {
Name string `json:"name" required:"true"`
}
type PostableRegisterOrgAndAdmin struct {
Name string `json:"name"`
Email valuer.Email `json:"email"`
@@ -121,13 +113,6 @@ func NewRootUser(displayName string, email valuer.Email, orgID valuer.UUID) (*Us
}, nil
}
func NewDeprecatedUserFromUserAndRole(user *User, role Role) *DeprecatedUser {
return &DeprecatedUser{
user,
role,
}
}
// Update applies mutable fields from the input to the user. Immutable fields
// (email, is_root, org_id, id) are preserved. Only non-zero input fields are applied.
func (u *User) Update(displayName string) {

39
tests/e2e/conftest.py Normal file
View File

@@ -0,0 +1,39 @@
# TODO: remove this file once enable_ai_observability flag is removed or defaulted to True
import pytest
from testcontainers.core.container import Network
from fixtures import types
from fixtures.signoz import create_signoz
@pytest.fixture(name="signoz", scope="package")
def signoz_e2e( # pylint: disable=too-many-arguments,too-many-positional-arguments
network: Network,
zeus: types.TestContainerDocker,
gateway: types.TestContainerDocker,
sqlstore: types.TestContainerSQL,
clickhouse: types.TestContainerClickhouse,
request: pytest.FixtureRequest,
pytestconfig: pytest.Config,
) -> types.SigNoz:
"""
E2E-scoped SigNoz override. Enables the experimental AI/LLM Observability
module (disabled by default in pkg/flagger/registry.go) so its routes render
instead of redirecting to /home — required by the llm-o11y e2e specs. Scoped
to the e2e package via this conftest so normal integration tests keep the
stock feature set. Follows the same pattern as
tests/integration/tests/metricreduction/conftest.py.
"""
return create_signoz(
network=network,
zeus=zeus,
gateway=gateway,
sqlstore=sqlstore,
clickhouse=clickhouse,
request=request,
pytestconfig=pytestconfig,
cache_key="signoz_e2e",
env_overrides={
"SIGNOZ_FLAGGER_CONFIG_BOOLEAN_ENABLE__AI__OBSERVABILITY": True,
},
)

View File

@@ -0,0 +1,53 @@
import type { APIRequestContext, Page } from '@playwright/test';
export const ATTRIBUTE_MAPPING_PATH = '/ai-observability/attribute-mapping';
export const ATTRIBUTE_MAPPING_PAGE_TESTID =
'llm-observability-attribute-mapping-page';
/** Navigate to the Attribute Mapping page and wait for its root container. */
export async function gotoAttributeMapping(page: Page): Promise<void> {
await page.goto(ATTRIBUTE_MAPPING_PATH);
await page
.getByTestId(ATTRIBUTE_MAPPING_PAGE_TESTID)
.waitFor({ state: 'visible' });
}
interface SpanMapperGroup {
id: string;
name: string;
}
/** List span-mapper groups via API (`GET /api/v1/span_mapper_groups`). */
async function listSpanMapperGroups(
request: APIRequestContext,
token: string,
): Promise<SpanMapperGroup[]> {
const res = await request.get('/api/v1/span_mapper_groups', {
headers: { Authorization: `Bearer ${token}` },
});
if (!res.ok()) {
return [];
}
const body = (await res.json()) as { data?: { items?: SpanMapperGroup[] } };
return body.data?.items ?? [];
}
export async function deleteSpanMapperGroupsByName(
request: APIRequestContext,
token: string,
names: string[],
): Promise<void> {
const wanted = new Set(names);
const groups = await listSpanMapperGroups(request, token).catch(() => []);
await Promise.all(
groups
.filter((group) => wanted.has(group.name))
.map((group) =>
request
.delete(`/api/v1/span_mapper_groups/${group.id}`, {
headers: { Authorization: `Bearer ${token}` },
})
.catch(() => undefined),
),
);
}

View File

@@ -0,0 +1,52 @@
import type { APIRequestContext, Page } from '@playwright/test';
export const LLM_PRICING_PATH = '/ai-observability/configuration';
export const LLM_PRICING_PAGE_TESTID = 'llm-observability-model-pricing-page';
export async function gotoLlmPricing(page: Page): Promise<void> {
await page.goto(LLM_PRICING_PATH);
await page.getByTestId(LLM_PRICING_PAGE_TESTID).waitFor({ state: 'visible' });
}
interface PricingRule {
id: string;
modelName: string;
}
/** List pricing rules via API (`GET /api/v1/llm_pricing_rules`). */
async function listPricingRules(
request: APIRequestContext,
token: string,
): Promise<PricingRule[]> {
const res = await request.get('/api/v1/llm_pricing_rules', {
params: { offset: 0, limit: 100 },
headers: { Authorization: `Bearer ${token}` },
});
if (!res.ok()) {
return [];
}
const body = (await res.json()) as {
data?: { items?: PricingRule[] | null };
};
return body.data?.items ?? [];
}
export async function deletePricingRulesByModelName(
request: APIRequestContext,
token: string,
modelNames: string[],
): Promise<void> {
const wanted = new Set(modelNames);
const rules = await listPricingRules(request, token).catch(() => []);
await Promise.all(
rules
.filter((rule) => wanted.has(rule.modelName))
.map((rule) =>
request
.delete(`/api/v1/llm_pricing_rules/${rule.id}`, {
headers: { Authorization: `Bearer ${token}` },
})
.catch(() => undefined),
),
);
}

View File

@@ -0,0 +1,94 @@
import { expect, test } from '../../fixtures/auth';
import { newAdminContext } from '../../helpers/auth';
import {
deleteSpanMapperGroupsByName,
gotoAttributeMapping,
} from '../../helpers/llm-o11y/attribute-mapping';
import { authToken } from '../../helpers/common';
test.describe.configure({ mode: 'serial' });
const GROUP_NAME = 'e2e-attr-mapping-happy';
const TARGET_ATTR = 'gen_ai.content.prompt';
const SOURCE_ATTR = 'my_company.llm.input';
test.afterAll(async ({ browser }) => {
const ctx = await newAdminContext(browser);
const page = await ctx.newPage();
try {
const token = await authToken(page);
await deleteSpanMapperGroupsByName(ctx.request, token, [GROUP_NAME]);
} finally {
await ctx.close();
}
});
test.describe('LLM Observability — Attribute Mapping', () => {
test('basic flow: add a mapping, test it, then save it to the backend', async ({
authedPage: page,
}) => {
await gotoAttributeMapping(page);
await page.getByTestId('add-group-row').click();
const groupDrawer = page.getByTestId('group-form-drawer');
await expect(groupDrawer).toBeVisible();
await page.getByTestId('group-form-name').fill(GROUP_NAME);
await page.getByTestId('group-form-save').click();
await expect(groupDrawer).toBeHidden();
const groupHeader = page
.locator('[data-testid^="group-expand-"]')
.filter({ hasText: GROUP_NAME });
await expect(groupHeader).toBeVisible();
await groupHeader.click();
await page.locator('[data-testid^="add-mapper-"]').click();
const mapperDrawer = page.getByTestId('mapper-form-drawer');
await expect(mapperDrawer).toBeVisible();
await page.getByTestId('mapper-form-target').fill(TARGET_ATTR);
await page.getByTestId('mapper-form-source-0').fill(SOURCE_ATTR);
const mapperSave = page.getByTestId('mapper-form-save');
await expect(mapperSave).toBeEnabled();
await mapperSave.click();
await expect(mapperDrawer).toBeHidden();
await expect(page.getByTestId('unsaved-changes')).toBeVisible();
await page.getByRole('tab', { name: 'Test' }).click();
await expect(page.getByTestId('test-tab')).toBeVisible();
const testResponse = page.waitForResponse(
(r) =>
new URL(r.url()).pathname.endsWith('/span_mapper_groups/test') &&
r.request().method() === 'POST',
);
await page.getByTestId('run-test-button').click();
expect((await testResponse).ok()).toBeTruthy();
await expect(page.getByTestId('test-error')).toHaveCount(0);
await expect(page.getByTestId('test-results')).toBeVisible();
const createGroup = page.waitForResponse(
(r) =>
new URL(r.url()).pathname.endsWith('/span_mapper_groups') &&
r.request().method() === 'POST',
);
const createMapper = page.waitForResponse(
(r) =>
/\/span_mapper_groups\/[^/]+\/span_mappers$/.test(
new URL(r.url()).pathname,
) && r.request().method() === 'POST',
);
await page.getByTestId('save-changes-btn').click();
expect((await createGroup).ok()).toBeTruthy();
expect((await createMapper).ok()).toBeTruthy();
await expect(page.getByText('Attribute mapping changes saved')).toBeVisible();
await expect(page.getByTestId('unsaved-changes')).toHaveCount(0);
});
});

View File

@@ -0,0 +1,58 @@
import { expect, test } from '../../fixtures/auth';
import { newAdminContext } from '../../helpers/auth';
import { authToken } from '../../helpers/common';
import {
deletePricingRulesByModelName,
gotoLlmPricing,
} from '../../helpers/llm-o11y/llm-pricing';
test.describe.configure({ mode: 'serial' });
const MODEL_NAME = 'e2e-pricing-happy-model';
const INPUT_COST = '5.50';
const OUTPUT_COST = '17.00';
test.afterAll(async ({ browser }) => {
const ctx = await newAdminContext(browser);
const page = await ctx.newPage();
try {
const token = await authToken(page);
await deletePricingRulesByModelName(ctx.request, token, [MODEL_NAME]);
} finally {
await ctx.close();
}
});
test.describe('LLM Observability — Model Pricing', () => {
test('basic flow: add a user-override model cost and save it to the backend', async ({
authedPage: page,
}) => {
await gotoLlmPricing(page);
await page.getByTestId('add-model-cost-btn').click();
await expect(page.getByTestId('drawer-model-id-input')).toBeVisible();
await page.getByTestId('drawer-model-id-input').fill(MODEL_NAME);
await page.getByTestId('drawer-input-cost').fill(INPUT_COST);
await page.getByTestId('drawer-output-cost').fill(OUTPUT_COST);
const putResponse = page.waitForResponse(
(r) =>
new URL(r.url()).pathname.endsWith('/llm_pricing_rules') &&
r.request().method() === 'PUT',
);
const saveBtn = page.getByTestId('drawer-save-btn');
await expect(saveBtn).toBeEnabled();
await saveBtn.click();
expect((await putResponse).ok()).toBeTruthy();
await expect(page.getByTestId('drawer-model-id-input')).toHaveCount(0);
await expect(page.getByTestId('model-costs-table')).toBeVisible();
await expect(
page
.locator('[data-testid^="model-cell-name-"]')
.filter({ hasText: MODEL_NAME }),
).toBeVisible();
});
});

View File

@@ -583,15 +583,15 @@ def test_dashboard_authz_cleanup(
actor = find_user_by_email(signoz, admin_token, _ACTOR_EMAIL)
response = requests.get(
signoz.self.host_configs["8080"].get(f"/api/v2/users/{actor['id']}/roles"),
signoz.self.host_configs["8080"].get(f"/api/v2/users/{actor['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=5,
)
assert response.status_code == HTTPStatus.OK, response.text
actor_entry = next((role for role in response.json()["data"] if role["name"] == _ACTOR_ROLE_NAME), None)
actor_entry = next((ur for ur in response.json()["data"]["userRoles"] if ur["role"]["name"] == _ACTOR_ROLE_NAME), None)
if actor_entry is not None:
response = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/users/{actor['id']}/roles/{actor_entry['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/user_roles/{actor_entry['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=5,
)

View File

@@ -260,12 +260,12 @@ def test_role_fga_cleanup(
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
user = find_user_by_email(signoz, admin_token, _ACTOR_USER_EMAIL)
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}/roles"), headers={"Authorization": f"Bearer {admin_token}"}, timeout=5)
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}"), headers={"Authorization": f"Bearer {admin_token}"}, timeout=5)
assert resp.status_code == HTTPStatus.OK, resp.text
actor_entry = next((r for r in resp.json()["data"] if r["name"] == _ACTOR_ROLE_NAME), None)
actor_entry = next((ur for ur in resp.json()["data"]["userRoles"] if ur["role"]["name"] == _ACTOR_ROLE_NAME), None)
if actor_entry is not None:
resp = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}/roles/{actor_entry['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/user_roles/{actor_entry['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=5,
)

View File

@@ -319,12 +319,12 @@ def test_saved_view_fga_cleanup(
admin_token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
user = find_user_by_email(signoz, admin_token, _SAVED_VIEW_FGA_CUSTOM_USER_EMAIL)
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}/roles"), headers={"Authorization": f"Bearer {admin_token}"}, timeout=5)
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}"), headers={"Authorization": f"Bearer {admin_token}"}, timeout=5)
assert resp.status_code == HTTPStatus.OK, resp.text
custom_entry = next((r for r in resp.json()["data"] if r["name"] == _SAVED_VIEW_FGA_CUSTOM_ROLE_NAME), None)
custom_entry = next((ur for ur in resp.json()["data"]["userRoles"] if ur["role"]["name"] == _SAVED_VIEW_FGA_CUSTOM_ROLE_NAME), None)
if custom_entry is not None:
resp = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}/roles/{custom_entry['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/user_roles/{custom_entry['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=5,
)

View File

@@ -285,12 +285,12 @@ def test_delete_custom_role_cleanup(
role_id = find_role_by_name(signoz, admin_token, _SA_FGA_CUSTOM_ROLE_NAME)
user = find_user_by_email(signoz, admin_token, _SA_FGA_CUSTOM_USER_EMAIL)
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}/roles"), headers={"Authorization": f"Bearer {admin_token}"}, timeout=5)
resp = requests.get(signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}"), headers={"Authorization": f"Bearer {admin_token}"}, timeout=5)
assert resp.status_code == HTTPStatus.OK, resp.text
custom_entry = next((r for r in resp.json()["data"] if r["name"] == _SA_FGA_CUSTOM_ROLE_NAME), None)
custom_entry = next((ur for ur in resp.json()["data"]["userRoles"] if ur["role"]["name"] == _SA_FGA_CUSTOM_ROLE_NAME), None)
if custom_entry is not None:
resp = requests.delete(
signoz.self.host_configs["8080"].get(f"/api/v2/users/{user['id']}/roles/{custom_entry['id']}"),
signoz.self.host_configs["8080"].get(f"/api/v2/user_roles/{custom_entry['id']}"),
headers={"Authorization": f"Bearer {admin_token}"},
timeout=5,
)