Compare commits

...

19 Commits

Author SHA1 Message Date
Piyush Singariya
0224921203 fix: lint fix 2026-04-28 18:53:01 +05:30
Piyush Singariya
582db60acc fix: correct env var 2026-04-28 18:46:14 +05:30
Piyush Singariya
4017849fbe Merge branch 'main' into json-feature-flag 2026-04-28 18:17:33 +05:30
Piyush Singariya
82ea5400a8 Merge branch 'main' into json-feature-flag 2026-04-28 18:04:14 +05:30
Piyush Singariya
2a6c23f349 fix: unit tests 2026-04-28 18:03:40 +05:30
Pandey
738ae6f21b docs: add servers block to the openapi spec (#11129)
Some checks are pending
build-staging / prepare (push) Waiting to run
build-staging / js-build (push) Blocked by required conditions
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
Release Drafter / update_release_draft (push) Waiting to run
* docs: add servers block to the openapi spec

* docs: add info block
2026-04-28 12:32:35 +00:00
Piyush Singariya
ce5f7d38d3 chore: remove querybuilder helper 2026-04-28 18:00:04 +05:30
SagarRajput-7
be9dd7de0e feat(billing-page): added cancel subscription option in billing page (#11118)
* feat(billing-page): added cancel subscription option in billing page

* feat(billing-page): semantic token correction

* feat(billing-page): added test cases

* feat(billing-page): used css module

* feat(billing-page): added license condition and log event

* feat(billing-page): added test cases

* feat(billing-page): addressed design feedbacks
2026-04-28 11:50:19 +00:00
Piyush Singariya
be6aba7f47 Merge branch 'main' into json-feature-flag 2026-04-28 17:19:56 +05:30
Nikhil Mantri
ffd493617f feat(infra-monitoring): v2 pods list API (#10833)
* chore: baseline setup

* chore: endpoint detail update

* chore: added logic for hosts v3 api

* fix: bug fix

* chore: disk usage

* chore: added validate function

* chore: added some unit tests

* chore: return status as a string

* chore: yarn generate api

* chore: removed isSendingK8sAgentsMetricsCode

* chore: moved funcs

* chore: added validation on order by

* chore: added pods list logic

* chore: updated openapi yml

* chore: updated spec

* chore: pods api meta start time

* chore: nil pointer check

* chore: nil pointer dereference fix in req.Filter

* chore: added temporalities of metrics

* chore: added pods metrics temporality

* chore: unified composite key function

* chore: code improvements

* chore: added pods list api updates

* chore: hostStatusNone added for clarity that this field can be left empty as well in payload

* chore: yarn generate api

* chore: return errors from getMetadata and lint fix

* chore: return errors from getMetadata and lint fix

* chore: added hostName logic

* chore: modified getMetadata query

* chore: add type for response and files rearrange

* chore: warnings added passing from queryResponse warning to host lists response struct

* chore: added better metrics existence check

* chore: added a TODO remark

* chore: added required metrics check

* chore: distributed samples table to local table change for get metadata

* chore: frontend fix

* chore: endpoint correction

* chore: endpoint modification openapi

* chore: escape backtick to prevent sql injection

* chore: rearrage

* chore: improvements

* chore: validate order by to validate function

* chore: improved description

* chore: added TODOs and made filterByStatus a part of filter struct

* chore: ignore empty string hosts in get active hosts

* feat(infra-monitoring): v2 hosts list - return counts of active & inactive hosts for custom group by attributes (#10956)

* chore: add functionality for showing active and inactive counts in custom group by

* chore: bug fix

* chore: added subquery for active and total count

* chore: ignore empty string hosts in get active hosts

* fix: sinceUnixMilli for determining active hosts compute once per request

* chore: refactor code

* chore: rename HostsList -> ListHosts

* chore: rearrangement

* chore: inframonitoring types renaming

* chore: added types package

* chore: file structure further breakdown for clarity

* chore: comments correction

* chore: removed temporalities

* chore: pods code restructuring

* chore: comments resolve

* chore: added json tag required: true

* chore: removed pod metric temporalities

* chore: removed internal server error

* chore: added status unauthorized

* chore: remove a defensive nil map check, the function ensure non-nil map when err nil

* chore: cleanup and rename

* chore: make sort stable in case of tiebreaker by comparing composite group by keys

* chore: regen api client for inframonitoring

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: added required tags

* chore: added support for pod phase unknown

* chore: removed pods - order by phase

* chore: improved api description to document -1 as no data in numeric fields

* fix: rebase fixes

* feat(infra-monitoring): v2 pods list apis - phase counts when custom grouping (#11088)

* chore: added phase counts feature

* chore: added queries for pod phase counts in custom group by

* chore: added unknown phase count

* fix: isPodUIDInGroupBy in buildPodRecords

* chore: 3 cte --> 2 cte

* chore: pod phase with local table of time series as counts

* chore: comment correction

* chore: corrected comment

* chore: value column for samples table added

* chore: removed query G for phase counts

* chore: rename variable

* chore: added PodPhaseNum constants to types

* chore: updated comment

* chore: formatted file

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ashwin Bhatkal <ashwin96@gmail.com>
2026-04-28 10:19:26 +00:00
Pandey
3f95d35ad5 chore: add @therealpandey with @srikanthcvv (#11126)
#### Temporary

- add @therealpandey with @srikanthcvv
2026-04-28 10:01:39 +00:00
Piyush Singariya
9689408a99 Merge branch 'main' into json-feature-flag 2026-04-23 11:21:14 +05:30
Piyush Singariya
2d918c95e0 chore: rename field 2026-04-23 11:20:36 +05:30
Piyush Singariya
efb415960c fix: minor changes 2026-04-23 11:15:12 +05:30
Piyush Singariya
412b497bdf test: removed nil checks 2026-04-23 11:10:34 +05:30
Piyush Singariya
67d95e1d1e fix: flagger threaded into tests 2026-04-23 10:47:00 +05:30
Piyush Singariya
57cccba19f feat: flagger integration in flow 2026-04-22 20:35:52 +05:30
Piyush Singariya
c5c2525ff0 fix: still using global bool 2026-04-22 15:14:56 +05:30
Piyush Singariya
1e4f865a15 chore: add json enabled as feature flag for FE 2026-04-22 09:58:24 +05:30
78 changed files with 2285 additions and 287 deletions

56
.github/CODEOWNERS vendored
View File

@@ -52,49 +52,49 @@ go.mod @therealpandey
# Querier Owners
/pkg/querier/ @srikanthccv
/pkg/variables/ @srikanthccv
/pkg/types/querybuildertypes/ @srikanthccv
/pkg/types/telemetrytypes/ @srikanthccv
/pkg/querybuilder/ @srikanthccv
/pkg/telemetrylogs/ @srikanthccv
/pkg/telemetrymetadata/ @srikanthccv
/pkg/telemetrymetrics/ @srikanthccv
/pkg/telemetrytraces/ @srikanthccv
/pkg/querier/ @srikanthccv @therealpandey
/pkg/variables/ @srikanthccv @therealpandey
/pkg/types/querybuildertypes/ @srikanthccv @therealpandey
/pkg/types/telemetrytypes/ @srikanthccv @therealpandey
/pkg/querybuilder/ @srikanthccv @therealpandey
/pkg/telemetrylogs/ @srikanthccv @therealpandey
/pkg/telemetrymetadata/ @srikanthccv @therealpandey
/pkg/telemetrymetrics/ @srikanthccv @therealpandey
/pkg/telemetrytraces/ @srikanthccv @therealpandey
# Metrics
/pkg/types/metrictypes/ @srikanthccv
/pkg/types/metricsexplorertypes/ @srikanthccv
/pkg/modules/metricsexplorer/ @srikanthccv
/pkg/prometheus/ @srikanthccv
/pkg/types/metrictypes/ @srikanthccv @therealpandey
/pkg/types/metricsexplorertypes/ @srikanthccv @therealpandey
/pkg/modules/metricsexplorer/ @srikanthccv @therealpandey
/pkg/prometheus/ @srikanthccv @therealpandey
# APM
/pkg/types/servicetypes/ @srikanthccv
/pkg/types/apdextypes/ @srikanthccv
/pkg/modules/apdex/ @srikanthccv
/pkg/modules/services/ @srikanthccv
/pkg/types/servicetypes/ @srikanthccv @therealpandey
/pkg/types/apdextypes/ @srikanthccv @therealpandey
/pkg/modules/apdex/ @srikanthccv @therealpandey
/pkg/modules/services/ @srikanthccv @therealpandey
# Dashboard
/pkg/types/dashboardtypes/ @srikanthccv
/pkg/modules/dashboard/ @srikanthccv
/pkg/types/dashboardtypes/ @srikanthccv @therealpandey
/pkg/modules/dashboard/ @srikanthccv @therealpandey
# Rule/Alertmanager
/pkg/types/ruletypes/ @srikanthccv
/pkg/types/alertmanagertypes @srikanthccv
/pkg/alertmanager/ @srikanthccv
/pkg/ruler/ @srikanthccv
/pkg/modules/rulestatehistory/ @srikanthccv
/pkg/types/rulestatehistorytypes/ @srikanthccv
/pkg/types/ruletypes/ @srikanthccv @therealpandey
/pkg/types/alertmanagertypes @srikanthccv @therealpandey
/pkg/alertmanager/ @srikanthccv @therealpandey
/pkg/ruler/ @srikanthccv @therealpandey
/pkg/modules/rulestatehistory/ @srikanthccv @therealpandey
/pkg/types/rulestatehistorytypes/ @srikanthccv @therealpandey
# Correlation-adjacent
/pkg/contextlinks/ @srikanthccv
/pkg/types/parsertypes/ @srikanthccv
/pkg/queryparser/ @srikanthccv
/pkg/contextlinks/ @srikanthccv @therealpandey
/pkg/types/parsertypes/ @srikanthccv @therealpandey
/pkg/queryparser/ @srikanthccv @therealpandey
# AuthN / AuthZ Owners

View File

@@ -2474,6 +2474,97 @@ components:
- requiredMetricsCheck
- endTimeBeforeRetention
type: object
InframonitoringtypesPodPhase:
enum:
- pending
- running
- succeeded
- failed
- unknown
- ""
type: string
InframonitoringtypesPodRecord:
properties:
failedPodCount:
type: integer
meta:
additionalProperties: {}
nullable: true
type: object
pendingPodCount:
type: integer
podAge:
format: int64
type: integer
podCPU:
format: double
type: number
podCPULimit:
format: double
type: number
podCPURequest:
format: double
type: number
podMemory:
format: double
type: number
podMemoryLimit:
format: double
type: number
podMemoryRequest:
format: double
type: number
podPhase:
$ref: '#/components/schemas/InframonitoringtypesPodPhase'
podUID:
type: string
runningPodCount:
type: integer
succeededPodCount:
type: integer
unknownPodCount:
type: integer
required:
- podUID
- podCPU
- podCPURequest
- podCPULimit
- podMemory
- podMemoryRequest
- podMemoryLimit
- podPhase
- pendingPodCount
- runningPodCount
- succeededPodCount
- failedPodCount
- unknownPodCount
- podAge
- meta
type: object
InframonitoringtypesPods:
properties:
endTimeBeforeRetention:
type: boolean
records:
items:
$ref: '#/components/schemas/InframonitoringtypesPodRecord'
nullable: true
type: array
requiredMetricsCheck:
$ref: '#/components/schemas/InframonitoringtypesRequiredMetricsCheck'
total:
type: integer
type:
$ref: '#/components/schemas/InframonitoringtypesResponseType'
warning:
$ref: '#/components/schemas/Querybuildertypesv5QueryWarnData'
required:
- type
- records
- total
- requiredMetricsCheck
- endTimeBeforeRetention
type: object
InframonitoringtypesPostableHosts:
properties:
end:
@@ -2500,6 +2591,32 @@ components:
- end
- limit
type: object
InframonitoringtypesPostablePods:
properties:
end:
format: int64
type: integer
filter:
$ref: '#/components/schemas/Querybuildertypesv5Filter'
groupBy:
items:
$ref: '#/components/schemas/Querybuildertypesv5GroupByKey'
nullable: true
type: array
limit:
type: integer
offset:
type: integer
orderBy:
$ref: '#/components/schemas/Querybuildertypesv5OrderBy'
start:
format: int64
type: integer
required:
- start
- end
- limit
type: object
InframonitoringtypesRequiredMetricsCheck:
properties:
missingMetrics:
@@ -5179,7 +5296,12 @@ components:
scheme: bearer
type: http
info:
contact:
email: support@signoz.io
name: SigNoz Support
url: https://signoz.io
description: OpenTelemetry-Native Logs, Metrics and Traces in a single pane
termsOfService: https://signoz.io/terms-of-service/
title: SigNoz
version: ""
openapi: 3.0.3
@@ -10886,7 +11008,9 @@ paths:
five metrics, and pagination via offset/limit. The response type is ''list''
for the default host.name grouping or ''grouped_list'' for custom groupBy
keys. Also reports missing required metrics and whether the requested time
range falls before the data retention boundary.'
range falls before the data retention boundary. Numeric metric fields (cpu,
memory, wait, load15, diskUsage) return -1 as a sentinel when no data is available
for that field.'
operationId: ListHosts
requestBody:
content:
@@ -10940,6 +11064,79 @@ paths:
summary: List Hosts for Infra Monitoring
tags:
- inframonitoring
/api/v2/infra_monitoring/pods:
post:
deprecated: false
description: 'Returns a paginated list of Kubernetes pods with key metrics:
CPU usage, CPU request/limit utilization, memory working set, memory request/limit
utilization, current pod phase (pending/running/succeeded/failed/unknown),
and pod age (ms since start time). Each pod includes metadata attributes (namespace,
node, workload owner such as deployment/statefulset/daemonset/job/cronjob,
cluster). Supports filtering via a filter expression, custom groupBy to aggregate
pods by any attribute, ordering by any of the six metrics (cpu, cpu_request,
cpu_limit, memory, memory_request, memory_limit), and pagination via offset/limit.
The response type is ''list'' for the default k8s.pod.uid grouping (each row
is one pod with its current phase) or ''grouped_list'' for custom groupBy
keys (each row aggregates pods in the group with per-phase counts: pendingPodCount,
runningPodCount, succeededPodCount, failedPodCount, unknownPodCount derived
from each pod''s latest phase in the window). Also reports missing required
metrics and whether the requested time range falls before the data retention
boundary. Numeric metric fields (podCPU, podCPURequest, podCPULimit, podMemory,
podMemoryRequest, podMemoryLimit, podAge) return -1 as a sentinel when no
data is available for that field.'
operationId: ListPods
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/InframonitoringtypesPostablePods'
responses:
"200":
content:
application/json:
schema:
properties:
data:
$ref: '#/components/schemas/InframonitoringtypesPods'
status:
type: string
required:
- status
- data
type: object
description: OK
"400":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Bad Request
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- VIEWER
- tokenizer:
- VIEWER
summary: List Pods for Infra Monitoring
tags:
- inframonitoring
/api/v2/livez:
get:
deprecated: false
@@ -17320,3 +17517,16 @@ paths:
summary: Replace variables
tags:
- querier
servers:
- description: The fully qualified URL to the SigNoz APIServer.
url: https://{host}:{port}{base_path}
variables:
base_path:
default: /
description: The base path of the SigNoz APIServer
host:
default: localhost
description: The host of the SigNoz APIServer
port:
default: "8080"
description: The port of the SigNoz APIServer

View File

@@ -71,6 +71,15 @@ func (ah *APIHandler) getFeatureFlags(w http.ResponseWriter, r *http.Request) {
Route: "",
})
bodyJSONQuery := ah.Signoz.Flagger.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureBodyJSONQuery.String()),
Active: bodyJSONQuery,
Usage: 0,
UsageLimit: -1,
Route: "",
})
if constants.IsDotMetricsEnabled {
for idx, feature := range featureSet {
if feature.Name == licensetypes.DotMetricsEnabled {

View File

@@ -105,6 +105,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
signoz.SQLStore,
integrationsController.GetPipelinesForInstalledIntegrations,
reader,
signoz.Flagger,
)
if err != nil {
return nil, err

View File

@@ -105,7 +105,7 @@ function createMockLicense(
status: '',
updated_at: '0',
},
state: LicenseState.ACTIVE,
state: LicenseState.ACTIVATED,
status: LicenseStatus.VALID,
platform: LicensePlatform.CLOUD,
created_at: '0',
@@ -931,7 +931,7 @@ describe('PrivateRoute', () => {
isFetchingActiveLicense: false,
activeLicense: createMockLicense({
platform: LicensePlatform.CLOUD,
state: LicenseState.ACTIVE,
state: LicenseState.ACTIVATED,
}),
},
isCloudUser: true,
@@ -1522,7 +1522,7 @@ describe('PrivateRoute', () => {
isFetchingActiveLicense: false,
activeLicense: createMockLicense({
platform: LicensePlatform.CLOUD,
state: LicenseState.ACTIVE,
state: LicenseState.ACTIVATED,
}),
trialInfo: createMockTrialInfo({ workSpaceBlock: false }),
user: createMockUser({ role: USER_ROLES.ADMIN as ROLES }),

View File

@@ -13,7 +13,9 @@ import type {
import type {
InframonitoringtypesPostableHostsDTO,
InframonitoringtypesPostablePodsDTO,
ListHosts200,
ListPods200,
RenderErrorResponseDTO,
} from '../sigNoz.schemas';
@@ -21,7 +23,7 @@ import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType, BodyType } from '../../../generatedAPIInstance';
/**
* Returns a paginated list of hosts with key infrastructure metrics: CPU usage (%), memory usage (%), I/O wait (%), disk usage (%), and 15-minute load average. Each host includes its current status (active/inactive based on metrics reported in the last 10 minutes) and metadata attributes (e.g., os.type). Supports filtering via a filter expression, filtering by host status, custom groupBy to aggregate hosts by any attribute, ordering by any of the five metrics, and pagination via offset/limit. The response type is 'list' for the default host.name grouping or 'grouped_list' for custom groupBy keys. Also reports missing required metrics and whether the requested time range falls before the data retention boundary.
* Returns a paginated list of hosts with key infrastructure metrics: CPU usage (%), memory usage (%), I/O wait (%), disk usage (%), and 15-minute load average. Each host includes its current status (active/inactive based on metrics reported in the last 10 minutes) and metadata attributes (e.g., os.type). Supports filtering via a filter expression, filtering by host status, custom groupBy to aggregate hosts by any attribute, ordering by any of the five metrics, and pagination via offset/limit. The response type is 'list' for the default host.name grouping or 'grouped_list' for custom groupBy keys. Also reports missing required metrics and whether the requested time range falls before the data retention boundary. Numeric metric fields (cpu, memory, wait, load15, diskUsage) return -1 as a sentinel when no data is available for that field.
* @summary List Hosts for Infra Monitoring
*/
export const listHosts = (
@@ -104,3 +106,87 @@ export const useListHosts = <
return useMutation(mutationOptions);
};
/**
* Returns a paginated list of Kubernetes pods with key metrics: CPU usage, CPU request/limit utilization, memory working set, memory request/limit utilization, current pod phase (pending/running/succeeded/failed/unknown), and pod age (ms since start time). Each pod includes metadata attributes (namespace, node, workload owner such as deployment/statefulset/daemonset/job/cronjob, cluster). Supports filtering via a filter expression, custom groupBy to aggregate pods by any attribute, ordering by any of the six metrics (cpu, cpu_request, cpu_limit, memory, memory_request, memory_limit), and pagination via offset/limit. The response type is 'list' for the default k8s.pod.uid grouping (each row is one pod with its current phase) or 'grouped_list' for custom groupBy keys (each row aggregates pods in the group with per-phase counts: pendingPodCount, runningPodCount, succeededPodCount, failedPodCount, unknownPodCount derived from each pod's latest phase in the window). Also reports missing required metrics and whether the requested time range falls before the data retention boundary. Numeric metric fields (podCPU, podCPURequest, podCPULimit, podMemory, podMemoryRequest, podMemoryLimit, podAge) return -1 as a sentinel when no data is available for that field.
* @summary List Pods for Infra Monitoring
*/
export const listPods = (
inframonitoringtypesPostablePodsDTO: BodyType<InframonitoringtypesPostablePodsDTO>,
signal?: AbortSignal,
) => {
return GeneratedAPIInstance<ListPods200>({
url: `/api/v2/infra_monitoring/pods`,
method: 'POST',
headers: { 'Content-Type': 'application/json' },
data: inframonitoringtypesPostablePodsDTO,
signal,
});
};
export const getListPodsMutationOptions = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof listPods>>,
TError,
{ data: BodyType<InframonitoringtypesPostablePodsDTO> },
TContext
>;
}): UseMutationOptions<
Awaited<ReturnType<typeof listPods>>,
TError,
{ data: BodyType<InframonitoringtypesPostablePodsDTO> },
TContext
> => {
const mutationKey = ['listPods'];
const { mutation: mutationOptions } = options
? options.mutation &&
'mutationKey' in options.mutation &&
options.mutation.mutationKey
? options
: { ...options, mutation: { ...options.mutation, mutationKey } }
: { mutation: { mutationKey } };
const mutationFn: MutationFunction<
Awaited<ReturnType<typeof listPods>>,
{ data: BodyType<InframonitoringtypesPostablePodsDTO> }
> = (props) => {
const { data } = props ?? {};
return listPods(data);
};
return { mutationFn, ...mutationOptions };
};
export type ListPodsMutationResult = NonNullable<
Awaited<ReturnType<typeof listPods>>
>;
export type ListPodsMutationBody =
BodyType<InframonitoringtypesPostablePodsDTO>;
export type ListPodsMutationError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary List Pods for Infra Monitoring
*/
export const useListPods = <
TError = ErrorType<RenderErrorResponseDTO>,
TContext = unknown,
>(options?: {
mutation?: UseMutationOptions<
Awaited<ReturnType<typeof listPods>>,
TError,
{ data: BodyType<InframonitoringtypesPostablePodsDTO> },
TContext
>;
}): UseMutationResult<
Awaited<ReturnType<typeof listPods>>,
TError,
{ data: BodyType<InframonitoringtypesPostablePodsDTO> },
TContext
> => {
const mutationOptions = getListPodsMutationOptions(options);
return useMutation(mutationOptions);
};

View File

@@ -3243,6 +3243,108 @@ export interface InframonitoringtypesHostsDTO {
warning?: Querybuildertypesv5QueryWarnDataDTO;
}
export enum InframonitoringtypesPodPhaseDTO {
pending = 'pending',
running = 'running',
succeeded = 'succeeded',
failed = 'failed',
unknown = 'unknown',
'' = '',
}
/**
* @nullable
*/
export type InframonitoringtypesPodRecordDTOMeta = {
[key: string]: unknown;
} | null;
export interface InframonitoringtypesPodRecordDTO {
/**
* @type integer
*/
failedPodCount: number;
/**
* @type object
* @nullable true
*/
meta: InframonitoringtypesPodRecordDTOMeta;
/**
* @type integer
*/
pendingPodCount: number;
/**
* @type integer
* @format int64
*/
podAge: number;
/**
* @type number
* @format double
*/
podCPU: number;
/**
* @type number
* @format double
*/
podCPULimit: number;
/**
* @type number
* @format double
*/
podCPURequest: number;
/**
* @type number
* @format double
*/
podMemory: number;
/**
* @type number
* @format double
*/
podMemoryLimit: number;
/**
* @type number
* @format double
*/
podMemoryRequest: number;
podPhase: InframonitoringtypesPodPhaseDTO;
/**
* @type string
*/
podUID: string;
/**
* @type integer
*/
runningPodCount: number;
/**
* @type integer
*/
succeededPodCount: number;
/**
* @type integer
*/
unknownPodCount: number;
}
export interface InframonitoringtypesPodsDTO {
/**
* @type boolean
*/
endTimeBeforeRetention: boolean;
/**
* @type array
* @nullable true
*/
records: InframonitoringtypesPodRecordDTO[] | null;
requiredMetricsCheck: InframonitoringtypesRequiredMetricsCheckDTO;
/**
* @type integer
*/
total: number;
type: InframonitoringtypesResponseTypeDTO;
warning?: Querybuildertypesv5QueryWarnDataDTO;
}
export interface InframonitoringtypesPostableHostsDTO {
/**
* @type integer
@@ -3271,6 +3373,34 @@ export interface InframonitoringtypesPostableHostsDTO {
start: number;
}
export interface InframonitoringtypesPostablePodsDTO {
/**
* @type integer
* @format int64
*/
end: number;
filter?: Querybuildertypesv5FilterDTO;
/**
* @type array
* @nullable true
*/
groupBy?: Querybuildertypesv5GroupByKeyDTO[] | null;
/**
* @type integer
*/
limit: number;
/**
* @type integer
*/
offset?: number;
orderBy?: Querybuildertypesv5OrderByDTO;
/**
* @type integer
* @format int64
*/
start: number;
}
export interface InframonitoringtypesRequiredMetricsCheckDTO {
/**
* @type array
@@ -7350,6 +7480,14 @@ export type ListHosts200 = {
status: string;
};
export type ListPods200 = {
data: InframonitoringtypesPodsDTO;
/**
* @type string
*/
status: string;
};
export type Livez200 = {
data: FactoryResponseDTO;
/**

View File

@@ -4,7 +4,13 @@ import {
notOfTrailResponse,
trialConvertedToSubscriptionResponse,
} from 'mocks-server/__mockdata__/licenses';
import { act, render, screen } from 'tests/test-utils';
import { act, render, screen, getAppContextMock } from 'tests/test-utils';
import APIError from 'types/api/error';
import {
LicensePlatform,
LicenseResModel,
LicenseState,
} from 'types/api/licensesV3/getActive';
import { getFormattedDate } from 'utils/timeUtils';
import BillingContainer from './BillingContainer';
@@ -99,6 +105,10 @@ describe('BillingContainer', () => {
await expect(
screen.findByRole('link', { name: /here/i }),
).resolves.toBeInTheDocument();
await expect(
screen.findByText('Cancel Subscription', { selector: 'span' }),
).resolves.toBeInTheDocument();
});
it('OnTrail but trialConvertedToSubscription', async () => {
@@ -138,6 +148,85 @@ describe('BillingContainer', () => {
const dayRemainingInBillingPeriod =
await screen.findByText(/1 days_remaining/i);
expect(dayRemainingInBillingPeriod).toBeInTheDocument();
await expect(
screen.findByText('Cancel Subscription', { selector: 'span' }),
).resolves.toBeInTheDocument();
});
});
describe('CancelSubscriptionBanner visibility', () => {
const baseActiveLicense = getAppContextMock('ADMIN')
.activeLicense as LicenseResModel;
it('should render when license is ACTIVATED and platform is CLOUD', async () => {
render(<BillingContainer />);
await expect(
screen.findByText('Cancel Subscription', { selector: 'span' }),
).resolves.toBeInTheDocument();
});
it.each([
['EXPIRED', LicenseState.EXPIRED],
['TERMINATED', LicenseState.TERMINATED],
['CANCELLED', LicenseState.CANCELLED],
['EVALUATION_EXPIRED', LicenseState.EVALUATION_EXPIRED],
['DEFAULTED', LicenseState.DEFAULTED],
['ISSUED', LicenseState.ISSUED],
['EVALUATING', LicenseState.EVALUATING],
])('should not render when license state is %s', async (_, state) => {
render(
<BillingContainer />,
{},
{
appContextOverrides: {
activeLicense: { ...baseActiveLicense, state },
},
},
);
await screen.findByText('billing');
expect(
screen.queryByText('Cancel Subscription', { selector: 'span' }),
).not.toBeInTheDocument();
});
const makeAPIError = (statusCode: number): APIError =>
new APIError({
httpStatusCode: statusCode as any,
error: { code: 'error', message: 'error', url: '', errors: [] },
});
it.each([
[
'Self-Hosted platform',
{
activeLicense: {
...baseActiveLicense,
platform: LicensePlatform.SELF_HOSTED,
},
activeLicenseFetchError: null,
},
],
[
'Community Enterprise user (license API 404)',
{
activeLicense: null,
activeLicenseFetchError: makeAPIError(404),
},
],
[
'Community user (license API 501)',
{
activeLicense: null,
activeLicenseFetchError: makeAPIError(501),
},
],
])('should not render for %s', async (_, overrides) => {
render(<BillingContainer />, {}, { appContextOverrides: overrides });
await screen.findByText('billing');
expect(
screen.queryByText('Cancel Subscription', { selector: 'span' }),
).not.toBeInTheDocument();
});
});

View File

@@ -34,10 +34,12 @@ import { CheckoutSuccessPayloadProps } from 'types/api/billing/checkout';
import { getBaseUrl } from 'utils/basePath';
import { getFormattedDate, getRemainingDays } from 'utils/timeUtils';
import CancelSubscriptionBanner from './CancelSubscriptionBanner';
import { BillingUsageGraph } from './BillingUsageGraph/BillingUsageGraph';
import { prepareCsvData } from './BillingUsageGraph/utils';
import './BillingContainer.styles.scss';
import { LicenseState } from 'types/api/licensesV3/getActive';
interface DataType {
key: string;
@@ -317,7 +319,7 @@ export default function BillingContainer(): JSX.Element {
const handleBilling = useCallback(async () => {
if (!trialInfo?.trialConvertedToSubscription) {
logEvent('Billing : Upgrade Plan', {
void logEvent('Billing : Upgrade Plan', {
user: pick(user, ['email', 'userId', 'name']),
org,
});
@@ -326,7 +328,7 @@ export default function BillingContainer(): JSX.Element {
url: getBaseUrl(),
});
} else {
logEvent('Billing : Manage Billing', {
void logEvent('Billing : Manage Billing', {
user: pick(user, ['email', 'userId', 'name']),
org,
});
@@ -535,6 +537,10 @@ export default function BillingContainer(): JSX.Element {
{(isLoading || isFetchingBillingData) && renderTableSkeleton()}
</div>
{isCloudUserVal && activeLicense?.state === LicenseState.ACTIVATED && (
<CancelSubscriptionBanner />
)}
{!trialInfo?.trialConvertedToSubscription && (
<div className="upgrade-plan-benefits">
<Row

View File

@@ -0,0 +1,35 @@
.banner {
display: flex;
justify-content: space-between;
align-items: center;
padding: var(--padding-4);
border-radius: 4px;
border: 1px solid var(--callout-error-border);
background-color: var(--callout-error-background);
margin: var(--spacing-4) 0 var(--spacing-12);
}
.info {
display: flex;
flex-direction: column;
gap: var(--spacing-2);
}
.title {
font-size: var(--font-size-sm);
font-weight: var(--font-weight-medium);
color: var(--callout-error-title);
}
.subtitle {
font-size: var(--paragraph-base-400-font-size);
font-weight: var(--paragraph-base-400-font-weight);
line-height: var(--paragraph-base-400-line-height);
color: var(--callout-error-icon);
}
.dialogBody {
font-size: var(--font-size-sm);
line-height: var(--line-height-relaxed);
color: var(--l2-foreground);
}

View File

@@ -0,0 +1,68 @@
import { render, screen, userEvent } from 'tests/test-utils';
import CancelSubscriptionBanner from './CancelSubscriptionBanner';
jest.mock('utils/basePath', () => ({
getBasePath: (): string => '/',
withBasePath: (path: string): string => path,
getAbsoluteUrl: (path: string): string => `https://test.signoz.io${path}`,
getBaseUrl: (): string => 'https://test.signoz.io',
}));
describe('CancelSubscriptionBanner', () => {
it('renders banner with title and subtitle', () => {
render(<CancelSubscriptionBanner />);
expect(
screen.getByText('Cancel Subscription', { selector: 'span' }),
).toBeInTheDocument();
expect(
screen.getByText('Cancel your SigNoz subscription.'),
).toBeInTheDocument();
});
it('opens dialog with correct content when Cancel Subscription is clicked', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
render(<CancelSubscriptionBanner />);
await user.click(
screen.getByRole('button', { name: /cancel subscription/i }),
);
expect(screen.getByRole('dialog')).toBeInTheDocument();
expect(
screen.getByText(/reach out to our support team/i),
).toBeInTheDocument();
expect(
screen.getByRole('button', { name: /keep subscription/i }),
).toBeInTheDocument();
expect(
screen.getByRole('button', { name: /contact support/i }),
).toBeInTheDocument();
});
it('sends mailto to cloud-support with correct subject on Contact Support', async () => {
const realCreateElement = document.createElement.bind(document);
const mockClick = jest.fn();
const mockAnchor = { href: '', click: mockClick };
jest.spyOn(document, 'createElement').mockImplementation((tag: string) => {
if (tag === 'a') {
return mockAnchor as unknown as HTMLAnchorElement;
}
return realCreateElement(tag);
});
const user = userEvent.setup({ pointerEventsCheck: 0 });
render(<CancelSubscriptionBanner />);
await user.click(
screen.getByRole('button', { name: /cancel subscription/i }),
);
await user.click(screen.getByRole('button', { name: /contact support/i }));
expect(mockAnchor.href).toContain('mailto:cloud-support@signoz.io');
expect(mockAnchor.href).toContain('Cancel%20My%20SigNoz%20Subscription');
expect(mockClick).toHaveBeenCalledTimes(1);
jest.restoreAllMocks();
});
});

View File

@@ -0,0 +1,106 @@
import { useState } from 'react';
import { X } from '@signozhq/icons';
import { Button, DialogWrapper } from '@signozhq/ui';
import logEvent from 'api/common/logEvent';
import { pick } from 'lodash-es';
import { useAppContext } from 'providers/App/App';
import { getBaseUrl } from 'utils/basePath';
import styles from './CancelSubscriptionBanner.module.scss';
function CancelSubscriptionBanner(): JSX.Element {
const [open, setOpen] = useState(false);
const { user, org } = useAppContext();
const handleOpenCancelDialog = (): void => {
void logEvent('Billing : Cancel Subscription Clicked', {
user: pick(user, ['email', 'displayName', 'role', 'organization']),
role: user?.role,
});
setOpen(true);
};
const handleContactSupport = (): void => {
void logEvent('Billing : Cancel Subscription Confirmed', {
user: pick(user, ['email', 'displayName', 'role', 'organization']),
role: user?.role,
});
const subject = encodeURIComponent('Cancel My SigNoz Subscription');
const orgName = org?.[0]?.displayName ?? '';
const body = encodeURIComponent(
[
'Hi SigNoz Team,',
'',
'I would like to cancel my SigNoz Cloud subscription.',
'Please find my account details below.',
'',
'Account Details:',
` • SigNoz URL: ${getBaseUrl()}`,
...(orgName ? [` • Organization: ${orgName}`] : []),
` • Account Email: ${user?.email ?? ''}`,
'',
'Reason for Cancellation:',
'[Please share the reason for cancellation]',
'',
'Additional feedback (optional):',
'[Any other feedback]',
'',
'Regards,',
'[user name or team name]',
].join('\n'),
);
const link = document.createElement('a');
link.href = `mailto:cloud-support@signoz.io?subject=${subject}&body=${body}`;
link.click();
setOpen(false);
};
const footer = (
<>
<Button
variant="solid"
color="secondary"
onClick={(): void => setOpen(false)}
>
Keep Subscription
</Button>
<Button variant="solid" color="destructive" onClick={handleContactSupport}>
Contact Support
</Button>
</>
);
return (
<>
<div className={styles.banner}>
<div className={styles.info}>
<span className={styles.title}>Cancel Subscription</span>
<span className={styles.subtitle}>Cancel your SigNoz subscription.</span>
</div>
<Button
variant="solid"
color="destructive"
prefix={<X size={12} />}
onClick={handleOpenCancelDialog}
>
Cancel Subscription
</Button>
</div>
<DialogWrapper
open={open}
onOpenChange={setOpen}
title="Cancel your subscription"
width="narrow"
showCloseButton={false}
footer={footer}
>
<p className={styles.dialogBody}>
To cancel your SigNoz subscription, please reach out to our support team.
We&apos;ll be happy to assist you.
</p>
</DialogWrapper>
</>
);
}
export default CancelSubscriptionBanner;

View File

@@ -139,8 +139,8 @@ function ChartPreview({
if (startTime && endTime && startTime !== endTime) {
dispatch(
UpdateTimeInterval('custom', [
parseInt(getTimeString(startTime), 10),
parseInt(getTimeString(endTime), 10),
Number.parseInt(getTimeString(startTime), 10),
Number.parseInt(getTimeString(endTime), 10),
]),
);
}

View File

@@ -370,7 +370,7 @@ function FormAlertRules({
// onQueryCategoryChange handles changes to query category
// in state as well as sets additional defaults
const onQueryCategoryChange = (val: EQueryType): void => {
const element = document.getElementById('top');
const element = document.querySelector('#top');
if (element) {
element.scrollIntoView({ behavior: 'smooth' });
}

View File

@@ -279,7 +279,7 @@ function Explorer(): JSX.Element {
[],
);
const [warning, setWarning] = useState<Warning | undefined>(undefined);
const [warning, setWarning] = useState<Warning | undefined>();
const oneChartPerQueryDisabledTooltip = useMemo(() => {
if (splitedQueries.length <= 1) {
@@ -291,7 +291,7 @@ function Explorer(): JSX.Element {
if (disableOneChartPerQuery) {
return 'One chart per query cannot be disabled for multiple queries with different units.';
}
return undefined;
return;
}, [disableOneChartPerQuery, splitedQueries.length, units.length]);
// Show the y axis unit selector if -

View File

@@ -217,7 +217,7 @@ function Inspect({
);
}
if (!inspectMetricsTimeSeries.length) {
if (inspectMetricsTimeSeries.length === 0) {
return renderFallback(
'inspect-metrics-empty',
<Empty description="No time series found for this metric to inspect." />,

View File

@@ -254,10 +254,10 @@ export function useInspectMetrics(
const valuesMap = new Map<number, number>();
series.values.forEach(({ timestamp, value }) => {
valuesMap.set(timestamp, parseFloat(value));
valuesMap.set(timestamp, Number.parseFloat(value));
});
return timestamps.map((timestamp) => valuesMap.get(timestamp) ?? NaN);
return timestamps.map((timestamp) => valuesMap.get(timestamp) ?? Number.NaN);
});
const rawData = [timestamps, ...timeseriesArray];
@@ -271,7 +271,7 @@ export function useInspectMetrics(
labels.add(label);
});
});
return Array.from(labels);
return [...labels];
}, [inspectMetricsData]);
const reset = useCallback(() => {

View File

@@ -115,8 +115,10 @@ export const useGetQueryRange: UseGetQueryRange = (
const updatedQuery = updateBarStepInterval(
requestData.query,
requestData.start ? requestData.start * 1e3 : parseInt(start, 10) * 1e3,
requestData.end ? requestData.end * 1e3 : parseInt(end, 10) * 1e3,
requestData.start
? requestData.start * 1e3
: Number.parseInt(start, 10) * 1e3,
requestData.end ? requestData.end * 1e3 : Number.parseInt(end, 10) * 1e3,
);
return {

View File

@@ -98,7 +98,7 @@ function LogsExplorer(): JSX.Element {
setIsLoadingQueries(false);
}, [queryClient]);
const [warning, setWarning] = useState<Warning | undefined>(undefined);
const [warning, setWarning] = useState<Warning | undefined>();
const handleChangeSelectedView = useCallback(
(view: ExplorerViews, querySearchParameters?: ICurrentQueryData): void => {

View File

@@ -101,7 +101,7 @@ function TracesExplorer(): JSX.Element {
getExplorerViewFromUrl(searchParams, panelTypesFromUrl),
);
const [warning, setWarning] = useState<Warning | undefined>(undefined);
const [warning, setWarning] = useState<Warning | undefined>();
const [isOpen, setOpen] = useState<boolean>(true);
const defaultQuery = useMemo(

View File

@@ -119,7 +119,7 @@ export function getAppContextMock(
status: '',
updated_at: '0',
},
state: LicenseState.ACTIVE,
state: LicenseState.ACTIVATED,
status: LicenseStatus.VALID,
platform: LicensePlatform.CLOUD,
created_at: '0',

View File

@@ -11,7 +11,7 @@ export enum LicenseStatus {
export enum LicenseState {
DEFAULTED = 'DEFAULTED',
ACTIVE = 'ACTIVE',
ACTIVATED = 'ACTIVATED',
EXPIRED = 'EXPIRED',
ISSUED = 'ISSUED',
EVALUATING = 'EVALUATING',

View File

@@ -16,7 +16,7 @@ func (provider *provider) addInfraMonitoringRoutes(router *mux.Router) error {
ID: "ListHosts",
Tags: []string{"inframonitoring"},
Summary: "List Hosts for Infra Monitoring",
Description: "Returns a paginated list of hosts with key infrastructure metrics: CPU usage (%), memory usage (%), I/O wait (%), disk usage (%), and 15-minute load average. Each host includes its current status (active/inactive based on metrics reported in the last 10 minutes) and metadata attributes (e.g., os.type). Supports filtering via a filter expression, filtering by host status, custom groupBy to aggregate hosts by any attribute, ordering by any of the five metrics, and pagination via offset/limit. The response type is 'list' for the default host.name grouping or 'grouped_list' for custom groupBy keys. Also reports missing required metrics and whether the requested time range falls before the data retention boundary.",
Description: "Returns a paginated list of hosts with key infrastructure metrics: CPU usage (%), memory usage (%), I/O wait (%), disk usage (%), and 15-minute load average. Each host includes its current status (active/inactive based on metrics reported in the last 10 minutes) and metadata attributes (e.g., os.type). Supports filtering via a filter expression, filtering by host status, custom groupBy to aggregate hosts by any attribute, ordering by any of the five metrics, and pagination via offset/limit. The response type is 'list' for the default host.name grouping or 'grouped_list' for custom groupBy keys. Also reports missing required metrics and whether the requested time range falls before the data retention boundary. Numeric metric fields (cpu, memory, wait, load15, diskUsage) return -1 as a sentinel when no data is available for that field.",
Request: new(inframonitoringtypes.PostableHosts),
RequestContentType: "application/json",
Response: new(inframonitoringtypes.Hosts),
@@ -29,5 +29,24 @@ func (provider *provider) addInfraMonitoringRoutes(router *mux.Router) error {
return err
}
if err := router.Handle("/api/v2/infra_monitoring/pods", handler.New(
provider.authZ.ViewAccess(provider.infraMonitoringHandler.ListPods),
handler.OpenAPIDef{
ID: "ListPods",
Tags: []string{"inframonitoring"},
Summary: "List Pods for Infra Monitoring",
Description: "Returns a paginated list of Kubernetes pods with key metrics: CPU usage, CPU request/limit utilization, memory working set, memory request/limit utilization, current pod phase (pending/running/succeeded/failed/unknown), and pod age (ms since start time). Each pod includes metadata attributes (namespace, node, workload owner such as deployment/statefulset/daemonset/job/cronjob, cluster). Supports filtering via a filter expression, custom groupBy to aggregate pods by any attribute, ordering by any of the six metrics (cpu, cpu_request, cpu_limit, memory, memory_request, memory_limit), and pagination via offset/limit. The response type is 'list' for the default k8s.pod.uid grouping (each row is one pod with its current phase) or 'grouped_list' for custom groupBy keys (each row aggregates pods in the group with per-phase counts: pendingPodCount, runningPodCount, succeededPodCount, failedPodCount, unknownPodCount derived from each pod's latest phase in the window). Also reports missing required metrics and whether the requested time range falls before the data retention boundary. Numeric metric fields (podCPU, podCPURequest, podCPULimit, podMemory, podMemoryRequest, podMemoryLimit, podAge) return -1 as a sentinel when no data is available for that field.",
Request: new(inframonitoringtypes.PostablePods),
RequestContentType: "application/json",
Response: new(inframonitoringtypes.Pods),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusUnauthorized},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
})).Methods(http.MethodPost).GetError(); err != nil {
return err
}
return nil
}

View File

@@ -0,0 +1,52 @@
// Package flaggertest provides helpers for creating Flagger instances in tests.
package flaggertest
import (
"context"
"testing"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/configflagger"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
)
// New returns a Flagger with all flags at their registry defaults (all disabled).
// Use this in tests that do not need any feature flag enabled.
func New(t *testing.T) flagger.Flagger {
t.Helper()
registry := flagger.MustNewRegistry()
fl, err := flagger.New(
context.Background(),
instrumentationtest.New().ToProviderSettings(),
flagger.Config{},
registry,
configflagger.NewFactory(registry),
)
if err != nil {
t.Fatalf("flaggertest.New: %v", err)
}
return fl
}
// WithBodyJSON returns a Flagger with body_json_enabled set to the given value.
func WithBodyJSON(t *testing.T, enabled bool) flagger.Flagger {
t.Helper()
registry := flagger.MustNewRegistry()
cfg := flagger.Config{}
if enabled {
cfg.Config.Boolean = map[string]bool{
flagger.FeatureBodyJSONQuery.String(): true,
}
}
fl, err := flagger.New(
context.Background(),
instrumentationtest.New().ToProviderSettings(),
cfg,
registry,
configflagger.NewFactory(registry),
)
if err != nil {
t.Fatalf("flaggertest.WithBodyJSON: %v", err)
}
return fl
}

View File

@@ -8,6 +8,7 @@ var (
FeatureHideRootUser = featuretypes.MustNewName("hide_root_user")
FeatureGetMetersFromZeus = featuretypes.MustNewName("get_meters_from_zeus")
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureBodyJSONQuery = featuretypes.MustNewName("body_json_enabled")
)
func MustNewRegistry() featuretypes.Registry {
@@ -52,6 +53,14 @@ func MustNewRegistry() featuretypes.Registry {
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureBodyJSONQuery,
Kind: featuretypes.KindBoolean,
Stage: featuretypes.StageExperimental,
Description: "Controls whether body JSON querying is enabled",
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
)
if err != nil {
panic(err)

View File

@@ -45,3 +45,27 @@ func (h *handler) ListHosts(rw http.ResponseWriter, req *http.Request) {
render.Success(rw, http.StatusOK, result)
}
func (h *handler) ListPods(rw http.ResponseWriter, req *http.Request) {
claims, err := authtypes.ClaimsFromContext(req.Context())
if err != nil {
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
var parsedReq inframonitoringtypes.PostablePods
if err := binding.JSON.BindBody(req.Body, &parsedReq); err != nil {
render.Error(rw, err)
return
}
result, err := h.module.ListPods(req.Context(), orgID, &parsedReq)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, result)
}

View File

@@ -14,3 +14,12 @@ type groupHostStatusCounts struct {
Active int
Inactive int
}
// podPhaseCounts holds per-group pod counts bucketed by latest phase in window.
type podPhaseCounts struct {
Pending int
Running int
Succeeded int
Failed int
Unknown int
}

View File

@@ -159,3 +159,86 @@ func (m *module) ListHosts(ctx context.Context, orgID valuer.UUID, req *inframon
return resp, nil
}
func (m *module) ListPods(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostablePods) (*inframonitoringtypes.Pods, error) {
if err := req.Validate(); err != nil {
return nil, err
}
resp := &inframonitoringtypes.Pods{}
if req.OrderBy == nil {
req.OrderBy = &qbtypes.OrderBy{
Key: qbtypes.OrderByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: inframonitoringtypes.PodsOrderByCPU,
},
},
Direction: qbtypes.OrderDirectionDesc,
}
}
if len(req.GroupBy) == 0 {
req.GroupBy = []qbtypes.GroupByKey{podUIDGroupByKey}
resp.Type = inframonitoringtypes.ResponseTypeList
} else {
resp.Type = inframonitoringtypes.ResponseTypeGroupedList
}
missingMetrics, minFirstReportedUnixMilli, err := m.getMetricsExistenceAndEarliestTime(ctx, podsTableMetricNamesList)
if err != nil {
return nil, err
}
if len(missingMetrics) > 0 {
resp.RequiredMetricsCheck = inframonitoringtypes.RequiredMetricsCheck{MissingMetrics: missingMetrics}
resp.Records = []inframonitoringtypes.PodRecord{}
resp.Total = 0
return resp, nil
}
if req.End < int64(minFirstReportedUnixMilli) {
resp.EndTimeBeforeRetention = true
resp.Records = []inframonitoringtypes.PodRecord{}
resp.Total = 0
return resp, nil
}
resp.RequiredMetricsCheck = inframonitoringtypes.RequiredMetricsCheck{MissingMetrics: []string{}}
metadataMap, err := m.getPodsTableMetadata(ctx, req)
if err != nil {
return nil, err
}
resp.Total = len(metadataMap)
pageGroups, err := m.getTopPodGroups(ctx, orgID, req, metadataMap)
if err != nil {
return nil, err
}
if len(pageGroups) == 0 {
resp.Records = []inframonitoringtypes.PodRecord{}
return resp, nil
}
filterExpr := ""
if req.Filter != nil {
filterExpr = req.Filter.Expression
}
fullQueryReq := buildFullQueryRequest(req.Start, req.End, filterExpr, req.GroupBy, pageGroups, m.newPodsTableListQuery())
queryResp, err := m.querier.QueryRange(ctx, orgID, fullQueryReq)
if err != nil {
return nil, err
}
phaseCounts, err := m.getPerGroupPodPhaseCounts(ctx, req, pageGroups)
if err != nil {
return nil, err
}
isPodUIDInGroupBy := isKeyInGroupByAttrs(req.GroupBy, podUIDAttrKey)
resp.Records = buildPodRecords(isPodUIDInGroupBy, queryResp, pageGroups, req.GroupBy, metadataMap, phaseCounts, req.End)
resp.Warning = queryResp.Warning
return resp, nil
}

View File

@@ -0,0 +1,334 @@
package implinframonitoring
import (
"context"
"fmt"
"slices"
"strings"
"time"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
// buildPodRecords assembles the page records. Phase counts come from
// phaseCounts in both modes. In list mode (isPodUIDInGroupBy=true) each
// group is one pod, so exactly one count is 1; PodPhase is derived from
// which one. In grouped_list mode PodPhase stays PodPhaseNone.
func buildPodRecords(
isPodUIDInGroupBy bool,
resp *qbtypes.QueryRangeResponse,
pageGroups []map[string]string,
groupBy []qbtypes.GroupByKey,
metadataMap map[string]map[string]string,
phaseCounts map[string]podPhaseCounts,
reqEnd int64,
) []inframonitoringtypes.PodRecord {
metricsMap := parseFullQueryResponse(resp, groupBy)
records := make([]inframonitoringtypes.PodRecord, 0, len(pageGroups))
for _, labels := range pageGroups {
compositeKey := compositeKeyFromLabels(labels, groupBy)
podUID := labels[podUIDAttrKey]
record := inframonitoringtypes.PodRecord{ // initialize with default values
PodUID: podUID,
PodPhase: inframonitoringtypes.PodPhaseNone,
PodCPU: -1,
PodCPURequest: -1,
PodCPULimit: -1,
PodMemory: -1,
PodMemoryRequest: -1,
PodMemoryLimit: -1,
PodAge: -1,
Meta: map[string]any{},
}
if metrics, ok := metricsMap[compositeKey]; ok {
if v, exists := metrics["A"]; exists {
record.PodCPU = v
}
if v, exists := metrics["B"]; exists {
record.PodCPURequest = v
}
if v, exists := metrics["C"]; exists {
record.PodCPULimit = v
}
if v, exists := metrics["D"]; exists {
record.PodMemory = v
}
if v, exists := metrics["E"]; exists {
record.PodMemoryRequest = v
}
if v, exists := metrics["F"]; exists {
record.PodMemoryLimit = v
}
}
if phaseCountsForGroup, ok := phaseCounts[compositeKey]; ok {
record.PendingPodCount = phaseCountsForGroup.Pending
record.RunningPodCount = phaseCountsForGroup.Running
record.SucceededPodCount = phaseCountsForGroup.Succeeded
record.FailedPodCount = phaseCountsForGroup.Failed
record.UnknownPodCount = phaseCountsForGroup.Unknown
// In list mode each group is one pod; the count==1 bucket identifies the phase.
if isPodUIDInGroupBy {
switch {
case phaseCountsForGroup.Pending == 1:
record.PodPhase = inframonitoringtypes.PodPhasePending
case phaseCountsForGroup.Running == 1:
record.PodPhase = inframonitoringtypes.PodPhaseRunning
case phaseCountsForGroup.Succeeded == 1:
record.PodPhase = inframonitoringtypes.PodPhaseSucceeded
case phaseCountsForGroup.Failed == 1:
record.PodPhase = inframonitoringtypes.PodPhaseFailed
case phaseCountsForGroup.Unknown == 1:
record.PodPhase = inframonitoringtypes.PodPhaseUnknown
}
}
}
if attrs, ok := metadataMap[compositeKey]; ok && isPodUIDInGroupBy {
// the condition above ensures we deduce age only if pod uid is in group by because if
// it's not in group by then we might have multiple pod uids in the same group and hence then podAge wont make sense
if startTimeStr, exists := attrs[podStartTimeAttrKey]; exists && startTimeStr != "" {
if t, err := time.Parse(time.RFC3339, startTimeStr); err == nil {
startTimeMs := t.UnixMilli()
if startTimeMs > 0 {
record.PodAge = reqEnd - startTimeMs
}
}
}
for k, v := range attrs {
record.Meta[k] = v
}
}
records = append(records, record)
}
return records
}
func (m *module) getTopPodGroups(
ctx context.Context,
orgID valuer.UUID,
req *inframonitoringtypes.PostablePods,
metadataMap map[string]map[string]string,
) ([]map[string]string, error) {
orderByKey := req.OrderBy.Key.Name
queryNamesForOrderBy := orderByToPodsQueryNames[orderByKey]
rankingQueryName := queryNamesForOrderBy[len(queryNamesForOrderBy)-1]
topReq := &qbtypes.QueryRangeRequest{
Start: uint64(req.Start),
End: uint64(req.End),
RequestType: qbtypes.RequestTypeScalar,
CompositeQuery: qbtypes.CompositeQuery{
Queries: make([]qbtypes.QueryEnvelope, 0, len(queryNamesForOrderBy)),
},
}
for _, envelope := range m.newPodsTableListQuery().CompositeQuery.Queries {
if !slices.Contains(queryNamesForOrderBy, envelope.GetQueryName()) {
continue
}
copied := envelope
if copied.Type == qbtypes.QueryTypeBuilder {
existingExpr := ""
if f := copied.GetFilter(); f != nil {
existingExpr = f.Expression
}
reqFilterExpr := ""
if req.Filter != nil {
reqFilterExpr = req.Filter.Expression
}
merged := mergeFilterExpressions(existingExpr, reqFilterExpr)
copied.SetFilter(&qbtypes.Filter{Expression: merged})
copied.SetGroupBy(req.GroupBy)
}
topReq.CompositeQuery.Queries = append(topReq.CompositeQuery.Queries, copied)
}
resp, err := m.querier.QueryRange(ctx, orgID, topReq)
if err != nil {
return nil, err
}
allMetricGroups := parseAndSortGroups(resp, rankingQueryName, req.GroupBy, req.OrderBy.Direction)
return paginateWithBackfill(allMetricGroups, metadataMap, req.GroupBy, req.Offset, req.Limit), nil
}
func (m *module) getPodsTableMetadata(ctx context.Context, req *inframonitoringtypes.PostablePods) (map[string]map[string]string, error) {
var nonGroupByAttrs []string
for _, key := range podAttrKeysForMetadata {
if !isKeyInGroupByAttrs(req.GroupBy, key) {
nonGroupByAttrs = append(nonGroupByAttrs, key)
}
}
return m.getMetadata(ctx, podsTableMetricNamesList, req.GroupBy, nonGroupByAttrs, req.Filter, req.Start, req.End)
}
// getPerGroupPodPhaseCounts computes per-group pod counts bucketed by each
// pod's latest phase in the requested window.
// Pipeline:
//
// timeSeriesFPs: fp ↔ (pod_uid, groupBy cols) from the time_series table.
// User filter + page-groups filter applied here.
// latestPhasePerPod: INNER JOIN samples × timeSeriesFPs, collapsed to
// the latest phase per pod via argMax(value, unix_milli).
// countPodsPerPhase: per-group uniqExactIf into 5 phase buckets.
//
// Groups absent from the result map have implicit zero counts (caller default).
func (m *module) getPerGroupPodPhaseCounts(
ctx context.Context,
req *inframonitoringtypes.PostablePods,
pageGroups []map[string]string,
) (map[string]podPhaseCounts, error) {
if len(pageGroups) == 0 || len(req.GroupBy) == 0 {
return map[string]podPhaseCounts{}, nil
}
// Merged filter expression (user filter + page-groups IN clauses).
reqFilterExpr := ""
if req.Filter != nil {
reqFilterExpr = req.Filter.Expression
}
pageGroupsFilterExpr := buildPageGroupsFilterExpr(pageGroups)
filterExpr := mergeFilterExpressions(reqFilterExpr, pageGroupsFilterExpr)
// Resolve tables. Same convention as hosts (distributed names from helpers).
adjustedStart, adjustedEnd, _, localTimeSeriesTable := telemetrymetrics.WhichTSTableToUse(
uint64(req.Start), uint64(req.End), nil,
)
samplesTable := telemetrymetrics.WhichSamplesTableToUse(
uint64(req.Start), uint64(req.End),
metrictypes.UnspecifiedType, metrictypes.TimeAggregationUnspecified, nil,
)
valueCol := telemetrymetrics.ValueColumnForSamplesTable(samplesTable)
// ----- timeSeriesFPs -----
timeSeriesFPs := sqlbuilder.NewSelectBuilder()
timeSeriesFPsSelectCols := []string{
"fingerprint",
fmt.Sprintf("JSONExtractString(labels, %s) AS pod_uid", timeSeriesFPs.Var(podUIDAttrKey)),
}
for _, key := range req.GroupBy {
timeSeriesFPsSelectCols = append(timeSeriesFPsSelectCols,
fmt.Sprintf("JSONExtractString(labels, %s) AS %s", timeSeriesFPs.Var(key.Name), quoteIdentifier(key.Name)),
)
}
timeSeriesFPs.Select(timeSeriesFPsSelectCols...)
timeSeriesFPs.From(fmt.Sprintf("%s.%s", telemetrymetrics.DBName, localTimeSeriesTable))
timeSeriesFPs.Where(
timeSeriesFPs.E("metric_name", podPhaseMetricName),
timeSeriesFPs.GE("unix_milli", adjustedStart),
timeSeriesFPs.L("unix_milli", adjustedEnd),
)
if filterExpr != "" {
filterClause, err := m.buildFilterClause(ctx, &qbtypes.Filter{Expression: filterExpr}, req.Start, req.End)
if err != nil {
return nil, err
}
if filterClause != nil {
timeSeriesFPs.AddWhereClause(filterClause)
}
}
timeSeriesFPsGroupBy := []string{"fingerprint", "pod_uid"}
for _, key := range req.GroupBy {
timeSeriesFPsGroupBy = append(timeSeriesFPsGroupBy, quoteIdentifier(key.Name))
}
timeSeriesFPs.GroupBy(timeSeriesFPsGroupBy...)
timeSeriesFPsSQL, timeSeriesFPsArgs := timeSeriesFPs.BuildWithFlavor(sqlbuilder.ClickHouse)
latestPhasePerPod := sqlbuilder.NewSelectBuilder()
latestPhasePerPodSelectCols := []string{"tsfp.pod_uid AS pod_uid"}
latestPhasePerPodGroupBy := []string{"pod_uid"}
for _, key := range req.GroupBy {
col := quoteIdentifier(key.Name)
latestPhasePerPodSelectCols = append(latestPhasePerPodSelectCols, fmt.Sprintf("tsfp.%s AS %s", col, col))
latestPhasePerPodGroupBy = append(latestPhasePerPodGroupBy, col)
}
latestPhasePerPodSelectCols = append(latestPhasePerPodSelectCols,
fmt.Sprintf("argMax(samples.%s, samples.unix_milli) AS phase_value", valueCol),
)
latestPhasePerPod.Select(latestPhasePerPodSelectCols...)
latestPhasePerPod.From(fmt.Sprintf(
"%s.%s AS samples INNER JOIN time_series_fps AS tsfp ON samples.fingerprint = tsfp.fingerprint",
telemetrymetrics.DBName, samplesTable,
))
latestPhasePerPod.Where(
latestPhasePerPod.E("samples.metric_name", podPhaseMetricName),
latestPhasePerPod.GE("samples.unix_milli", req.Start),
latestPhasePerPod.L("samples.unix_milli", req.End),
"tsfp.pod_uid != ''",
)
latestPhasePerPod.GroupBy(latestPhasePerPodGroupBy...)
latestPhasePerPodSQL, latestPhasePerPodArgs := latestPhasePerPod.BuildWithFlavor(sqlbuilder.ClickHouse)
// ----- countPodsPerPhase (outer SELECT) -----
countPodsPerPhaseSelectCols := make([]string, 0, len(req.GroupBy)+5)
countPodsPerPhaseGroupBy := make([]string, 0, len(req.GroupBy))
for _, key := range req.GroupBy {
col := quoteIdentifier(key.Name)
countPodsPerPhaseSelectCols = append(countPodsPerPhaseSelectCols, col)
countPodsPerPhaseGroupBy = append(countPodsPerPhaseGroupBy, col)
}
countPodsPerPhaseSelectCols = append(countPodsPerPhaseSelectCols,
fmt.Sprintf("uniqExactIf(pod_uid, phase_value = %d) AS pending_count", inframonitoringtypes.PodPhaseNumPending),
fmt.Sprintf("uniqExactIf(pod_uid, phase_value = %d) AS running_count", inframonitoringtypes.PodPhaseNumRunning),
fmt.Sprintf("uniqExactIf(pod_uid, phase_value = %d) AS succeeded_count", inframonitoringtypes.PodPhaseNumSucceeded),
fmt.Sprintf("uniqExactIf(pod_uid, phase_value = %d) AS failed_count", inframonitoringtypes.PodPhaseNumFailed),
fmt.Sprintf("uniqExactIf(pod_uid, phase_value = %d) AS unknown_count", inframonitoringtypes.PodPhaseNumUnknown),
)
countPodsPerPhaseSQL := fmt.Sprintf(
"SELECT %s FROM latest_phase_per_pod GROUP BY %s",
strings.Join(countPodsPerPhaseSelectCols, ", "),
strings.Join(countPodsPerPhaseGroupBy, ", "),
)
// Combine CTEs + outer.
cteFragments := []string{
fmt.Sprintf("time_series_fps AS (%s)", timeSeriesFPsSQL),
fmt.Sprintf("latest_phase_per_pod AS (%s)", latestPhasePerPodSQL),
}
finalSQL := querybuilder.CombineCTEs(cteFragments) + countPodsPerPhaseSQL
finalArgs := querybuilder.PrependArgs([][]any{timeSeriesFPsArgs, latestPhasePerPodArgs}, nil)
rows, err := m.telemetryStore.ClickhouseDB().Query(ctx, finalSQL, finalArgs...)
if err != nil {
return nil, err
}
defer rows.Close()
result := make(map[string]podPhaseCounts)
for rows.Next() {
groupVals := make([]string, len(req.GroupBy))
scanPtrs := make([]any, 0, len(req.GroupBy)+5)
for i := range groupVals {
scanPtrs = append(scanPtrs, &groupVals[i])
}
var pending, running, succeeded, failed, unknown uint64
scanPtrs = append(scanPtrs, &pending, &running, &succeeded, &failed, &unknown)
if err := rows.Scan(scanPtrs...); err != nil {
return nil, err
}
result[compositeKeyFromList(groupVals)] = podPhaseCounts{
Pending: int(pending),
Running: int(running),
Succeeded: int(succeeded),
Failed: int(failed),
Unknown: int(unknown),
}
}
if err := rows.Err(); err != nil {
return nil, err
}
return result, nil
}

View File

@@ -0,0 +1,178 @@
package implinframonitoring
import (
"github.com/SigNoz/signoz/pkg/types/inframonitoringtypes"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
const (
podUIDAttrKey = "k8s.pod.uid"
podStartTimeAttrKey = "k8s.pod.start_time"
podPhaseMetricName = "k8s.pod.phase"
)
var podUIDGroupByKey = qbtypes.GroupByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: podUIDAttrKey,
FieldContext: telemetrytypes.FieldContextResource,
FieldDataType: telemetrytypes.FieldDataTypeString,
},
}
var podsTableMetricNamesList = []string{
"k8s.pod.cpu.usage",
"k8s.pod.cpu_request_utilization",
"k8s.pod.cpu_limit_utilization",
"k8s.pod.memory.working_set",
"k8s.pod.memory_request_utilization",
"k8s.pod.memory_limit_utilization",
"k8s.pod.phase",
}
var podAttrKeysForMetadata = []string{
"k8s.pod.uid",
"k8s.pod.name",
"k8s.namespace.name",
"k8s.node.name",
"k8s.deployment.name",
"k8s.statefulset.name",
"k8s.daemonset.name",
"k8s.job.name",
"k8s.cronjob.name",
"k8s.cluster.name",
"k8s.pod.start_time",
}
var orderByToPodsQueryNames = map[string][]string{
inframonitoringtypes.PodsOrderByCPU: {"A"},
inframonitoringtypes.PodsOrderByCPURequest: {"B"},
inframonitoringtypes.PodsOrderByCPULimit: {"C"},
inframonitoringtypes.PodsOrderByMemory: {"D"},
inframonitoringtypes.PodsOrderByMemoryRequest: {"E"},
inframonitoringtypes.PodsOrderByMemoryLimit: {"F"},
}
// newPodsTableListQuery builds the composite QB v5 request for the pods list.
// Pod phase is derived separately via getPerGroupPodPhaseCounts (works for both
// list and grouped_list modes), so no phase query is included here.
func (m *module) newPodsTableListQuery() *qbtypes.QueryRangeRequest {
queries := []qbtypes.QueryEnvelope{
// Query A: CPU usage
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "A",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "k8s.pod.cpu.usage",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
ReduceTo: qbtypes.ReduceToAvg,
},
},
GroupBy: []qbtypes.GroupByKey{podUIDGroupByKey},
Disabled: false,
},
},
// Query B: CPU request utilization
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "B",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "k8s.pod.cpu_request_utilization",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationAvg,
ReduceTo: qbtypes.ReduceToAvg,
},
},
GroupBy: []qbtypes.GroupByKey{podUIDGroupByKey},
Disabled: false,
},
},
// Query C: CPU limit utilization
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "C",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "k8s.pod.cpu_limit_utilization",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationAvg,
ReduceTo: qbtypes.ReduceToAvg,
},
},
GroupBy: []qbtypes.GroupByKey{podUIDGroupByKey},
Disabled: false,
},
},
// Query D: Memory working set
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "D",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "k8s.pod.memory.working_set",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationSum,
ReduceTo: qbtypes.ReduceToAvg,
},
},
GroupBy: []qbtypes.GroupByKey{podUIDGroupByKey},
Disabled: false,
},
},
// Query E: Memory request utilization
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "E",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "k8s.pod.memory_request_utilization",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationAvg,
ReduceTo: qbtypes.ReduceToAvg,
},
},
GroupBy: []qbtypes.GroupByKey{podUIDGroupByKey},
Disabled: false,
},
},
// Query F: Memory limit utilization
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.MetricAggregation]{
Name: "F",
Signal: telemetrytypes.SignalMetrics,
Aggregations: []qbtypes.MetricAggregation{
{
MetricName: "k8s.pod.memory_limit_utilization",
TimeAggregation: metrictypes.TimeAggregationAvg,
SpaceAggregation: metrictypes.SpaceAggregationAvg,
ReduceTo: qbtypes.ReduceToAvg,
},
},
GroupBy: []qbtypes.GroupByKey{podUIDGroupByKey},
Disabled: false,
},
},
}
return &qbtypes.QueryRangeRequest{
RequestType: qbtypes.RequestTypeScalar,
CompositeQuery: qbtypes.CompositeQuery{
Queries: queries,
},
}
}

View File

@@ -10,8 +10,10 @@ import (
type Handler interface {
ListHosts(http.ResponseWriter, *http.Request)
ListPods(http.ResponseWriter, *http.Request)
}
type Module interface {
ListHosts(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostableHosts) (*inframonitoringtypes.Hosts, error)
ListPods(ctx context.Context, orgID valuer.UUID, req *inframonitoringtypes.PostablePods) (*inframonitoringtypes.Pods, error)
}

View File

@@ -72,13 +72,14 @@ func newProvider(
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataLocalTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,
flagger,
)
// Create trace statement builder
traceFieldMapper := telemetrytraces.NewFieldMapper()
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, traceFieldMapper, traceConditionBuilder, nil)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(settings, nil, traceFieldMapper, traceConditionBuilder, nil, flagger)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
settings,
telemetryMetadataStore,
@@ -86,6 +87,7 @@ func newProvider(
traceConditionBuilder,
traceAggExprRewriter,
telemetryStore,
flagger,
)
// Create trace operator statement builder
@@ -96,17 +98,19 @@ func newProvider(
traceConditionBuilder,
traceStmtBuilder,
traceAggExprRewriter,
flagger,
)
// Create log statement builder
logFieldMapper := telemetrylogs.NewFieldMapper()
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper)
logFieldMapper := telemetrylogs.NewFieldMapper(flagger)
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, flagger)
logAggExprRewriter := querybuilder.NewAggExprRewriter(
settings,
telemetrylogs.DefaultFullTextColumn,
logFieldMapper,
logConditionBuilder,
telemetrylogs.GetBodyJSONKey,
flagger,
)
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
settings,
@@ -116,6 +120,7 @@ func newProvider(
logAggExprRewriter,
telemetrylogs.DefaultFullTextColumn,
telemetrylogs.GetBodyJSONKey,
flagger,
)
// Create audit statement builder
@@ -127,6 +132,7 @@ func newProvider(
auditFieldMapper,
auditConditionBuilder,
nil,
flagger,
)
auditStmtBuilder := telemetryaudit.NewAuditQueryStatementBuilder(
settings,
@@ -136,6 +142,7 @@ func newProvider(
auditAggExprRewriter,
telemetryaudit.DefaultFullTextColumn,
nil,
flagger,
)
// Create metric statement builder

View File

@@ -1774,6 +1774,15 @@ func (aH *APIHandler) getFeatureFlags(w http.ResponseWriter, r *http.Request) {
Route: "",
})
bodyJSONQuery := aH.Signoz.Flagger.BooleanOrEmpty(r.Context(), flagger.FeatureBodyJSONQuery, evalCtx)
featureSet = append(featureSet, &licensetypes.Feature{
Name: valuer.NewString(flagger.FeatureBodyJSONQuery.String()),
Active: bodyJSONQuery,
Usage: 0,
UsageLimit: -1,
Route: "",
})
if constants.IsDotMetricsEnabled {
for idx, feature := range featureSet {
if feature.Name == licensetypes.DotMetricsEnabled {

View File

@@ -11,15 +11,16 @@ import (
"github.com/google/uuid"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/query-service/agentConf"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/interfaces"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/query-service/utils"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/types/opamptypes"
"github.com/SigNoz/signoz/pkg/types/pipelinetypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -38,18 +39,21 @@ type LogParsingPipelineController struct {
GetIntegrationPipelines func(context.Context, string) ([]pipelinetypes.GettablePipeline, error)
// TODO(Piyush): remove with qbv5 migration
reader interfaces.Reader
fl flagger.Flagger
}
func NewLogParsingPipelinesController(
sqlStore sqlstore.SQLStore,
getIntegrationPipelines func(context.Context, string) ([]pipelinetypes.GettablePipeline, error),
reader interfaces.Reader,
fl flagger.Flagger,
) (*LogParsingPipelineController, error) {
repo := NewRepo(sqlStore)
return &LogParsingPipelineController{
Repo: repo,
GetIntegrationPipelines: getIntegrationPipelines,
reader: reader,
fl: fl,
}, nil
}
@@ -363,14 +367,14 @@ func (pc *LogParsingPipelineController) AgentFeatureType() agentConf.AgentFeatur
// Implements agentConf.AgentFeature interface.
// RecommendAgentConfig generates the collector config to be sent to agents.
// The normalize pipeline (when BodyJSONQueryEnabled) is injected here, after
// The normalize pipeline (when body_json_enabled feature flag is on) is injected here, after
// rawPipelineData is serialized. So it is only present in the config sent to
// the collector and never persisted to the database as part of the user's pipeline list.
//
// NOTE: The configId sent to agents is derived from the pipeline version number
// (e.g. "LogPipelines:5"), not the YAML content. If server-side logic changes
// the generated YAML without bumping the version (e.g. toggling BodyJSONQueryEnabled
// or updating operator IfExpressions), agents that already applied that version will
// the generated YAML without bumping the version (e.g. toggling the body_json_enabled
// flag or updating operator IfExpressions), agents that already applied that version will
// not re-apply the new config. In such cases, users must save a new pipeline version
// via the API to force agents to pick up the change.
func (pc *LogParsingPipelineController) RecommendAgentConfig(
@@ -398,7 +402,8 @@ func (pc *LogParsingPipelineController) RecommendAgentConfig(
return nil, "", err
}
if querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if pc.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(orgId)) {
// add default normalize pipeline at the beginning, only for sending to collector
enrichedPipelines = append([]pipelinetypes.GettablePipeline{pc.getNormalizePipeline()}, enrichedPipelines...)
}

View File

@@ -93,6 +93,7 @@ func NewServer(config signoz.Config, signoz *signoz.SigNoz) (*Server, error) {
signoz.SQLStore,
integrationsController.GetPipelinesForInstalledIntegrations,
reader,
signoz.Flagger,
)
if err != nil {
return nil, err

View File

@@ -15,6 +15,8 @@ import (
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/require"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
)
func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.TelemetryStore) (querier.Querier, *telemetrytypestest.MockMetadataStore) {
@@ -54,8 +56,8 @@ func prepareQuerierForMetrics(t *testing.T, telemetryStore telemetrystore.Teleme
), metadataStore
}
func prepareQuerierForLogs(telemetryStore telemetrystore.TelemetryStore, keysMap map[string][]*telemetrytypes.TelemetryFieldKey) querier.Querier {
func prepareQuerierForLogs(t *testing.T, telemetryStore telemetrystore.TelemetryStore, keysMap map[string][]*telemetrytypes.TelemetryFieldKey) querier.Querier {
t.Helper()
providerSettings := instrumentationtest.New().ToProviderSettings()
metadataStore := telemetrytypestest.NewMockMetadataStore()
@@ -66,14 +68,16 @@ func prepareQuerierForLogs(telemetryStore telemetrystore.TelemetryStore, keysMap
}
metadataStore.KeysMap = keysMap
logFieldMapper := telemetrylogs.NewFieldMapper()
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper)
fl := flaggertest.New(t)
logFieldMapper := telemetrylogs.NewFieldMapper(fl)
logConditionBuilder := telemetrylogs.NewConditionBuilder(logFieldMapper, fl)
logAggExprRewriter := querybuilder.NewAggExprRewriter(
providerSettings,
telemetrylogs.DefaultFullTextColumn,
logFieldMapper,
logConditionBuilder,
telemetrylogs.GetBodyJSONKey,
fl,
)
logStmtBuilder := telemetrylogs.NewLogQueryStatementBuilder(
providerSettings,
@@ -83,6 +87,7 @@ func prepareQuerierForLogs(telemetryStore telemetrystore.TelemetryStore, keysMap
logAggExprRewriter,
telemetrylogs.DefaultFullTextColumn,
telemetrylogs.GetBodyJSONKey,
fl,
)
return querier.New(
@@ -100,7 +105,8 @@ func prepareQuerierForLogs(telemetryStore telemetrystore.TelemetryStore, keysMap
)
}
func prepareQuerierForTraces(telemetryStore telemetrystore.TelemetryStore, keysMap map[string][]*telemetrytypes.TelemetryFieldKey) querier.Querier {
func prepareQuerierForTraces(t *testing.T, telemetryStore telemetrystore.TelemetryStore, keysMap map[string][]*telemetrytypes.TelemetryFieldKey) querier.Querier {
t.Helper()
providerSettings := instrumentationtest.New().ToProviderSettings()
metadataStore := telemetrytypestest.NewMockMetadataStore()
@@ -116,7 +122,8 @@ func prepareQuerierForTraces(telemetryStore telemetrystore.TelemetryStore, keysM
traceFieldMapper := telemetrytraces.NewFieldMapper()
traceConditionBuilder := telemetrytraces.NewConditionBuilder(traceFieldMapper)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(providerSettings, nil, traceFieldMapper, traceConditionBuilder, nil)
fl := flaggertest.New(t)
traceAggExprRewriter := querybuilder.NewAggExprRewriter(providerSettings, nil, traceFieldMapper, traceConditionBuilder, nil, fl)
traceStmtBuilder := telemetrytraces.NewTraceQueryStatementBuilder(
providerSettings,
metadataStore,
@@ -124,6 +131,7 @@ func prepareQuerierForTraces(telemetryStore telemetrystore.TelemetryStore, keysM
traceConditionBuilder,
traceAggExprRewriter,
telemetryStore,
fl,
)
return querier.New(

View File

@@ -829,7 +829,7 @@ func TestThresholdRuleTracesLink(t *testing.T) {
WithArgs(nil, nil, nil, nil, nil, nil, nil).
WillReturnRows(rows)
querier := prepareQuerierForTraces(telemetryStore, keysMap)
querier := prepareQuerierForTraces(t, telemetryStore, keysMap)
postableRule.RuleCondition.CompareOperator = c.compareOperator
postableRule.RuleCondition.MatchType = c.matchType
@@ -946,7 +946,7 @@ func TestThresholdRuleLogsLink(t *testing.T) {
WithArgs(nil, nil, nil, nil, nil, nil, nil, nil, nil, nil).
WillReturnRows(rows)
querier := prepareQuerierForLogs(telemetryStore, keysMap)
querier := prepareQuerierForLogs(t, telemetryStore, keysMap)
postableRule.RuleCondition.CompareOperator = c.compareOperator
postableRule.RuleCondition.MatchType = c.matchType

View File

@@ -9,6 +9,8 @@ import (
chparser "github.com/AfterShip/clickhouse-sql-parser/parser"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -21,6 +23,7 @@ type aggExprRewriter struct {
fieldMapper qbtypes.FieldMapper
conditionBuilder qbtypes.ConditionBuilder
jsonKeyToKey qbtypes.JsonKeyToFieldFunc
flagger flagger.Flagger
}
var _ qbtypes.AggExprRewriter = (*aggExprRewriter)(nil)
@@ -31,6 +34,7 @@ func NewAggExprRewriter(
fieldMapper qbtypes.FieldMapper,
conditionBuilder qbtypes.ConditionBuilder,
jsonKeyToKey qbtypes.JsonKeyToFieldFunc,
fl flagger.Flagger,
) *aggExprRewriter {
set := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/querybuilder/agg_rewrite")
@@ -40,6 +44,7 @@ func NewAggExprRewriter(
fieldMapper: fieldMapper,
conditionBuilder: conditionBuilder,
jsonKeyToKey: jsonKeyToKey,
flagger: fl,
}
}
@@ -86,6 +91,7 @@ func (r *aggExprRewriter) Rewrite(
r.fieldMapper,
r.conditionBuilder,
r.jsonKeyToKey,
r.flagger,
)
// Rewrite the first select item (our expression)
if err := sel.SelectItems[0].Accept(visitor); err != nil {
@@ -138,6 +144,7 @@ type exprVisitor struct {
fieldMapper qbtypes.FieldMapper
conditionBuilder qbtypes.ConditionBuilder
jsonKeyToKey qbtypes.JsonKeyToFieldFunc
flagger flagger.Flagger
Modified bool
chArgs []any
isRate bool
@@ -153,6 +160,7 @@ func newExprVisitor(
fieldMapper qbtypes.FieldMapper,
conditionBuilder qbtypes.ConditionBuilder,
jsonKeyToKey qbtypes.JsonKeyToFieldFunc,
fl flagger.Flagger,
) *exprVisitor {
return &exprVisitor{
ctx: ctx,
@@ -164,6 +172,7 @@ func newExprVisitor(
fieldMapper: fieldMapper,
conditionBuilder: conditionBuilder,
jsonKeyToKey: jsonKeyToKey,
flagger: fl,
}
}
@@ -197,6 +206,9 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
dataType = telemetrytypes.FieldDataTypeFloat64
}
//
bodyJSONEnabled := v.flagger.BooleanOrEmpty(v.ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
// Handle *If functions with predicate + values
if aggFunc.FuncCombinator {
// Map the predicate (last argument)
@@ -209,6 +221,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
FieldKeys: v.fieldKeys,
FieldMapper: v.fieldMapper,
ConditionBuilder: v.conditionBuilder,
BodyJSONEnabled: bodyJSONEnabled,
FullTextColumn: v.fullTextColumn,
JsonKeyToKey: v.jsonKeyToKey,
StartNs: v.startNs,
@@ -237,7 +250,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
for i := 0; i < len(args)-1; i++ {
origVal := args[i].String()
fieldKey := telemetrytypes.GetFieldKeyFromKeyText(origVal)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return errors.WrapInvalidInputf(err, errors.CodeInvalidInput, "failed to get table field name for %q", origVal)
}
@@ -255,7 +268,7 @@ func (v *exprVisitor) VisitFunctionExpr(fn *chparser.FunctionExpr) error {
for i, arg := range args {
orig := arg.String()
fieldKey := telemetrytypes.GetFieldKeyFromKeyText(orig)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey)
expr, exprArgs, err := CollisionHandledFinalExpr(v.ctx, v.startNs, v.endNs, &fieldKey, v.fieldMapper, v.conditionBuilder, v.fieldKeys, dataType, v.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return err
}

View File

@@ -1,9 +1,5 @@
package querybuilder
import (
"os"
)
const (
TrueConditionLiteral = "true"
SkipConditionLiteral = "__skip__"
@@ -13,15 +9,3 @@ const (
var (
SkippableConditionLiterals = []string{SkipConditionLiteral, ErrorConditionLiteral}
)
var (
BodyJSONQueryEnabled = GetOrDefaultEnv("BODY_JSON_QUERY_ENABLED", "false") == "true"
)
func GetOrDefaultEnv(key string, fallback string) string {
v := os.Getenv(key)
if len(v) == 0 {
return fallback
}
return v
}

View File

@@ -27,6 +27,7 @@ func CollisionHandledFinalExpr(
keys map[string][]*telemetrytypes.TelemetryFieldKey,
requiredDataType telemetrytypes.FieldDataType,
jsonKeyToKey qbtypes.JsonKeyToFieldFunc,
bodyJSONEnabled bool,
) (string, []any, error) {
if requiredDataType != telemetrytypes.FieldDataTypeString &&
@@ -106,7 +107,7 @@ func CollisionHandledFinalExpr(
}
// first if condition covers the older tests and second if condition covers the array conditions
if !BodyJSONQueryEnabled && field.FieldContext == telemetrytypes.FieldContextBody && jsonKeyToKey != nil {
if !bodyJSONEnabled && field.FieldContext == telemetrytypes.FieldContextBody && jsonKeyToKey != nil {
return "", nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "Group by/Aggregation isn't available for the body column")
} else if strings.Contains(field.Name, telemetrytypes.ArraySep) || strings.Contains(field.Name, telemetrytypes.ArrayAnyIndex) {
return "", nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "Group by/Aggregation isn't available for the Array Paths: %s", field.Name)

View File

@@ -36,6 +36,7 @@ type filterExpressionVisitor struct {
builder *sqlbuilder.SelectBuilder
fullTextColumn *telemetrytypes.TelemetryFieldKey
jsonKeyToKey qbtypes.JsonKeyToFieldFunc
bodyJSONEnabled bool
skipResourceFilter bool
skipFullTextFilter bool
skipFunctionCalls bool
@@ -56,6 +57,7 @@ type FilterExprVisitorOpts struct {
Builder *sqlbuilder.SelectBuilder
FullTextColumn *telemetrytypes.TelemetryFieldKey
JsonKeyToKey qbtypes.JsonKeyToFieldFunc
BodyJSONEnabled bool
SkipResourceFilter bool
SkipFullTextFilter bool
SkipFunctionCalls bool
@@ -76,6 +78,7 @@ func newFilterExpressionVisitor(opts FilterExprVisitorOpts) *filterExpressionVis
builder: opts.Builder,
fullTextColumn: opts.FullTextColumn,
jsonKeyToKey: opts.JsonKeyToKey,
bodyJSONEnabled: opts.BodyJSONEnabled,
skipResourceFilter: opts.SkipResourceFilter,
skipFullTextFilter: opts.SkipFullTextFilter,
skipFunctionCalls: opts.SkipFunctionCalls,
@@ -751,8 +754,8 @@ func (v *filterExpressionVisitor) VisitFunctionCall(ctx *grammar.FunctionCallCon
return ErrorConditionLiteral
}
// filter arrays from keys
if BodyJSONQueryEnabled && functionName != "hasToken" {
// TODO(Tushar): thread orgID here to evaluate correctly
if v.bodyJSONEnabled && functionName != "hasToken" {
filteredKeys := []*telemetrytypes.TelemetryFieldKey{}
for _, key := range keys {
if key.FieldDataType.IsArray() {
@@ -793,7 +796,7 @@ func (v *filterExpressionVisitor) VisitFunctionCall(ctx *grammar.FunctionCallCon
// this is that all other functions only support array fields
if key.FieldContext == telemetrytypes.FieldContextBody {
var err error
if BodyJSONQueryEnabled {
if v.bodyJSONEnabled {
fieldName, err = v.fieldMapper.FieldFor(v.context, v.startNs, v.endNs, key)
if err != nil {
v.errors = append(v.errors, fmt.Sprintf("failed to get field name for key %s: %s", key.Name, err.Error()))
@@ -936,7 +939,8 @@ func (v *filterExpressionVisitor) VisitKey(ctx *grammar.KeyContext) any {
// Note: Skip this logic if body json query is enabled so we can look up the key inside fields
//
// TODO(Piyush): After entire migration this is supposed to be removed.
if !BodyJSONQueryEnabled && fieldKey.FieldContext == telemetrytypes.FieldContextBody {
// TODO(Tushar): thread orgID here to evaluate correctly
if fieldKey.FieldContext == telemetrytypes.FieldContextBody && !v.bodyJSONEnabled {
fieldKeysForName = append(fieldKeysForName, &fieldKey)
}

View File

@@ -79,8 +79,10 @@ func TestPrepareWhereClause_EmptyVariableList(t *testing.T) {
}
// createTestVisitor creates a filterExpressionVisitor for testing VisitKey.
func createTestVisitor(fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey, ignoreNotFoundKeys bool) *filterExpressionVisitor {
func createTestVisitor(t *testing.T, fieldKeys map[string][]*telemetrytypes.TelemetryFieldKey, ignoreNotFoundKeys bool) *filterExpressionVisitor {
t.Helper()
return &filterExpressionVisitor{
context: t.Context(),
logger: slog.Default(),
fieldKeys: fieldKeys,
ignoreNotFoundKeys: ignoreNotFoundKeys,
@@ -572,7 +574,7 @@ func TestVisitKey(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
visitor := createTestVisitor(tt.fieldKeys, tt.ignoreNotFoundKeys)
visitor := createTestVisitor(t, tt.fieldKeys, tt.ignoreNotFoundKeys)
keyCtx := parseKeyContext(tt.keyText)
if keyCtx == nil {
@@ -776,7 +778,8 @@ type visitComparisonCase struct {
// visitComparisonOpts builds the two FilterExprVisitorOpts shared by all
// TestVisitComparison_* tests.
func visitComparisonOpts() (rsbOpts, sbOpts FilterExprVisitorOpts) {
func visitComparisonOpts(t *testing.T) (rsbOpts, sbOpts FilterExprVisitorOpts) {
t.Helper()
allVariable := map[string]qbtypes.VariableItem{
"service": {
Type: qbtypes.DynamicVariableType,
@@ -790,6 +793,7 @@ func visitComparisonOpts() (rsbOpts, sbOpts FilterExprVisitorOpts) {
FieldDataType: telemetrytypes.FieldDataTypeString,
}
rsbOpts = FilterExprVisitorOpts{
Context: t.Context(),
FieldKeys: visitTestKeys,
ConditionBuilder: &resourceConditionBuilder{},
Variables: allVariable,
@@ -799,6 +803,7 @@ func visitComparisonOpts() (rsbOpts, sbOpts FilterExprVisitorOpts) {
IgnoreNotFoundKeys: true,
}
sbOpts = FilterExprVisitorOpts{
Context: t.Context(),
FieldKeys: visitTestKeys,
ConditionBuilder: &conditionBuilder{},
Variables: allVariable,
@@ -814,7 +819,7 @@ func visitComparisonOpts() (rsbOpts, sbOpts FilterExprVisitorOpts) {
// TestVisitComparison_AND covers AND expressions with attribute keys (a, b, c →
// TrueConditionLiteral in RSB) and resource keys (x, y, z → "{name}_cond" in RSB).
func TestVisitComparison_AND(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
name: "single attribute key",
@@ -892,7 +897,7 @@ func TestVisitComparison_AND(t *testing.T) {
// - NOT inside a comparison (e.g. NOT LIKE, NOT EXISTS): the inner NOT is folded
// into the operator token; conditionBuilder ignores it, so no extra NOT is emitted.
func TestVisitComparison_NOT(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
// Unary NOT on an attribute key: NOT(SkipConditionLiteral) → SkipConditionLiteral (guard).
@@ -985,7 +990,7 @@ func TestVisitComparison_NOT(t *testing.T) {
// SkipResourceFilter to false when an OR token is detected in the expression,
// so resource keys become visible in sbOpts for all cases in this suite.
func TestVisitComparison_OR(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
name: "resource OR resource",
@@ -1086,7 +1091,7 @@ func TestVisitComparison_OR(t *testing.T) {
// TestVisitComparison_Precedence covers AND/OR/NOT operator precedence
// (AND binds tighter than OR; NOT binds tightest).
func TestVisitComparison_Precedence(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
// a→true short-circuits OR.
@@ -1168,7 +1173,7 @@ func TestVisitComparison_Precedence(t *testing.T) {
// VisitPrimary adds one extra layer of parens around real conditions;
// TrueConditionLiteral passes through unwrapped.
func TestVisitComparison_Parens(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
// RSB: SkipConditionLiteral passes through unwrapped. SB: VisitPrimary wraps in parens.
@@ -1271,7 +1276,7 @@ func TestVisitComparison_Parens(t *testing.T) {
// rsbOpts has SkipFullTextFilter=true → TrueConditionLiteral.
// sbOpts has SkipFullTextFilter=false, FullTextColumn=bodyCol → "body_cond".
func TestVisitComparison_FullText(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
name: "standalone full-text term",
@@ -1428,7 +1433,7 @@ func TestVisitComparison_FullText(t *testing.T) {
// Equality with __all__ does NOT short-circuit — the variable resolves to the literal
// "__all__" string and ConditionFor is called normally.
func TestVisitComparison_AllVariable(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
name: "IN allVariable alone",
@@ -1536,7 +1541,7 @@ func TestVisitComparison_AllVariable(t *testing.T) {
// sbOpts has SkipFunctionCalls=false; has/hasAny/hasAll only support FieldContextBody,
// so calls on attribute/resource keys return an error.
func TestVisitComparison_FunctionCalls(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
name: "has on attribute key",
@@ -1615,7 +1620,7 @@ func TestVisitComparison_FunctionCalls(t *testing.T) {
// (no keys resolved); SkipConditionLiteral short-circuits OR and is stripped from AND.
// sbOpts has IgnoreNotFoundKeys=false → key lookup appends an error.
func TestVisitComparison_UnknownKeys(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{
// RSB: unknown_key → SkipConditionLiteral (no keys resolved); stripped from AND; x_cond survives.
@@ -1682,7 +1687,7 @@ func TestVisitComparison_UnknownKeys(t *testing.T) {
// TestVisitComparison_SkippableLiteralValues guards against two distinct collision risks
// involving SkippableConditionLiterals ("true", "__skip__", "__skip_because_of_error__"):.
func TestVisitComparison_SkippableLiteralValues(t *testing.T) {
rsbOpts, sbOpts := visitComparisonOpts()
rsbOpts, sbOpts := visitComparisonOpts(t)
tests := []visitComparisonCase{
{

View File

@@ -52,7 +52,7 @@ func TestNewHandlers(t *testing.T) {
userRoleStore := impluser.NewUserRoleStore(sqlstore, providerSettings)
userGetter := impluser.NewGetter(impluser.NewStore(sqlstore, providerSettings), userRoleStore, flagger)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, nil, nil)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, nil, nil, flagger)
querierHandler := querier.NewHandler(providerSettings, nil, nil)
registryHandler := factory.NewHandler(nil)

View File

@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/emailing"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/modules/apdex"
"github.com/SigNoz/signoz/pkg/modules/apdex/implapdex"
"github.com/SigNoz/signoz/pkg/modules/authdomain"
@@ -102,6 +103,7 @@ func NewModules(
userRoleStore authtypes.UserRoleStore,
serviceAccount serviceaccount.Module,
cloudIntegrationModule cloudintegration.Module,
fl flagger.Flagger,
) Modules {
quickfilter := implquickfilter.NewModule(implquickfilter.NewStore(sqlstore))
orgSetter := implorganization.NewSetter(implorganization.NewStore(sqlstore), alertmanager, quickfilter)

View File

@@ -56,7 +56,7 @@ func TestNewModules(t *testing.T) {
serviceAccount := implserviceaccount.NewModule(implserviceaccount.NewStore(sqlstore), nil, nil, nil, providerSettings, serviceaccount.Config{})
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, serviceAccount, implcloudintegration.NewModule())
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, nil, nil, nil, nil, nil, nil, nil, queryParser, Config{}, dashboardModule, userGetter, userRoleStore, serviceAccount, implcloudintegration.NewModule(), flagger)
reflectVal := reflect.ValueOf(modules)
for i := 0; i < reflectVal.NumField(); i++ {

View File

@@ -98,8 +98,31 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
return defaultDefName
}))
reflector.SpecSchema().SetTitle("SigNoz")
reflector.SpecSchema().SetDescription("OpenTelemetry-Native Logs, Metrics and Traces in a single pane")
reflector.Spec.WithInfo(*(&openapi3.Info{}).
WithTitle("SigNoz").
WithDescription("OpenTelemetry-Native Logs, Metrics and Traces in a single pane").
WithTermsOfService("https://signoz.io/terms-of-service/").
WithContact(*(&openapi3.Contact{}).
WithName("SigNoz Support").
WithURL("https://signoz.io").
WithEmail("support@signoz.io")),
)
reflector.Spec.WithServers(
// Default server
*(&openapi3.Server{}).WithURL("https://{host}:{port}{base_path}").
WithDescription("The fully qualified URL to the SigNoz APIServer.").
WithVariablesItem("host", *(&openapi3.ServerVariable{}).
WithDefault("localhost").
WithDescription("The host of the SigNoz APIServer")).
WithVariablesItem("port", *(&openapi3.ServerVariable{}).
WithDefault("8080").
WithDescription("The port of the SigNoz APIServer")).
WithVariablesItem("base_path", *(&openapi3.ServerVariable{}).
WithDefault("/").
WithDescription("The base path of the SigNoz APIServer")),
)
reflector.SpecSchema().SetAPIKeySecurity(authtypes.IdentNProviderAPIKey.StringValue(), "SigNoz-Api-Key", openapi.InHeader, "API Keys")
reflector.SpecSchema().SetHTTPBearerTokenSecurity(authtypes.IdentNProviderTokenizer.StringValue(), "Tokenizer", "Tokens generated by the tokenizer")

View File

@@ -419,6 +419,7 @@ func New(
telemetrymetadata.DBName,
telemetrymetadata.AttributesMetadataLocalTableName,
telemetrymetadata.ColumnEvolutionMetadataTableName,
flagger,
)
global, err := factory.NewProviderFromNamedMap(
@@ -440,7 +441,7 @@ func New(
}
// Initialize all modules
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, analytics, querier, telemetrystore, telemetryMetadataStore, authNs, authz, cache, queryParser, config, dashboard, userGetter, userRoleStore, serviceAccount, cloudIntegrationModule)
modules := NewModules(sqlstore, tokenizer, emailing, providerSettings, orgGetter, alertmanager, analytics, querier, telemetrystore, telemetryMetadataStore, authNs, authz, cache, queryParser, config, dashboard, userGetter, userRoleStore, serviceAccount, cloudIntegrationModule, flagger)
// Initialize ruler from the variant-specific provider factories
rulerInstance, err := factory.NewProviderFromNamedMap(ctx, providerSettings, config.Ruler, rulerProviderFactories(cache, alertmanager, sqlstore, telemetrystore, telemetryMetadataStore, prometheus, orgGetter, modules.RuleStateHistory, querier, queryParser), "signoz")

View File

@@ -8,6 +8,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -36,6 +37,7 @@ func NewAuditQueryStatementBuilder(
aggExprRewriter qbtypes.AggExprRewriter,
fullTextColumn *telemetrytypes.TelemetryFieldKey,
jsonKeyToKey qbtypes.JsonKeyToFieldFunc,
flagger flagger.Flagger,
) *auditQueryStatementBuilder {
auditSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryaudit")
@@ -48,6 +50,7 @@ func NewAuditQueryStatementBuilder(
metadataStore,
fullTextColumn,
jsonKeyToKey,
flagger,
)
return &auditQueryStatementBuilder{
@@ -319,7 +322,7 @@ func (b *auditQueryStatementBuilder) buildTimeSeriesQuery(
fieldNames := make([]string, 0, len(query.GroupBy))
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, false)
if err != nil {
return nil, err
}
@@ -456,7 +459,7 @@ func (b *auditQueryStatementBuilder) buildScalarQuery(
var allGroupByArgs []any
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, false)
if err != nil {
return nil, err
}

View File

@@ -10,6 +10,7 @@ import (
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/stretchr/testify/require"
)
@@ -46,13 +47,15 @@ func auditFieldKeyMap() map[string][]*telemetrytypes.TelemetryFieldKey {
}
}
func newTestAuditStatementBuilder() *auditQueryStatementBuilder {
func newTestAuditStatementBuilder(t *testing.T) *auditQueryStatementBuilder {
t.Helper()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = auditFieldKeyMap()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
return NewAuditQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -62,11 +65,12 @@ func newTestAuditStatementBuilder() *auditQueryStatementBuilder {
aggExprRewriter,
DefaultFullTextColumn,
nil,
fl,
)
}
func TestStatementBuilder(t *testing.T) {
statementBuilder := newTestAuditStatementBuilder()
statementBuilder := newTestAuditStatementBuilder(t)
ctx := context.Background()
testCases := []struct {

View File

@@ -6,19 +6,23 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
type conditionBuilder struct {
fm qbtypes.FieldMapper
fl flagger.Flagger
}
func NewConditionBuilder(fm qbtypes.FieldMapper) *conditionBuilder {
return &conditionBuilder{fm: fm}
func NewConditionBuilder(fm qbtypes.FieldMapper, fl flagger.Flagger) *conditionBuilder {
return &conditionBuilder{fm: fm, fl: fl}
}
func (c *conditionBuilder) conditionFor(
@@ -36,7 +40,8 @@ func (c *conditionBuilder) conditionFor(
// TODO(Piyush): Update this to support multiple JSON columns based on evolutions
for _, column := range columns {
if column.Type.GetType() == schema.ColumnTypeEnumJSON && querybuilder.BodyJSONQueryEnabled && key.Name != messageSubField {
// TODO(Tushar): thread orgID here to evaluate correctly
if column.Type.GetType() == schema.ColumnTypeEnumJSON && c.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) && key.Name != messageSubField {
valueType, value := InferDataType(value, operator, key)
cond, err := NewJSONConditionBuilder(key, valueType).buildJSONCondition(operator, value, sb)
if err != nil {
@@ -56,7 +61,8 @@ func (c *conditionBuilder) conditionFor(
}
// Check if this is a body JSON search - either by FieldContext
if key.FieldContext == telemetrytypes.FieldContextBody && !querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if key.FieldContext == telemetrytypes.FieldContextBody && !c.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) {
fieldExpression, value = GetBodyJSONKey(ctx, key, operator, value)
}
@@ -167,7 +173,8 @@ func (c *conditionBuilder) conditionFor(
// in the UI based query builder, `exists` and `not exists` are used for
// key membership checks, so depending on the column type, the condition changes
case qbtypes.FilterOperatorExists, qbtypes.FilterOperatorNotExists:
if key.FieldContext == telemetrytypes.FieldContextBody && !querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if key.FieldContext == telemetrytypes.FieldContextBody && !c.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) {
if operator == qbtypes.FilterOperatorExists {
return GetBodyJSONKeyForExists(ctx, key, operator, value), nil
} else {
@@ -287,7 +294,8 @@ func (c *conditionBuilder) ConditionFor(
case telemetrytypes.FieldContextBody:
// Querying JSON fields already account for Nullability of fields
// so additional exists checks are not needed
if querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if c.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) {
return condition, nil
}
}

View File

@@ -5,6 +5,7 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/huandu/go-sqlbuilder"
@@ -122,8 +123,9 @@ func TestExistsConditionForWithEvolutions(t *testing.T) {
expectedError: nil,
},
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
conditionBuilder := NewConditionBuilder(fm, fl)
ctx := context.Background()
for _, tc := range testCases {
@@ -513,8 +515,9 @@ func TestConditionFor(t *testing.T) {
expectedError: qbtypes.ErrColumnNotFound,
},
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
conditionBuilder := NewConditionBuilder(fm, fl)
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
t.Run(tc.name, func(t *testing.T) {
@@ -566,8 +569,9 @@ func TestConditionForMultipleKeys(t *testing.T) {
},
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
conditionBuilder := NewConditionBuilder(fm, fl)
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()
@@ -825,8 +829,9 @@ func TestConditionForJSONBodySearch(t *testing.T) {
},
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
conditionBuilder := NewConditionBuilder(fm, fl)
for _, tc := range testCases {
sb := sqlbuilder.NewSelectBuilder()

View File

@@ -4,7 +4,6 @@ import (
"fmt"
"github.com/SigNoz/signoz-otel-collector/constants"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
@@ -41,7 +40,7 @@ const (
BodyPromotedColumnPrefix = constants.BodyPromotedColumnPrefix
// messageSubColumn is the ClickHouse sub-column that body searches map to
// when BodyJSONQueryEnabled is true.
// when body_json_enabled feature flag is true.
messageSubField = "message"
messageSubColumn = "body_v2.message"
bodySearchDefaultWarning = "body searches default to `body.message:string`. Use `body.<key>` to search a different field inside body"
@@ -128,8 +127,8 @@ var (
}
)
func bodyAliasExpression() string {
if !querybuilder.BodyJSONQueryEnabled {
func bodyAliasExpression(bodyJSONEnabled bool) string {
if !bodyJSONEnabled {
return LogsV2BodyColumn
}

View File

@@ -12,9 +12,11 @@ import (
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz-otel-collector/utils"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
"golang.org/x/exp/maps"
@@ -66,13 +68,15 @@ var (
}
)
type fieldMapper struct{}
func NewFieldMapper() qbtypes.FieldMapper {
return &fieldMapper{}
type fieldMapper struct {
fl flagger.Flagger
}
func (m *fieldMapper) getColumn(_ context.Context, key *telemetrytypes.TelemetryFieldKey) ([]*schema.Column, error) {
func NewFieldMapper(fl flagger.Flagger) qbtypes.FieldMapper {
return &fieldMapper{fl: fl}
}
func (m *fieldMapper) getColumn(ctx context.Context, key *telemetrytypes.TelemetryFieldKey) ([]*schema.Column, error) {
switch key.FieldContext {
case telemetrytypes.FieldContextResource:
columns := []*schema.Column{logsV2Columns["resources_string"], logsV2Columns["resource"]}
@@ -96,7 +100,8 @@ func (m *fieldMapper) getColumn(_ context.Context, key *telemetrytypes.Telemetry
}
case telemetrytypes.FieldContextBody:
// Body context is for JSON body fields. Use body_v2 if feature flag is enabled.
if querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if m.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) {
if key.Name == messageSubField {
return []*schema.Column{logsV2Columns[messageSubColumn]}, nil
}
@@ -105,7 +110,8 @@ func (m *fieldMapper) getColumn(_ context.Context, key *telemetrytypes.Telemetry
// Fall back to legacy body column
return []*schema.Column{logsV2Columns["body"]}, nil
case telemetrytypes.FieldContextLog, telemetrytypes.FieldContextUnspecified:
if key.Name == LogsV2BodyColumn && querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if key.Name == LogsV2BodyColumn && m.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) {
return []*schema.Column{logsV2Columns[messageSubColumn]}, nil
}
col, ok := logsV2Columns[key.Name]
@@ -113,7 +119,8 @@ func (m *fieldMapper) getColumn(_ context.Context, key *telemetrytypes.Telemetry
// check if the key has body JSON search
if strings.HasPrefix(key.Name, telemetrytypes.BodyJSONStringSearchPrefix) {
// Use body_v2 if feature flag is enabled and we have a body condition builder
if querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if m.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) {
// TODO(Piyush): Update this to support multiple JSON columns based on evolutions
// i.e return both the body json and body json promoted and let the evolutions decide which one to use
// based on the query range time.

View File

@@ -6,6 +6,7 @@ import (
"time"
schema "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/stretchr/testify/assert"
@@ -165,7 +166,8 @@ func TestGetColumn(t *testing.T) {
},
}
fm := NewFieldMapper()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
@@ -273,7 +275,8 @@ func TestGetFieldKeyName(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fm := NewFieldMapper()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
result, err := fm.FieldFor(ctx, 0, 0, &tc.key)
if tc.expectedError != nil {
@@ -514,7 +517,8 @@ func TestFieldForWithEvolutions(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fm := NewFieldMapper()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
tsStart := uint64(tc.tsStartTime.UnixNano())
tsEnd := uint64(tc.tsEndTime.UnixNano())
@@ -963,7 +967,8 @@ func TestFieldForWithMaterialized(t *testing.T) {
},
}
fm := NewFieldMapper()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {

View File

@@ -5,6 +5,7 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/stretchr/testify/require"
@@ -12,9 +13,10 @@ import (
// TestLikeAndILikeWithoutWildcards_Warns Tests that LIKE/ILIKE without wildcards add warnings and include docs URL.
func TestLikeAndILikeWithoutWildcards_Warns(t *testing.T) {
fl := flaggertest.New(t)
ctx := context.Background()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
keys := buildCompleteFieldKeyMap(releaseTime)
@@ -51,8 +53,9 @@ func TestLikeAndILikeWithoutWildcards_Warns(t *testing.T) {
// TestLikeAndILikeWithWildcards_NoWarn Tests that LIKE/ILIKE with wildcards do not add warnings.
func TestLikeAndILikeWithWildcards_NoWarn(t *testing.T) {
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
keys := buildCompleteFieldKeyMap(releaseTime)

View File

@@ -6,6 +6,7 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -15,8 +16,9 @@ import (
// TestFilterExprLogsBodyJSON tests a comprehensive set of query patterns for body JSON search.
func TestFilterExprLogsBodyJSON(t *testing.T) {
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
// Define a comprehensive set of field keys to support all test cases
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
keys := buildCompleteFieldKeyMap(releaseTime)
@@ -27,10 +29,8 @@ func TestFilterExprLogsBodyJSON(t *testing.T) {
FieldMapper: fm,
ConditionBuilder: cb,
FieldKeys: keys,
FullTextColumn: &telemetrytypes.TelemetryFieldKey{
Name: "body",
},
JsonKeyToKey: GetBodyJSONKey,
FullTextColumn: &telemetrytypes.TelemetryFieldKey{Name: "body"},
JsonKeyToKey: GetBodyJSONKey,
}
testCases := []struct {

View File

@@ -8,6 +8,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -17,10 +18,11 @@ import (
// TestFilterExprLogs tests a comprehensive set of query patterns for logs search.
func TestFilterExprLogs(t *testing.T) {
fl := flaggertest.New(t)
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
ctx := context.Background()
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
// Define a comprehensive set of field keys to support all test cases
keys := buildCompleteFieldKeyMap(releaseTime)
@@ -2429,8 +2431,9 @@ func TestFilterExprLogs(t *testing.T) {
// TestFilterExprLogs tests a comprehensive set of query patterns for logs search.
func TestFilterExprLogsConflictNegation(t *testing.T) {
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
// Define a comprehensive set of field keys to support all test cases
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)

View File

@@ -8,6 +8,7 @@ import (
"time"
schemamigrator "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -32,9 +33,6 @@ func (t TestExpected) GetQuery() string {
}
func TestJSONStmtBuilder_TimeSeries(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, false)
cases := []struct {
@@ -115,9 +113,6 @@ func TestJSONStmtBuilder_TimeSeries(t *testing.T) {
not a body_promoted.* column. These tests assumed the old coalesce(body_promoted.x, body_v2.x) path.
func TestStmtBuilderTimeSeriesBodyGroupByPromoted(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, "user.age", "user.name")
cases := []struct {
@@ -176,10 +171,6 @@ func TestStmtBuilderTimeSeriesBodyGroupByPromoted(t *testing.T) {
*/
func TestJSONStmtBuilder_PrimitivePaths(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, false)
cases := []struct {
name string
@@ -340,10 +331,6 @@ func TestJSONStmtBuilder_PrimitivePaths(t *testing.T) {
(direct sub-column access), not a body_promoted.* column.
func TestStatementBuilderListQueryBodyPromoted(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, "education", "tags")
cases := []struct {
name string
@@ -507,10 +494,6 @@ func TestStatementBuilderListQueryBodyPromoted(t *testing.T) {
*/
func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, false)
cases := []struct {
name string
@@ -816,10 +799,6 @@ func TestJSONStmtBuilder_ArrayPaths(t *testing.T) {
}
func TestJSONStmtBuilder_IndexedPaths(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, true)
cases := []struct {
name string
@@ -939,9 +918,6 @@ func TestJSONStmtBuilder_IndexedPaths(t *testing.T) {
}
func TestJSONStmtBuilder_SelectField(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, false)
cases := []struct {
@@ -1030,9 +1006,6 @@ func TestJSONStmtBuilder_SelectField(t *testing.T) {
}
func TestJSONStmtBuilder_OrderBy(t *testing.T) {
enable, disable := jsonQueryTestUtil(t)
enable()
defer disable()
statementBuilder := buildJSONTestStatementBuilder(t, false)
cases := []struct {
@@ -1151,11 +1124,14 @@ func buildTestTelemetryMetadataStore(t *testing.T, addIndexes bool) *telemetryty
}
func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) *logQueryStatementBuilder {
mockMetadataStore := buildTestTelemetryMetadataStore(t, addIndexes)
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
t.Helper()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
mockMetadataStore := buildTestTelemetryMetadataStore(t, addIndexes)
fl := flaggertest.WithBodyJSON(t, true)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -1165,18 +1141,8 @@ func buildJSONTestStatementBuilder(t *testing.T, addIndexes bool) *logQueryState
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
return statementBuilder
}
func jsonQueryTestUtil(_ *testing.T) (func(), func()) {
enable := func() {
querybuilder.BodyJSONQueryEnabled = true
}
disable := func() {
querybuilder.BodyJSONQueryEnabled = false
}
return enable, disable
}

View File

@@ -8,10 +8,13 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -22,6 +25,7 @@ type logQueryStatementBuilder struct {
cb qbtypes.ConditionBuilder
resourceFilterStmtBuilder qbtypes.StatementBuilder[qbtypes.LogAggregation]
aggExprRewriter qbtypes.AggExprRewriter
fl flagger.Flagger
fullTextColumn *telemetrytypes.TelemetryFieldKey
jsonKeyToKey qbtypes.JsonKeyToFieldFunc
@@ -37,6 +41,7 @@ func NewLogQueryStatementBuilder(
aggExprRewriter qbtypes.AggExprRewriter,
fullTextColumn *telemetrytypes.TelemetryFieldKey,
jsonKeyToKey qbtypes.JsonKeyToFieldFunc,
fl flagger.Flagger,
) *logQueryStatementBuilder {
logsSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrylogs")
@@ -49,6 +54,7 @@ func NewLogQueryStatementBuilder(
metadataStore,
fullTextColumn,
jsonKeyToKey,
fl,
)
return &logQueryStatementBuilder{
@@ -58,6 +64,7 @@ func NewLogQueryStatementBuilder(
cb: conditionBuilder,
resourceFilterStmtBuilder: resourceFilterStmtBuilder,
aggExprRewriter: aggExprRewriter,
fl: fl,
fullTextColumn: fullTextColumn,
jsonKeyToKey: jsonKeyToKey,
}
@@ -75,8 +82,10 @@ func (b *logQueryStatementBuilder) Build(
start = querybuilder.ToNanoSecs(start)
end = querybuilder.ToNanoSecs(end)
// TODO(Tushar): thread orgID here to evaluate correctly
bodyJSONEnabled := b.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
keySelectors, warnings := getKeySelectors(query)
keySelectors, warnings := getKeySelectors(ctx, query, bodyJSONEnabled)
keys, _, err := b.metadataStore.GetKeysMulti(ctx, keySelectors)
if err != nil {
return nil, err
@@ -107,7 +116,7 @@ func (b *logQueryStatementBuilder) Build(
return stmt, nil
}
func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]) ([]*telemetrytypes.FieldKeySelector, []string) {
func getKeySelectors(ctx context.Context, query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation], bodyJSONEnabled bool) ([]*telemetrytypes.FieldKeySelector, []string) {
var keySelectors []*telemetrytypes.FieldKeySelector
var warnings []string
@@ -159,7 +168,7 @@ func getKeySelectors(query qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]) ([
// When the new JSON body experience is enabled, warn the user if they use the bare
// "body" key in the filter — queries on plain "body" default to body.message:string.
// TODO(Piyush): Setup better for coming FTS support.
if querybuilder.BodyJSONQueryEnabled {
if bodyJSONEnabled {
for _, sel := range keySelectors {
if sel.Name == LogsV2BodyColumn {
warnings = append(warnings, bodySearchDefaultWarning)
@@ -258,6 +267,8 @@ func (b *logQueryStatementBuilder) buildListQuery(
var (
cteFragments []string
cteArgs [][]any
// TODO(Tushar): thread orgID here to evaluate correctly
bodyJSONEnabled = b.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
@@ -279,7 +290,7 @@ func (b *logQueryStatementBuilder) buildListQuery(
sb.SelectMore(LogsV2SeverityNumberColumn)
sb.SelectMore(LogsV2ScopeNameColumn)
sb.SelectMore(LogsV2ScopeVersionColumn)
sb.SelectMore(bodyAliasExpression())
sb.SelectMore(bodyAliasExpression(bodyJSONEnabled))
sb.SelectMore(LogsV2AttributesStringColumn)
sb.SelectMore(LogsV2AttributesNumberColumn)
sb.SelectMore(LogsV2AttributesBoolColumn)
@@ -360,6 +371,8 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
var (
cteFragments []string
cteArgs [][]any
// TODO(Tushar): thread orgID here to evaluate correctly
bodyJSONEnabled = b.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
@@ -379,7 +392,7 @@ func (b *logQueryStatementBuilder) buildTimeSeriesQuery(
// Keep original column expressions so we can build the tuple
fieldNames := make([]string, 0, len(query.GroupBy))
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return nil, err
}
@@ -518,6 +531,8 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
var (
cteFragments []string
cteArgs [][]any
// TODO(Tushar): thread orgID here to evaluate correctly
bodyJSONEnabled = b.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
)
if frag, args, err := b.maybeAttachResourceFilter(ctx, sb, query, start, end, variables); err != nil {
@@ -532,7 +547,7 @@ func (b *logQueryStatementBuilder) buildScalarQuery(
var allGroupByArgs []any
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, b.jsonKeyToKey, bodyJSONEnabled)
if err != nil {
return nil, err
}
@@ -635,6 +650,8 @@ func (b *logQueryStatementBuilder) addFilterCondition(
var preparedWhereClause *querybuilder.PreparedWhereClause
var err error
// TODO(Tushar): thread orgID here to evaluate correctly
bodyJSONEnabled := b.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
if query.Filter != nil && query.Filter.Expression != "" {
// add filter expression
@@ -644,6 +661,7 @@ func (b *logQueryStatementBuilder) addFilterCondition(
FieldMapper: b.fm,
ConditionBuilder: b.cb,
FieldKeys: keys,
BodyJSONEnabled: bodyJSONEnabled,
SkipResourceFilter: true,
FullTextColumn: b.fullTextColumn,
JsonKeyToKey: b.jsonKeyToKey,

View File

@@ -6,6 +6,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -15,7 +16,6 @@ import (
)
func TestStatementBuilderTimeSeries(t *testing.T) {
// Create a test release time
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
releaseTimeNano := uint64(releaseTime.UnixNano())
@@ -191,16 +191,17 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
}
ctx := context.Background()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
keysMap := buildCompleteFieldKeyMap(releaseTime)
mockMetadataStore.KeysMap = keysMap
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -210,6 +211,7 @@ func TestStatementBuilderTimeSeries(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
for _, c := range cases {
@@ -313,15 +315,16 @@ func TestStatementBuilderListQuery(t *testing.T) {
}
ctx := context.Background()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fm := NewFieldMapper()
fm := NewFieldMapper(fl)
// Create a test release time
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
cb := NewConditionBuilder(fm)
cb := NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -331,6 +334,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
for _, c := range cases {
@@ -454,14 +458,15 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
}
ctx := context.Background()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fm := NewFieldMapper()
fm := NewFieldMapper(fl)
// Create a test release time
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
cb := NewConditionBuilder(fm)
cb := NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -471,6 +476,7 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
for _, c := range cases {
@@ -528,14 +534,15 @@ func TestStatementBuilderTimeSeriesBodyGroupBy(t *testing.T) {
}
ctx := context.Background()
fl := flaggertest.New(t)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fm := NewFieldMapper()
fm := NewFieldMapper(fl)
// Create a test release time
releaseTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap(releaseTime)
cb := NewConditionBuilder(fm)
cb := NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -545,6 +552,7 @@ func TestStatementBuilderTimeSeriesBodyGroupBy(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
for _, c := range cases {
@@ -624,11 +632,12 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
ctx := context.Background()
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
fm := NewFieldMapper()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
mockMetadataStore.KeysMap = buildCompleteFieldKeyMapCollision()
cb := NewConditionBuilder(fm)
cb := NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -638,6 +647,7 @@ func TestStatementBuilderListQueryServiceCollision(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
for _, c := range cases {
@@ -845,12 +855,13 @@ func TestAdjustKey(t *testing.T) {
},
}
fm := NewFieldMapper()
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMapCollision()
cb := NewConditionBuilder(fm)
cb := NewConditionBuilder(fm, fl)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -860,6 +871,7 @@ func TestAdjustKey(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
for _, c := range cases {
@@ -984,25 +996,18 @@ func TestStmtBuilderBodyField(t *testing.T) {
},
}
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
enable, disable := jsonQueryTestUtil(t)
defer disable()
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
if c.enableBodyJSONQuery {
enable()
} else {
disable()
}
// build the key map after enabling/disabling body JSON query
fl := flaggertest.WithBodyJSON(t, c.enableBodyJSONQuery)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
// build the key map
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
for _, field := range IntrinsicFields {
f := field
mockMetadataStore.KeysMap[field.Name] = append(mockMetadataStore.KeysMap[field.Name], &f)
}
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
@@ -1011,6 +1016,7 @@ func TestStmtBuilderBodyField(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
@@ -1072,25 +1078,18 @@ func TestStmtBuilderBodyFullTextSearch(t *testing.T) {
},
}
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
enable, disable := jsonQueryTestUtil(t)
defer disable()
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
if c.enableBodyJSONQuery {
enable()
} else {
disable()
}
// build the key map after enabling/disabling body JSON query
fl := flaggertest.WithBodyJSON(t, c.enableBodyJSONQuery)
fm := NewFieldMapper(fl)
cb := NewConditionBuilder(fm, fl)
// build the key map
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
for _, field := range IntrinsicFields {
f := field
mockMetadataStore.KeysMap[field.Name] = append(mockMetadataStore.KeysMap[field.Name], &f)
}
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewLogQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
@@ -1099,6 +1098,7 @@ func TestStmtBuilderBodyFullTextSearch(t *testing.T) {
aggExprRewriter,
DefaultFullTextColumn,
GetBodyJSONKey,
fl,
)
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)

View File

@@ -12,6 +12,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryaudit"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
@@ -19,10 +20,12 @@ import (
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
var (
@@ -63,6 +66,7 @@ type telemetryMetaStore struct {
fm qbtypes.FieldMapper
conditionBuilder qbtypes.ConditionBuilder
fl flagger.Flagger
jsonColumnMetadata map[telemetrytypes.Signal]map[telemetrytypes.FieldContext]telemetrytypes.JSONColumnMetadata
}
@@ -94,9 +98,13 @@ func NewTelemetryMetaStore(
relatedMetadataDBName string,
relatedMetadataTblName string,
columnEvolutionMetadataTblName string,
fl flagger.Flagger,
) telemetrytypes.MetadataStore {
metadataSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrymetadata")
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
t := &telemetryMetaStore{
logger: metadataSettings.Logger(),
telemetrystore: telemetrystore,
@@ -129,14 +137,11 @@ func NewTelemetryMetaStore(
},
},
},
fl: fl,
fm: fm,
conditionBuilder: conditionBuilder,
}
fm := NewFieldMapper()
conditionBuilder := NewConditionBuilder(fm)
t.fm = fm
t.conditionBuilder = conditionBuilder
return t
}
@@ -416,7 +421,8 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, fieldKeySelectors
}
// body keys are gated behind the feature flag
queryBodyTable = queryBodyTable && querybuilder.BodyJSONQueryEnabled
// TODO(Tushar): thread orgID here to evaluate correctly
queryBodyTable = queryBodyTable && t.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
// requestedFieldKeySelectors is the set of names the user explicitly asked for.
// Used to ensure a name that is both a parent path AND a directly requested field still surfaces
@@ -676,7 +682,8 @@ func (t *telemetryMetaStore) getLogsKeys(ctx context.Context, fieldKeySelectors
}
// enrich body keys with promoted paths, indexes, and JSON access plans
if querybuilder.BodyJSONQueryEnabled {
// TODO(Tushar): thread orgID here to evaluate correctly
if t.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{})) {
if err := t.enrichJSONKeys(ctx, fieldKeySelectors, keys, parentTypes); err != nil {
return nil, false, err
}
@@ -1360,6 +1367,9 @@ func (t *telemetryMetaStore) getRelatedValues(ctx context.Context, fieldValueSel
instrumentationtypes.CodeFunctionName: "getRelatedValues",
})
// TODO(Tushar): thread orgID here to evaluate correctly
bodyJSONEnabled := t.fl.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
// nothing to return as "related" value if there is nothing to filter on
if fieldValueSelector.ExistingQuery == "" {
return nil, true, nil
@@ -1409,6 +1419,7 @@ func (t *telemetryMetaStore) getRelatedValues(ctx context.Context, fieldValueSel
FieldMapper: t.fm,
ConditionBuilder: t.conditionBuilder,
FieldKeys: keys,
BodyJSONEnabled: bodyJSONEnabled,
})
if err != nil {
t.logger.WarnContext(ctx, "error parsing existing query for related values", errors.Attr(err))

View File

@@ -15,6 +15,7 @@ import (
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
cmock "github.com/srikanthccv/ClickHouse-go-mock"
"github.com/stretchr/testify/assert"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/stretchr/testify/require"
)
@@ -46,6 +47,7 @@ func TestGetFirstSeenFromMetricMetadata(t *testing.T) {
DBName,
AttributesMetadataLocalTableName,
ColumnEvolutionMetadataTableName,
flaggertest.New(t),
)
lookupKeys := []telemetrytypes.MetricMetadataLookupKey{

View File

@@ -17,10 +17,12 @@ import (
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
cmock "github.com/srikanthccv/ClickHouse-go-mock"
"github.com/stretchr/testify/assert"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/stretchr/testify/require"
)
func newTestTelemetryMetaStoreTestHelper(store telemetrystore.TelemetryStore) telemetrytypes.MetadataStore {
func newTestTelemetryMetaStoreTestHelper(t *testing.T, store telemetrystore.TelemetryStore) telemetrytypes.MetadataStore {
t.Helper()
return NewTelemetryMetaStore(
instrumentationtest.New().ToProviderSettings(),
store,
@@ -45,6 +47,7 @@ func newTestTelemetryMetaStoreTestHelper(store telemetrystore.TelemetryStore) te
DBName,
AttributesMetadataLocalTableName,
ColumnEvolutionMetadataTableName,
flaggertest.New(t),
)
}
@@ -66,7 +69,7 @@ func TestGetKeys(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
rows := cmock.NewRows([]cmock.ColumnType{
{Name: "statement", Type: "String"},
@@ -176,7 +179,7 @@ func TestApplyBackwardCompatibleKeys(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
hasTraces := false
hasLogs := false
@@ -340,7 +343,7 @@ func TestGetMetricFieldValuesIntrinsicMetricName(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
valueRows := cmock.NewRows([]cmock.ColumnType{
{Name: "metric_name", Type: "String"},
@@ -379,7 +382,7 @@ func TestGetMetricFieldValuesIntrinsicBoolReturnsEmpty(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
metadataRows := cmock.NewRows([]cmock.ColumnType{
{Name: "attr_string_value", Type: "String"},
@@ -411,7 +414,7 @@ func TestGetMetricFieldValuesAppliesMetricNamespace(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
valueRows := cmock.NewRows([]cmock.ColumnType{
{Name: "attr_string_value", Type: "String"},
@@ -443,7 +446,7 @@ func TestGetMetricFieldValuesIntrinsicMetricNameAppliesMetricNamespace(t *testin
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
valueRows := cmock.NewRows([]cmock.ColumnType{
{Name: "metric_name", Type: "String"},
@@ -483,7 +486,7 @@ func TestGetMeterSourceMetricFieldValuesAppliesMetricNamespace(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
rows := cmock.NewRows([]cmock.ColumnType{
{Name: "attr", Type: "Array(String)"},
@@ -514,7 +517,7 @@ func TestGetMetricsKeysAppliesMetricNamespace(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
rows := cmock.NewRows([]cmock.ColumnType{
{Name: "name", Type: "String"},
@@ -549,7 +552,7 @@ func TestGetMeterSourceMetricKeysAppliesMetricNamespace(t *testing.T) {
mockTelemetryStore := telemetrystoretest.New(telemetrystore.Config{}, &regexMatcher{})
mock := mockTelemetryStore.Mock()
metadata := newTestTelemetryMetaStoreTestHelper(mockTelemetryStore)
metadata := newTestTelemetryMetaStoreTestHelper(t, mockTelemetryStore)
rows := cmock.NewRows([]cmock.ColumnType{
{Name: "attr_name", Type: "String"},

View File

@@ -5,7 +5,7 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
@@ -166,10 +166,7 @@ func TestStatementBuilder(t *testing.T) {
}
mockMetadataStore.KeysMap = keys
flagger, err := flagger.New(context.Background(), instrumentationtest.New().ToProviderSettings(), flagger.Config{}, flagger.MustNewRegistry())
if err != nil {
t.Fatalf("failed to create flagger: %v", err)
}
flagger := flaggertest.New(t)
metricStmtBuilder := telemetrymetrics.NewMetricQueryStatementBuilder(instrumentationtest.New().ToProviderSettings(), mockMetadataStore, fm, cb, flagger)

View File

@@ -124,6 +124,17 @@ func CountExpressionForSamplesTable(tableName string) string {
return "sum(count)"
}
// ValueColumnForSamplesTable returns the column name holding the sample value:
// "last" for the 5m/30m aggregated tables, "value" otherwise.
// note all the other columns in the aggregated samples tables are nothing but aggregations.
// and so "last" is the value column for these tables.
func ValueColumnForSamplesTable(tableName string) string {
if tableName == SamplesV4Agg5mTableName || tableName == SamplesV4Agg30mTableName {
return "last"
}
return "value"
}
// start and end are in milliseconds
// we have three tables for samples
// 1. distributed_samples_v4

View File

@@ -6,9 +6,12 @@ import (
"log/slog"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/types/featuretypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/huandu/go-sqlbuilder"
)
@@ -22,6 +25,7 @@ type resourceFilterStatementBuilder[T any] struct {
metadataStore telemetrytypes.MetadataStore
signal telemetrytypes.Signal
source telemetrytypes.Source
flagger flagger.Flagger
fullTextColumn *telemetrytypes.TelemetryFieldKey
jsonKeyToKey qbtypes.JsonKeyToFieldFunc
@@ -42,6 +46,7 @@ func New[T any](
metadataStore telemetrytypes.MetadataStore,
fullTextColumn *telemetrytypes.TelemetryFieldKey,
jsonKeyToKey qbtypes.JsonKeyToFieldFunc,
fl flagger.Flagger,
) *resourceFilterStatementBuilder[T] {
set := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetryresourcefilter")
fm := NewFieldMapper()
@@ -55,6 +60,7 @@ func New[T any](
metadataStore: metadataStore,
signal: signal,
source: source,
flagger: fl,
fullTextColumn: fullTextColumn,
jsonKeyToKey: jsonKeyToKey,
}
@@ -128,6 +134,10 @@ func (b *resourceFilterStatementBuilder[T]) addConditions(
keys map[string][]*telemetrytypes.TelemetryFieldKey,
variables map[string]qbtypes.VariableItem,
) (bool, error) {
// TODO(Tushar): thread orgID here to evaluate correctly
bodyJSONEnabled := b.flagger.BooleanOrEmpty(ctx, flagger.FeatureBodyJSONQuery, featuretypes.NewFlaggerEvaluationContext(valuer.UUID{}))
// Add filter condition if present
if query.Filter != nil && query.Filter.Expression != "" {
@@ -138,6 +148,7 @@ func (b *resourceFilterStatementBuilder[T]) addConditions(
FieldMapper: b.fieldMapper,
ConditionBuilder: b.conditionBuilder,
FieldKeys: keys,
BodyJSONEnabled: bodyJSONEnabled,
FullTextColumn: b.fullTextColumn,
JsonKeyToKey: b.jsonKeyToKey,
SkipFullTextFilter: true,

View File

@@ -4,6 +4,7 @@ import (
"context"
"testing"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
@@ -361,6 +362,7 @@ func TestResourceFilterStatementBuilder_Traces(t *testing.T) {
mockMetadataStore,
nil,
nil,
flaggertest.New(t),
)
for _, c := range cases {
@@ -554,6 +556,7 @@ func TestResourceFilterStatementBuilder_Logs(t *testing.T) {
mockMetadataStore,
nil,
nil,
flaggertest.New(t),
)
for _, c := range cases {
@@ -621,6 +624,7 @@ func TestResourceFilterStatementBuilder_Variables(t *testing.T) {
mockMetadataStore,
nil,
nil,
flaggertest.New(t),
)
for _, c := range cases {

View File

@@ -9,6 +9,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
"github.com/SigNoz/signoz/pkg/telemetrystore"
@@ -41,6 +42,7 @@ func NewTraceQueryStatementBuilder(
conditionBuilder qbtypes.ConditionBuilder,
aggExprRewriter qbtypes.AggExprRewriter,
telemetryStore telemetrystore.TelemetryStore,
flagger flagger.Flagger,
) *traceQueryStatementBuilder {
tracesSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrytraces")
@@ -53,6 +55,7 @@ func NewTraceQueryStatementBuilder(
metadataStore,
nil,
nil,
flagger,
)
return &traceQueryStatementBuilder{
@@ -510,7 +513,7 @@ func (b *traceQueryStatementBuilder) buildTimeSeriesQuery(
// Keep original column expressions so we can build the tuple
fieldNames := make([]string, 0, len(query.GroupBy))
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, nil)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, nil, false)
if err != nil {
return nil, err
}
@@ -658,7 +661,7 @@ func (b *traceQueryStatementBuilder) buildScalarQuery(
var allGroupByArgs []any
for _, gb := range query.GroupBy {
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, nil)
expr, args, err := querybuilder.CollisionHandledFinalExpr(ctx, start, end, &gb.TelemetryFieldKey, b.fm, b.cb, keys, telemetrytypes.FieldDataTypeString, nil, false)
if err != nil {
return nil, err
}

View File

@@ -6,6 +6,7 @@ import (
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -355,7 +356,8 @@ func TestStatementBuilder(t *testing.T) {
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -364,6 +366,7 @@ func TestStatementBuilder(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
vars := map[string]qbtypes.VariableItem{
@@ -648,7 +651,8 @@ func TestStatementBuilderListQuery(t *testing.T) {
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -657,6 +661,7 @@ func TestStatementBuilderListQuery(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
for _, c := range cases {
@@ -755,7 +760,8 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
if mockMetadataStore.KeysMap == nil {
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap()
}
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -764,6 +770,7 @@ func TestStatementBuilderListQueryWithCorruptData(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
q, err := statementBuilder.Build(context.Background(), 1747947419000, 1747983448000, c.requestType, c.query, nil)
@@ -905,7 +912,8 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -914,6 +922,7 @@ func TestStatementBuilderTraceQuery(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
for _, c := range cases {
@@ -1119,7 +1128,8 @@ func TestAdjustKey(t *testing.T) {
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
@@ -1127,6 +1137,7 @@ func TestAdjustKey(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
for _, c := range cases {
@@ -1391,7 +1402,8 @@ func TestAdjustKeys(t *testing.T) {
fm := NewFieldMapper()
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
mockMetadataStore,
@@ -1399,6 +1411,7 @@ func TestAdjustKeys(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
for _, c := range cases {

View File

@@ -560,6 +560,7 @@ func (b *traceOperatorCTEBuilder) buildTimeSeriesQuery(ctx context.Context, sele
keys,
telemetrytypes.FieldDataTypeString,
nil,
false,
)
if err != nil {
return nil, errors.NewInvalidInputf(
@@ -676,6 +677,7 @@ func (b *traceOperatorCTEBuilder) buildTraceQuery(ctx context.Context, selectFro
keys,
telemetrytypes.FieldDataTypeString,
nil,
false,
)
if err != nil {
return nil, errors.NewInvalidInputf(
@@ -822,6 +824,7 @@ func (b *traceOperatorCTEBuilder) buildScalarQuery(ctx context.Context, selectFr
keys,
telemetrytypes.FieldDataTypeString,
nil,
false,
)
if err != nil {
return nil, errors.NewInvalidInputf(

View File

@@ -5,6 +5,7 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -390,7 +391,8 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
traceStmtBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -399,6 +401,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
statementBuilder := NewTraceOperatorStatementBuilder(
@@ -408,6 +411,7 @@ func TestTraceOperatorStatementBuilder(t *testing.T) {
cb,
traceStmtBuilder,
aggExprRewriter,
fl,
)
for _, c := range cases {
@@ -503,7 +507,8 @@ func TestTraceOperatorStatementBuilderErrors(t *testing.T) {
cb := NewConditionBuilder(fm)
mockMetadataStore := telemetrytypestest.NewMockMetadataStore()
mockMetadataStore.KeysMap = buildCompleteFieldKeyMap()
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
traceStmtBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -512,6 +517,7 @@ func TestTraceOperatorStatementBuilderErrors(t *testing.T) {
cb,
aggExprRewriter,
nil,
fl,
)
statementBuilder := NewTraceOperatorStatementBuilder(
@@ -521,6 +527,7 @@ func TestTraceOperatorStatementBuilderErrors(t *testing.T) {
cb,
traceStmtBuilder,
aggExprRewriter,
fl,
)
for _, c := range cases {

View File

@@ -6,6 +6,7 @@ import (
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/flagger"
"github.com/SigNoz/signoz/pkg/querybuilder"
"github.com/SigNoz/signoz/pkg/telemetryresourcefilter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
@@ -31,6 +32,7 @@ func NewTraceOperatorStatementBuilder(
conditionBuilder qbtypes.ConditionBuilder,
traceStmtBuilder qbtypes.StatementBuilder[qbtypes.TraceAggregation],
aggExprRewriter qbtypes.AggExprRewriter,
flagger flagger.Flagger,
) *traceOperatorStatementBuilder {
tracesSettings := factory.NewScopedProviderSettings(settings, "github.com/SigNoz/signoz/pkg/telemetrytraces")
@@ -43,6 +45,7 @@ func NewTraceOperatorStatementBuilder(
metadataStore,
nil,
nil,
flagger,
)
return &traceOperatorStatementBuilder{

View File

@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/stretchr/testify/assert"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/stretchr/testify/require"
)
@@ -34,7 +35,8 @@ func TestTraceTimeRangeOptimization(t *testing.T) {
Signal: telemetrytypes.SignalTraces,
}}
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil)
fl := flaggertest.New(t)
aggExprRewriter := querybuilder.NewAggExprRewriter(instrumentationtest.New().ToProviderSettings(), nil, fm, cb, nil, fl)
statementBuilder := NewTraceQueryStatementBuilder(
instrumentationtest.New().ToProviderSettings(),
@@ -43,6 +45,7 @@ func TestTraceTimeRangeOptimization(t *testing.T) {
cb,
aggExprRewriter,
nil, // telemetryStore is nil - optimization won't happen but code path is tested
fl,
)
tests := []struct {

View File

@@ -49,7 +49,7 @@ type HostFilter struct {
FilterByStatus HostStatus `json:"filterByStatus"`
}
// Validate ensures HostsListRequest contains acceptable values.
// Validate ensures PostableHosts contains acceptable values.
func (req *PostableHosts) Validate() error {
if req == nil {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "request is nil")

View File

@@ -0,0 +1,109 @@
package inframonitoringtypes
import (
"encoding/json"
"slices"
"github.com/SigNoz/signoz/pkg/errors"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
)
type Pods struct {
Type ResponseType `json:"type" required:"true"`
Records []PodRecord `json:"records" required:"true"`
Total int `json:"total" required:"true"`
RequiredMetricsCheck RequiredMetricsCheck `json:"requiredMetricsCheck" required:"true"`
EndTimeBeforeRetention bool `json:"endTimeBeforeRetention" required:"true"`
Warning *qbtypes.QueryWarnData `json:"warning,omitempty"`
}
type PodRecord struct {
PodUID string `json:"podUID" required:"true"`
PodCPU float64 `json:"podCPU" required:"true"`
PodCPURequest float64 `json:"podCPURequest" required:"true"`
PodCPULimit float64 `json:"podCPULimit" required:"true"`
PodMemory float64 `json:"podMemory" required:"true"`
PodMemoryRequest float64 `json:"podMemoryRequest" required:"true"`
PodMemoryLimit float64 `json:"podMemoryLimit" required:"true"`
PodPhase PodPhase `json:"podPhase" required:"true"`
PendingPodCount int `json:"pendingPodCount" required:"true"`
RunningPodCount int `json:"runningPodCount" required:"true"`
SucceededPodCount int `json:"succeededPodCount" required:"true"`
FailedPodCount int `json:"failedPodCount" required:"true"`
UnknownPodCount int `json:"unknownPodCount" required:"true"`
PodAge int64 `json:"podAge" required:"true"`
Meta map[string]interface{} `json:"meta" required:"true"`
}
// PostablePods is the request body for the v2 pods list API.
type PostablePods struct {
Start int64 `json:"start" required:"true"`
End int64 `json:"end" required:"true"`
Filter *qbtypes.Filter `json:"filter"`
GroupBy []qbtypes.GroupByKey `json:"groupBy"`
OrderBy *qbtypes.OrderBy `json:"orderBy"`
Offset int `json:"offset"`
Limit int `json:"limit" required:"true"`
}
// Validate ensures PostablePods contains acceptable values.
func (req *PostablePods) Validate() error {
if req == nil {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "request is nil")
}
if req.Start <= 0 {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid start time %d: start must be greater than 0",
req.Start,
)
}
if req.End <= 0 {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid end time %d: end must be greater than 0",
req.End,
)
}
if req.Start >= req.End {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid time range: start (%d) must be less than end (%d)",
req.Start,
req.End,
)
}
if req.Limit < 1 || req.Limit > 5000 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "limit must be between 1 and 5000")
}
if req.Offset < 0 {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "offset cannot be negative")
}
if req.OrderBy != nil {
if !slices.Contains(PodsValidOrderByKeys, req.OrderBy.Key.Name) {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid order by key: %s", req.OrderBy.Key.Name)
}
if req.OrderBy.Direction != qbtypes.OrderDirectionAsc && req.OrderBy.Direction != qbtypes.OrderDirectionDesc {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid order by direction: %s", req.OrderBy.Direction)
}
}
return nil
}
// UnmarshalJSON validates input immediately after decoding.
func (req *PostablePods) UnmarshalJSON(data []byte) error {
type raw PostablePods
var decoded raw
if err := json.Unmarshal(data, &decoded); err != nil {
return err
}
*req = PostablePods(decoded)
return req.Validate()
}

View File

@@ -0,0 +1,55 @@
package inframonitoringtypes
import "github.com/SigNoz/signoz/pkg/valuer"
type PodPhase struct {
valuer.String
}
var (
PodPhasePending = PodPhase{valuer.NewString("pending")}
PodPhaseRunning = PodPhase{valuer.NewString("running")}
PodPhaseSucceeded = PodPhase{valuer.NewString("succeeded")}
PodPhaseFailed = PodPhase{valuer.NewString("failed")}
PodPhaseUnknown = PodPhase{valuer.NewString("unknown")}
PodPhaseNone = PodPhase{valuer.NewString("")}
)
func (PodPhase) Enum() []any {
return []any{
PodPhasePending,
PodPhaseRunning,
PodPhaseSucceeded,
PodPhaseFailed,
PodPhaseUnknown,
PodPhaseNone,
}
}
// Numeric pod phase values emitted by the k8s.pod.phase metric
// (source: OTel kubeletstats receiver).
const (
PodPhaseNumPending = 1
PodPhaseNumRunning = 2
PodPhaseNumSucceeded = 3
PodPhaseNumFailed = 4
PodPhaseNumUnknown = 5
)
const (
PodsOrderByCPU = "cpu"
PodsOrderByCPURequest = "cpu_request"
PodsOrderByCPULimit = "cpu_limit"
PodsOrderByMemory = "memory"
PodsOrderByMemoryRequest = "memory_request"
PodsOrderByMemoryLimit = "memory_limit"
)
var PodsValidOrderByKeys = []string{
PodsOrderByCPU,
PodsOrderByCPURequest,
PodsOrderByCPULimit,
PodsOrderByMemory,
PodsOrderByMemoryRequest,
PodsOrderByMemoryLimit,
}

View File

@@ -0,0 +1,219 @@
package inframonitoringtypes
import (
"testing"
"github.com/SigNoz/signoz/pkg/errors"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/stretchr/testify/require"
)
func TestPostablePods_Validate(t *testing.T) {
tests := []struct {
name string
req *PostablePods
wantErr bool
}{
{
name: "valid request",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 100,
Offset: 0,
},
wantErr: false,
},
{
name: "nil request",
req: nil,
wantErr: true,
},
{
name: "start time zero",
req: &PostablePods{
Start: 0,
End: 2000,
Limit: 100,
Offset: 0,
},
wantErr: true,
},
{
name: "start time negative",
req: &PostablePods{
Start: -1000,
End: 2000,
Limit: 100,
Offset: 0,
},
wantErr: true,
},
{
name: "end time zero",
req: &PostablePods{
Start: 1000,
End: 0,
Limit: 100,
Offset: 0,
},
wantErr: true,
},
{
name: "start time greater than end time",
req: &PostablePods{
Start: 2000,
End: 1000,
Limit: 100,
Offset: 0,
},
wantErr: true,
},
{
name: "start time equal to end time",
req: &PostablePods{
Start: 1000,
End: 1000,
Limit: 100,
Offset: 0,
},
wantErr: true,
},
{
name: "limit zero",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 0,
Offset: 0,
},
wantErr: true,
},
{
name: "limit negative",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: -10,
Offset: 0,
},
wantErr: true,
},
{
name: "limit exceeds max",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 5001,
Offset: 0,
},
wantErr: true,
},
{
name: "offset negative",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 100,
Offset: -5,
},
wantErr: true,
},
{
name: "orderBy nil is valid",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 100,
Offset: 0,
},
wantErr: false,
},
{
name: "orderBy with valid key cpu and direction asc",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 100,
Offset: 0,
OrderBy: &qbtypes.OrderBy{
Key: qbtypes.OrderByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: PodsOrderByCPU,
},
},
Direction: qbtypes.OrderDirectionAsc,
},
},
wantErr: false,
},
{
name: "orderBy with phase key is rejected",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 100,
Offset: 0,
OrderBy: &qbtypes.OrderBy{
Key: qbtypes.OrderByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "phase",
},
},
Direction: qbtypes.OrderDirectionDesc,
},
},
wantErr: true,
},
{
name: "orderBy with invalid key",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 100,
Offset: 0,
OrderBy: &qbtypes.OrderBy{
Key: qbtypes.OrderByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: "unknown",
},
},
Direction: qbtypes.OrderDirectionDesc,
},
},
wantErr: true,
},
{
name: "orderBy with valid key but invalid direction",
req: &PostablePods{
Start: 1000,
End: 2000,
Limit: 100,
Offset: 0,
OrderBy: &qbtypes.OrderBy{
Key: qbtypes.OrderByKey{
TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{
Name: PodsOrderByMemory,
},
},
Direction: qbtypes.OrderDirection{String: valuer.NewString("invalid")},
},
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.req.Validate()
if tt.wantErr {
require.Error(t, err)
require.True(t, errors.Ast(err, errors.TypeInvalidInput), "expected error to be of type InvalidInput")
} else {
require.NoError(t, err)
}
})
}
}

View File

@@ -62,6 +62,6 @@ def signoz_json_body(
pytestconfig=pytestconfig,
cache_key="signoz-json-body",
env_overrides={
"BODY_JSON_QUERY_ENABLED": "true",
"SIGNOZ_FLAGGER_CONFIG_BOOLEAN_BODY__JSON__ENABLED": True,
},
)