Compare commits

...

2 Commits

Author SHA1 Message Date
Tushar Vats
629ea3b8be feat: extend error responses with new error struct (#11542)
Some checks are pending
build-staging / js-build (push) Blocked by required conditions
build-staging / prepare (push) Waiting to run
build-staging / go-build (push) Blocked by required conditions
build-staging / staging (push) Blocked by required conditions
Release Drafter / update_release_draft (push) Waiting to run
* feat: extend error responses with new error struct

* fix: enriched error for dashboard api

* fix: merge issues

* fix: reverted dashboards changes and add for cloud integrations

* fix: delete file

* fix: add back file

* fix: added a helper

* fix: removed invlaid referencess

* fix: generate openapi

* fix: keeping additional along with suggestion

* Revert "fix: keeping additional along with suggestion"

This reverts commit be30e2ffd2.

* fix: added suggestions per additonal error

* fix: generate openapi

* fix: remove valid references

* fix: removeg valid references for select and group by and only did you mean is kept

* fix: unit test

* fix: use binding for deconding for both ee and community

* fix: trim down suggestions methods

* fix: added renamed methods and moved stuff around

* fix: typo

* fix: removed json decoder

* fix: added empty check

* fix: retain addtional

* fix: reverted re-structing of file
2026-06-15 11:58:12 +00:00
Pandey
287b60cbe6 feat(statsreporter): expose collected stats via GET /api/v1/stats (#11698)
* feat(statsreporter): expose collected stats via GET /api/v1/stats

Extract per-org stats collection out of the analytics reporter into an
always-on Aggregator (collector fan-out + telemetry-store counts) shared
by the reporter and a new HTTP handler. The GET /api/v1/stats endpoint
returns the caller's org stats regardless of whether scheduled reporting
is enabled.

* refactor(statsreporter): collect telemetry stats via the querier

Move the trace/log/metric row-count and last-observed queries out of the
stats aggregator and into the querier, which now implements
statsreporter.StatsCollector. The aggregator becomes a pure collector
fan-out and no longer depends on telemetrystore; the querier is wired in
as one of the stats collectors.

* chore: regenerate openapi spec and frontend client

Backend docs/api/openapi.yml gains the GET /api/v1/stats (GetStats)
operation; the Orval client gains a useGetStats hook and GetStats200
type.

* chore: remove comment from querier Collect

* fix(statsreporter): use MustNewUUID for org from claims

Claims come from validated auth context, so the org UUID is guaranteed
valid; drop the dead NewUUID error branch.

* fix(flagger): use MustNewUUID for org from claims

Claims come from validated auth context, so the org UUID is guaranteed
valid; drop the dead NewUUID error branch.

* docs(contributing): note MustNewUUID for IDs from claims

* perf(querier): combine count and last-observed into one query per signal

Each signal's COUNT(*) and max(timestamp) scan the same table, so fetch
both in a single query — 3 queries instead of 6. Same emitted keys and
empty-table guard.
2026-06-15 11:27:17 +00:00
49 changed files with 1294 additions and 864 deletions

View File

@@ -19,5 +19,8 @@
"editor.defaultFormatter": "vscode.html-language-features"
},
"python-envs.defaultEnvManager": "ms-python.python:system",
"python-envs.pythonProjects": []
"python-envs.pythonProjects": [],
"[json]": {
"editor.defaultFormatter": "vscode.json-language-features"
}
}

View File

@@ -3566,10 +3566,6 @@ components:
items:
$ref: '#/components/schemas/ErrorsResponseerroradditional'
type: array
invalidReferences:
items:
type: string
type: array
message:
type: string
retry:
@@ -3590,6 +3586,10 @@ components:
properties:
message:
type: string
suggestions:
items:
type: string
type: array
type: object
ErrorsResponseretryjson:
properties:
@@ -12783,6 +12783,53 @@ paths:
summary: Update a span mapper
tags:
- spanmapper
/api/v1/stats:
get:
deprecated: false
description: This endpoint returns the collected stats for the organization
operationId: GetStats
responses:
"200":
content:
application/json:
schema:
properties:
data:
additionalProperties: {}
type: object
status:
type: string
required:
- status
- data
type: object
description: OK
"401":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Unauthorized
"403":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"500":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Internal Server Error
security:
- api_key:
- VIEWER
- tokenizer:
- VIEWER
summary: Get stats
tags:
- stats
/api/v1/testChannel:
post:
deprecated: true

View File

@@ -109,6 +109,20 @@ func (h *handler) CreateThing(rw http.ResponseWriter, req *http.Request) {
}
```
When you need an ID from `claims` as a `valuer.UUID` (for example to pass it to a module), derive it with the `Must*` constructor instead of `NewUUID` plus an error check. Claims are validated by the auth middleware, so the conversion cannot fail and the error branch would be dead code:
```go
// Good — claims are pre-validated, the conversion cannot fail.
orgID := valuer.MustNewUUID(claims.OrgID)
// Avoid — the error path is unreachable.
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(rw, err)
return
}
```
### 3. Register the handler in `signozapiserver`
In `pkg/apiserver/signozapiserver`, add a route in the appropriate `add*Routes` function (`addUserRoutes`, `addSessionRoutes`, `addOrgRoutes`, etc.). The pattern is:
@@ -387,3 +401,4 @@ Note the discriminator property lives in the variants, not on the parent — the
- **Add `nullable:"true"`** on fields that can be `null`. Pay special attention to slices and maps -- in Go these default to `nil` which serializes to `null`. If the field should always be an array, initialize it and do not mark it nullable.
- **Implement `Enum()`** on every type that has a fixed set of acceptable values so the JSON schema generates proper `enum` constraints.
- **Add request examples** via `RequestExamples` in `OpenAPIDef` for any non-trivial endpoint. See `pkg/apiserver/signozapiserver/querier.go` for reference.
- **Derive IDs from `claims` with `valuer.MustNewUUID`** (e.g. `claims.OrgID`, `claims.UserID`). Claims are pre-validated by the auth middleware, so use the `Must*` constructor — don't write `NewUUID` followed by an `if err != nil { render.Error(...); return }` block.

View File

@@ -3,13 +3,13 @@ package querier
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http"
anomalyV2 "github.com/SigNoz/signoz/ee/anomaly"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/types/authtypes"
@@ -48,8 +48,8 @@ func (h *handler) QueryRange(rw http.ResponseWriter, req *http.Request) {
}
var queryRangeRequest qbtypes.QueryRangeRequest
if err := json.NewDecoder(req.Body).Decode(&queryRangeRequest); err != nil {
render.Error(rw, errors.NewInvalidInputf(errors.CodeInvalidInput, "failed to decode request body: %v", err))
if err := binding.JSON.BindBody(req.Body, &queryRangeRequest); err != nil {
render.Error(rw, err)
return
}

View File

@@ -2143,6 +2143,10 @@ export interface ErrorsResponseerroradditionalDTO {
* @type string
*/
message?: string;
/**
* @type array
*/
suggestions?: string[];
}
export interface ErrorsResponseretryjsonDTO {
@@ -2158,10 +2162,6 @@ export interface ErrorsJSONDTO {
* @type array
*/
errors?: ErrorsResponseerroradditionalDTO[];
/**
* @type array
*/
invalidReferences?: string[];
/**
* @type string
*/
@@ -9736,6 +9736,19 @@ export type UpdateSpanMapperPathParameters = {
groupId: string;
mapperId: string;
};
export type GetStats200Data = { [key: string]: unknown };
export type GetStats200 = {
/**
* @type object
*/
data: GetStats200Data;
/**
* @type string
*/
status: string;
};
export type GetTraceAggregationsPathParameters = {
traceID: string;
};

View File

@@ -0,0 +1,96 @@
/**
* ! Do not edit manually
* * The file has been auto-generated using Orval for SigNoz
* * regenerate with 'pnpm generate:api'
* SigNoz
*/
import { useQuery } from 'react-query';
import type {
InvalidateOptions,
QueryClient,
QueryFunction,
QueryKey,
UseQueryOptions,
UseQueryResult,
} from 'react-query';
import type { GetStats200, RenderErrorResponseDTO } from '../sigNoz.schemas';
import { GeneratedAPIInstance } from '../../../generatedAPIInstance';
import type { ErrorType } from '../../../generatedAPIInstance';
/**
* This endpoint returns the collected stats for the organization
* @summary Get stats
*/
export const getStats = (signal?: AbortSignal) => {
return GeneratedAPIInstance<GetStats200>({
url: `/api/v1/stats`,
method: 'GET',
signal,
});
};
export const getGetStatsQueryKey = () => {
return [`/api/v1/stats`] as const;
};
export const getGetStatsQueryOptions = <
TData = Awaited<ReturnType<typeof getStats>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<Awaited<ReturnType<typeof getStats>>, TError, TData>;
}) => {
const { query: queryOptions } = options ?? {};
const queryKey = queryOptions?.queryKey ?? getGetStatsQueryKey();
const queryFn: QueryFunction<Awaited<ReturnType<typeof getStats>>> = ({
signal,
}) => getStats(signal);
return { queryKey, queryFn, ...queryOptions } as UseQueryOptions<
Awaited<ReturnType<typeof getStats>>,
TError,
TData
> & { queryKey: QueryKey };
};
export type GetStatsQueryResult = NonNullable<
Awaited<ReturnType<typeof getStats>>
>;
export type GetStatsQueryError = ErrorType<RenderErrorResponseDTO>;
/**
* @summary Get stats
*/
export function useGetStats<
TData = Awaited<ReturnType<typeof getStats>>,
TError = ErrorType<RenderErrorResponseDTO>,
>(options?: {
query?: UseQueryOptions<Awaited<ReturnType<typeof getStats>>, TError, TData>;
}): UseQueryResult<TData, TError> & { queryKey: QueryKey } {
const queryOptions = getGetStatsQueryOptions(options);
const query = useQuery(queryOptions) as UseQueryResult<TData, TError> & {
queryKey: QueryKey;
};
return { ...query, queryKey: queryOptions.queryKey };
}
/**
* @summary Get stats
*/
export const invalidateGetStats = async (
queryClient: QueryClient,
options?: InvalidateOptions,
): Promise<QueryClient> => {
await queryClient.invalidateQueries(
{ queryKey: getGetStatsQueryKey() },
options,
);
return queryClient;
};

View File

@@ -31,6 +31,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/user"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/zeus"
@@ -70,6 +71,7 @@ type provider struct {
traceDetailHandler tracedetail.Handler
rulerHandler ruler.Handler
llmPricingRuleHandler llmpricingrule.Handler
statsHandler statsreporter.Handler
}
func NewFactory(
@@ -102,6 +104,7 @@ func NewFactory(
llmPricingRuleHandler llmpricingrule.Handler,
traceDetailHandler tracedetail.Handler,
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
) factory.ProviderFactory[apiserver.APIServer, apiserver.Config] {
return factory.NewProviderFactory(factory.MustNewName("signoz"), func(ctx context.Context, providerSettings factory.ProviderSettings, config apiserver.Config) (apiserver.APIServer, error) {
return newProvider(
@@ -137,6 +140,7 @@ func NewFactory(
llmPricingRuleHandler,
traceDetailHandler,
rulerHandler,
statsHandler,
)
})
}
@@ -174,6 +178,7 @@ func newProvider(
llmPricingRuleHandler llmpricingrule.Handler,
traceDetailHandler tracedetail.Handler,
rulerHandler ruler.Handler,
statsHandler statsreporter.Handler,
) (apiserver.APIServer, error) {
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/apiserver/signozapiserver")
router := mux.NewRouter().UseEncodedPath()
@@ -210,6 +215,7 @@ func newProvider(
traceDetailHandler: traceDetailHandler,
rulerHandler: rulerHandler,
llmPricingRuleHandler: llmPricingRuleHandler,
statsHandler: statsHandler,
}
provider.authzMiddleware = middleware.NewAuthZ(settings.Logger(), orgGetter, authzService)
@@ -334,6 +340,10 @@ func (provider *provider) AddToRouter(router *mux.Router) error {
return err
}
if err := provider.addStatsReporterRoutes(router); err != nil {
return err
}
return nil
}

View File

@@ -0,0 +1,33 @@
package signozapiserver
import (
"net/http"
"github.com/SigNoz/signoz/pkg/http/handler"
"github.com/SigNoz/signoz/pkg/types"
"github.com/gorilla/mux"
)
func (provider *provider) addStatsReporterRoutes(router *mux.Router) error {
if err := router.Handle("/api/v1/stats", handler.New(
provider.authzMiddleware.ViewAccess(provider.statsHandler.Get),
handler.OpenAPIDef{
ID: "GetStats",
Tags: []string{"stats"},
Summary: "Get stats",
Description: "This endpoint returns the collected stats for the organization",
Request: nil,
RequestContentType: "",
Response: map[string]any{},
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusOK,
ErrorStatusCodes: []int{},
Deprecated: false,
SecuritySchemes: newSecuritySchemes(types.RoleViewer),
},
)).Methods(http.MethodGet).GetError(); err != nil {
return err
}
return nil
}

View File

@@ -21,18 +21,25 @@ type base struct {
e error
// u denotes the url for the documentation (if present) for the error.
u string
// a denotes any additional error messages (if present).
a []string
// a denotes any additional error details (if present). Each detail carries an
// optional message and any user-facing suggestions closely related to it.
a []additional
// s contains the stacktrace captured at error creation time.
s fmt.Stringer
// r is the retry strategy for the error, if applicable.
r *retry
// suggestions is a list of user-facing suggestions related to the error, if present.
// For example, narrow the time range window or typo suggestion
// suggestions is a list of user-facing suggestions related to the error as a
// whole (not tied to a specific detail in a), if present. For example,
// "narrow the time range window". For a suggestion tied to a specific detail,
// use the suggestions field on additional instead.
suggestions []string
}
// additional is a single supplementary error detail: a message plus any
// user-facing suggestions (e.g. "did you mean: `x`") closely related to it.
type additional struct {
message string
suggestions []string
// invalidReferences is a list of references that were invalid and contributed to the error, if present.
// For example, a typo from user avg(sum), we return invalidRefences: ['sum']
invalidReferences []string
}
// Stacktrace returns the stacktrace captured at error creation time, formatted as a string.
@@ -47,16 +54,15 @@ func (b *base) Stacktrace() string {
// and returns a new base error.
func (b *base) WithStacktrace(s string) *base {
return &base{
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: b.a,
s: rawStacktrace(s),
r: b.r,
suggestions: b.suggestions,
invalidReferences: b.invalidReferences,
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: b.a,
s: rawStacktrace(s),
r: b.r,
suggestions: b.suggestions,
}
}
@@ -77,7 +83,7 @@ func New(t typ, code Code, message string) *base {
m: message,
e: nil,
u: "",
a: []string{},
a: []additional{},
s: newStackTrace(),
}
}
@@ -96,127 +102,144 @@ func Newf(t typ, code Code, format string, args ...any) *base {
// Wrapf returns a new error by formatting the error message with the supplied format specifier
// and wrapping another error with base.
func Wrapf(cause error, t typ, code Code, format string, args ...any) *base {
return &base{
b := &base{
t: t,
c: code,
m: fmt.Sprintf(format, args...),
e: cause,
s: newStackTrace(),
}
// Carry the user-facing hints forward from the wrapped cause. Otherwise
// wrapping a structured error (e.g. one returned from an UnmarshalJSON) would
// silently drop its suggestions / invalid references from the response.
// Propagation is transitive: each Wrapf copies from its immediate cause, so
// the hints survive arbitrarily deep wrapping as long as it goes through Wrapf.
if inner, ok := cause.(*base); ok {
b.r = inner.r
b.a = inner.a
b.suggestions = inner.suggestions
}
return b
}
// Wrap returns a new error by wrapping another error with base.
func Wrap(cause error, t typ, code Code, message string) *base {
return &base{
b := &base{
t: t,
c: code,
m: message,
e: cause,
s: newStackTrace(),
}
// Carry the user-facing hints forward from the wrapped cause. Otherwise
// wrapping a structured error (e.g. one returned from an UnmarshalJSON) would
// silently drop its suggestions / invalid references from the response.
// Propagation is transitive: each Wrapf copies from its immediate cause, so
// the hints survive arbitrarily deep wrapping as long as it goes through Wrapf.
if inner, ok := cause.(*base); ok {
b.r = inner.r
b.a = inner.a
b.suggestions = inner.suggestions
}
return b
}
// WithAdditionalf adds an additional error message to the existing error.
func WithAdditionalf(cause error, format string, args ...any) *base {
t, c, m, e, u, a := Unwrapb(cause)
var s fmt.Stringer
if original, ok := cause.(*base); ok {
s = original.s
}
b := &base{
t: t,
c: c,
m: m,
e: e,
u: u,
a: a,
s: s,
r: retryOf(cause),
suggestions: suggestionsOf(cause),
invalidReferences: invalidReferencesOf(cause),
if b, ok := cause.(*base); ok {
return b.WithAdditional(fmt.Sprintf(format, args...))
}
return b.WithAdditional(append(a, fmt.Sprintf(format, args...))...)
t, c, m, e, u, a := Unwrapb(cause)
b := &base{t: t, c: c, m: m, e: e, u: u, a: a, s: newStackTrace(), r: retryOf(cause)}
return b.WithAdditional(fmt.Sprintf(format, args...))
}
// WithSuggestiveAdditionalf appends a detail whose message is built from the format
// specifier and which carries the given user-facing suggestions closely related to
// it, returning a new base error.
func WithSuggestiveAdditionalf(cause error, suggestions []string, format string, args ...any) *base {
if b, ok := cause.(*base); ok {
return b.WithSuggestiveAdditional(fmt.Sprintf(format, args...), suggestions...)
}
t, c, m, e, u, a := Unwrapb(cause)
b := &base{t: t, c: c, m: m, e: e, u: u, a: a, s: newStackTrace(), r: retryOf(cause)}
return b.WithSuggestiveAdditional(fmt.Sprintf(format, args...), suggestions...)
}
// WithUrl adds a url to the base error and returns a new base error.
func (b *base) WithUrl(u string) *base {
return &base{
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: u,
a: b.a,
s: b.s,
r: b.r,
suggestions: b.suggestions,
invalidReferences: b.invalidReferences,
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: u,
a: b.a,
s: b.s,
r: b.r,
suggestions: b.suggestions,
}
}
// WithAdditional adds additional messages to the base error and returns a new base error.
func (b *base) WithAdditional(a ...string) *base {
return &base{
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: a,
s: b.s,
r: b.r,
suggestions: b.suggestions,
invalidReferences: b.invalidReferences,
// WithAdditional appends one or more message-only details and returns a new base error.
func (b *base) WithAdditional(messages ...string) *base {
extra := make([]additional, len(messages))
for i, m := range messages {
extra[i] = additional{message: m}
}
return b.WithAdditionals(extra...)
}
// WithAdditionals appends the given details and returns a new base error. It is also
// the way to re-attach details previously pulled out via Unwrapb.
func (b *base) WithAdditionals(additionals ...additional) *base {
nb := *b
nb.a = append(append([]additional{}, b.a...), additionals...)
return &nb
}
// withRetry adds retry metadata to the base error and returns a new base error.
func (b *base) withRetry(r retry) *base {
return &base{
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: b.a,
s: b.s,
r: &r,
suggestions: b.suggestions,
invalidReferences: b.invalidReferences,
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: b.a,
s: b.s,
r: &r,
suggestions: b.suggestions,
}
}
// WithSuggestions replaces the list of suggestions on the base error.
// WithSuggestions replaces the error-wide suggestions and returns a new base error.
// These relate to the error as a whole; for a suggestion tied to a specific detail,
// use WithSuggestiveAdditional.
func (b *base) WithSuggestions(suggestions ...string) *base {
return &base{
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: b.a,
s: b.s,
r: b.r,
suggestions: suggestions,
invalidReferences: b.invalidReferences,
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: b.a,
s: b.s,
r: b.r,
suggestions: suggestions,
}
}
// WithInvalidReferences replaces the list of invalid references on the base error.
func (b *base) WithInvalidReferences(invalidReferences ...string) *base {
return &base{
t: b.t,
c: b.c,
m: b.m,
e: b.e,
u: b.u,
a: b.a,
s: b.s,
r: b.r,
suggestions: b.suggestions,
invalidReferences: invalidReferences,
}
// WithSuggestiveAdditional appends a detail carrying a message together with the
// user-facing suggestions closely related to it, and returns a new base error.
func (b *base) WithSuggestiveAdditional(message string, suggestions ...string) *base {
return b.WithAdditionals(additional{message: message, suggestions: suggestions})
}
// WithRetryAfter sets the retry delay on the base error and returns a new base error.
@@ -231,13 +254,13 @@ func (b *base) WithRetryAfter(delay time.Duration) *base {
// and the error itself.
//
//nolint:staticcheck // ST1008: intentional return order matching struct field order (TCMEUA)
func Unwrapb(cause error) (typ, Code, string, error, string, []string) {
func Unwrapb(cause error) (typ, Code, string, error, string, []additional) {
base, ok := cause.(*base)
if ok {
return base.t, base.c, base.m, base.e, base.u, base.a
}
return TypeInternal, CodeUnknown, cause.Error(), cause, "", []string{}
return TypeInternal, CodeUnknown, cause.Error(), cause, "", []additional{}
}
// Ast checks if the provided error matches the specified custom error type.
@@ -371,11 +394,3 @@ func suggestionsOf(err error) []string {
}
return nil
}
func invalidReferencesOf(err error) []string {
base, ok := err.(*base)
if ok {
return base.invalidReferences
}
return nil
}

View File

@@ -48,7 +48,7 @@ func TestUnwrapb(t *testing.T) {
assert.Equal(t, "this is a base err", amessage)
assert.Equal(t, oerr, aerr)
assert.Equal(t, "https://docs", au)
assert.Equal(t, []string{"additional err"}, aa)
assert.Equal(t, []additional{{message: "additional err"}}, aa)
atyp, _, _, _, _, _ = Unwrapb(oerr)
assert.Equal(t, TypeInternal, atyp)
@@ -74,6 +74,19 @@ func TestWithSuggestions(t *testing.T) {
assert.Equal(t, []string{"first", "second"}, suggestionsOf(err))
}
func TestWithSuggestiveAdditional(t *testing.T) {
// WithSuggestiveAdditional attaches suggestions to a specific detail (in the
// errors array), distinct from the error-wide WithSuggestions.
err := NewInvalidInputf(MustNewCode("bad_field"), "unknown field %q", "filed").
WithSuggestiveAdditional("field `filed` not found", "did you mean: `field`")
j := AsJSON(err)
assert.Equal(t, []responseerroradditional{
{Message: "field `filed` not found", Suggestions: []string{"did you mean: `field`"}},
}, j.Errors)
assert.Nil(t, j.Suggestions, "detail-scoped suggestions must not leak into the error-wide list")
}
func TestWithRetryAfter(t *testing.T) {
err := New(TypeInternal, MustNewCode("test_code"), "test error").WithRetryAfter(5 * time.Microsecond)
r := retryOf(err)
@@ -81,24 +94,11 @@ func TestWithRetryAfter(t *testing.T) {
assert.Equal(t, 5, int(r.delay.Microseconds()))
}
func TestWithInvalidReferences(t *testing.T) {
// WithInvalidReferences populates the list.
err := New(TypeInvalidInput, MustNewCode("bad_ref"), "bad ref").
WithInvalidReferences("queries[0]", "queries[1]")
assert.Equal(t, []string{"queries[0]", "queries[1]"}, invalidReferencesOf(err))
// WithInvalidReferences replaces the entire list on each call.
err = err.WithInvalidReferences("queries[2]")
assert.Equal(t, []string{"queries[2]"}, invalidReferencesOf(err),
"WithInvalidReferences must replace the entire list")
}
func TestAsJSONBaseError(t *testing.T) {
err := New(TypeInvalidInput, MustNewCode("bad_input"), "field foo is bad").
WithUrl("https://docs/bad_input").
WithAdditional("hint1", "hint2").
WithSuggestions("try this").
WithInvalidReferences("queries[0]")
WithSuggestions("try this")
j := AsJSON(err)
@@ -113,7 +113,20 @@ func TestAsJSONBaseError(t *testing.T) {
assert.Nil(t, j.Retry, "bare New(...) should not populate a retry block")
assert.Equal(t, []string{"try this"}, j.Suggestions)
assert.Equal(t, []string{"queries[0]"}, j.InvalidReferences)
}
func TestAsJSONWrappedErrorPreservesHints(t *testing.T) {
// An inner base carries the user-facing hints (e.g. produced inside an
// UnmarshalJSON), then gets re-wrapped (e.g. WrapInvalidInputf). suggestionsOf
// must walk the cause chain so the hints still surface.
inner := NewInvalidInputf(MustNewCode("bad_kind"), "unknown panel kind %q", "boom").
WithSuggestions("valid references: a, b, c")
wrapped := WrapInvalidInputf(inner, MustNewCode("outer"), "%s", inner.Error())
j := AsJSON(wrapped)
assert.Equal(t, []string{"valid references: a, b, c"}, j.Suggestions,
"suggestions on an inner base must survive wrapping")
}
func TestAsJSONRetryBlock(t *testing.T) {
@@ -147,7 +160,6 @@ func TestAsJSONRetryBlock(t *testing.T) {
func TestAsJSONOptionalFieldsOmittedWhenEmpty(t *testing.T) {
j := AsJSON(New(TypeInternal, MustNewCode("boom"), "boom"))
assert.Nil(t, j.Suggestions, "no suggestions set => Suggestions must be nil so json omitempty drops it")
assert.Nil(t, j.InvalidReferences, "no invalid references set => InvalidReferences must be nil so json omitempty drops it")
}
func TestWithStacktrace(t *testing.T) {

View File

@@ -7,14 +7,13 @@ import (
)
type JSON struct {
Type string `json:"type,omitempty"`
Code string `json:"code" required:"true"`
Message string `json:"message" required:"true"`
Url string `json:"url,omitempty"`
Errors []responseerroradditional `json:"errors,omitempty"`
Retry *responseretryjson `json:"retry,omitempty"`
Suggestions []string `json:"suggestions,omitempty"`
InvalidReferences []string `json:"invalidReferences,omitempty"`
Type string `json:"type,omitempty"`
Code string `json:"code" required:"true"`
Message string `json:"message" required:"true"`
Url string `json:"url,omitempty"`
Errors []responseerroradditional `json:"errors,omitempty"`
Retry *responseretryjson `json:"retry,omitempty"`
Suggestions []string `json:"suggestions,omitempty"`
}
type responseretryjson struct {
@@ -22,7 +21,8 @@ type responseretryjson struct {
}
type responseerroradditional struct {
Message string `json:"message"`
Message string `json:"message,omitempty"`
Suggestions []string `json:"suggestions,omitempty"`
}
func AsJSON(cause error) *JSON {
@@ -31,7 +31,7 @@ func AsJSON(cause error) *JSON {
rea := make([]responseerroradditional, len(a))
for k, v := range a {
rea[k] = responseerroradditional{v}
rea[k] = responseerroradditional{Message: v.message, Suggestions: v.suggestions}
}
var retry *responseretryjson
@@ -40,14 +40,13 @@ func AsJSON(cause error) *JSON {
}
return &JSON{
Type: t.String(),
Code: c.String(),
Message: m,
Url: u,
Errors: rea,
Retry: retry,
Suggestions: suggestionsOf(cause),
InvalidReferences: invalidReferencesOf(cause),
Type: t.String(),
Code: c.String(),
Message: m,
Url: u,
Errors: rea,
Retry: retry,
Suggestions: suggestionsOf(cause),
}
}
@@ -57,7 +56,7 @@ func AsURLValues(cause error) url.Values {
rea := make([]responseerroradditional, len(a))
for k, v := range a {
rea[k] = responseerroradditional{v}
rea[k] = responseerroradditional{Message: v.message, Suggestions: v.suggestions}
}
errors, err := json.Marshal(rea)

165
pkg/errors/suggestions.go Normal file
View File

@@ -0,0 +1,165 @@
package errors
import (
"fmt"
"strings"
)
const (
typoSuggestionThreshold = 0.75
// maxValidReferences caps how many valid references are listed so
// high-cardinality sets (e.g. telemetry field keys) don't dump the entire
// set into the error.
maxValidReferences = 20
)
// SuggestionsOnLevenshteinDistance returns a "did you mean" correction (only
// when a close match at least typoSuggestionThreshold similar exists) followed
// by the valid-references list.
func SuggestionsOnLevenshteinDistance(invalidInput string, validInputs []string) []string {
suggestions := make([]string, 0, 2)
if match, ok := ClosestLevenshteinMatch(invalidInput, validInputs); ok {
suggestions = append(suggestions, didYouMean(match))
}
if refs := ValidReferences(validInputs...); refs != "" {
suggestions = append(suggestions, refs)
}
return suggestions
}
// ClosestLevenshteinMatch returns the candidate most similar to input that is at least
// typoSuggestionThreshold similar, or false when nothing is close enough.
func ClosestLevenshteinMatch(input string, candidates []string) (string, bool) {
var bestMatch string
bestSimilarity := 0.0
for _, candidate := range candidates {
sim := similarity(input, candidate)
if sim > bestSimilarity && sim >= typoSuggestionThreshold {
bestSimilarity = sim
bestMatch = candidate
}
}
if bestSimilarity >= typoSuggestionThreshold {
return bestMatch, true
}
return "", false
}
// SuggestionsFromFunc formats the string produce returns as a one-element
// "did you mean: `x`" slice, or nil when it returns the empty string (so callers
// with their own matching strategy compose into a suggestions list cleanly).
func SuggestionsFromFunc(produce func() string) []string {
s := produce()
if s == "" {
return nil
}
return []string{didYouMean(s)}
}
// ValidReferences formats values as "valid references: `a`, `b`", capped at
// maxValidReferences with a "(+N more)" suffix. Each value is rendered as its
// own string, an Enum() element's StringValue(), or fmt.Sprint as a fallback.
// It returns "" when there are no values, so callers don't surface a bare
// "valid references: " with nothing after it.
func ValidReferences[T any](values ...T) string {
if len(values) == 0 {
return ""
}
refs := make([]string, 0, len(values))
for _, v := range values {
switch t := any(v).(type) {
case string:
refs = append(refs, t)
case interface{ StringValue() string }:
refs = append(refs, t.StringValue())
default:
refs = append(refs, fmt.Sprint(t))
}
}
truncated := 0
if len(refs) > maxValidReferences {
truncated = len(refs) - maxValidReferences
refs = refs[:maxValidReferences]
}
quoted := make([]string, len(refs))
for i, r := range refs {
quoted[i] = "`" + r + "`"
}
out := "valid references: " + strings.Join(quoted, ", ")
if truncated > 0 {
out += fmt.Sprintf(" (+%d more)", truncated)
}
return out
}
func levenshteinDistance(s1, s2 string) int {
s1 = strings.ToLower(s1)
s2 = strings.ToLower(s2)
if len(s1) == 0 {
return len(s2)
}
if len(s2) == 0 {
return len(s1)
}
v0 := make([]int, len(s2)+1)
v1 := make([]int, len(s2)+1)
for i := 0; i <= len(s2); i++ {
v0[i] = i
}
for i := range len(s1) {
v1[0] = i + 1
for j := range len(s2) {
deletionCost := v0[j+1] + 1
insertionCost := v1[j] + 1
var substitutionCost int
if s1[i] == s2[j] {
substitutionCost = v0[j]
} else {
substitutionCost = v0[j] + 1
}
v1[j+1] = min(deletionCost, insertionCost, substitutionCost)
}
for j := 0; j <= len(s2); j++ {
v0[j] = v1[j]
}
}
return v1[len(s2)]
}
func similarity(s1, s2 string) float64 {
maxLen := max(len(s1), len(s2))
if maxLen == 0 {
return 1.0
}
distance := levenshteinDistance(s1, s2)
return 1.0 - float64(distance)/float64(maxLen)
}
// didYouMean formats a correction as "did you mean: `x`".
func didYouMean(match string) string {
return "did you mean: `" + match + "`"
}

View File

@@ -0,0 +1,31 @@
package errors
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestValidReferences(t *testing.T) {
// An empty set returns "" so callers don't surface a bare "valid references: ".
assert.Equal(t, "", ValidReferences[string]())
assert.Equal(t, "valid references: `a`, `b`", ValidReferences("a", "b"))
}
func TestSuggestionsOnLevenshteinDistance(t *testing.T) {
// No valid inputs => no suggestions at all (no bare "valid references: ").
assert.Empty(t, SuggestionsOnLevenshteinDistance("foo", nil))
// Close match => did-you-mean plus the valid-references list.
assert.Equal(t,
[]string{"did you mean: `name`", "valid references: `name`, `color`"},
SuggestionsOnLevenshteinDistance("nam", []string{"name", "color"}),
)
// No close match => valid-references list only.
assert.Equal(t,
[]string{"valid references: `name`, `color`"},
SuggestionsOnLevenshteinDistance("zzzzz", []string{"name", "color"}),
)
}

View File

@@ -35,11 +35,7 @@ func (handler *handler) GetFeatures(rw http.ResponseWriter, r *http.Request) {
return
}
orgID, err := valuer.NewUUID(claims.OrgID)
if err != nil {
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)

View File

@@ -20,6 +20,7 @@ var (
type bindBodyOptions struct {
DisallowUnknownFields bool
UseNumber bool
UnknownFieldContext string
}
type BindBodyOption func(*bindBodyOptions)
@@ -30,6 +31,12 @@ func WithDisallowUnknownFields(disallowUnknownFields bool) BindBodyOption {
}
}
func WithUnknownFieldContext(context string) BindBodyOption {
return func(options *bindBodyOptions) {
options.UnknownFieldContext = context
}
}
func WithUseNumber(useNumber bool) BindBodyOption {
return func(options *bindBodyOptions) {
options.UseNumber = useNumber

View File

@@ -3,6 +3,8 @@ package binding
import (
"encoding/json"
"io"
"reflect"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
)
@@ -59,8 +61,70 @@ func (b *jsonBinding) BindBody(body io.Reader, obj any, opts ...BindBodyOption)
WithAdditional("value of type '" + unmarshalTypeError.Value + "' was received, try sending '" + unmarshalTypeError.Type.String() + "' instead?")
}
// DisallowUnknownFields surfaces a bare `json: unknown field "x"`; turn it
// into a structured invalid-input error with did-you-mean/valid-reference
// suggestions drawn from obj's own JSON field names. Gated on the strict
// flag so an already-structured "unknown field" error bubbling up from a
// nested UnmarshalJSON is passed through unchanged, not re-wrapped here with
// the wrong (outer) field set.
if bindBodyOptions.DisallowUnknownFields && strings.Contains(err.Error(), "unknown field") {
if field := extractUnknownField(err.Error()); field != "" {
message := "unknown field %q"
if bindBodyOptions.UnknownFieldContext != "" {
message = "unknown field %q in " + bindBodyOptions.UnknownFieldContext
}
return errors.
NewInvalidInputf(errors.CodeInvalidInput, message, field).
WithSuggestions(errors.SuggestionsOnLevenshteinDistance(field, JSONFieldNames(obj))...)
}
}
return err
}
return nil
}
// JSONFieldNames returns the JSON field names of a struct (or pointer to one),
// skipping fields tagged "-" or without a json tag.
func JSONFieldNames(v any) []string {
var fields []string
t := reflect.TypeOf(v)
if t.Kind() == reflect.Pointer {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return fields
}
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
jsonTag := field.Tag.Get("json")
if jsonTag == "" || jsonTag == "-" {
continue
}
fieldName := strings.Split(jsonTag, ",")[0]
if fieldName != "" {
fields = append(fields, fieldName)
}
}
return fields
}
// extractUnknownField pulls fieldname out of a `json: unknown field "fieldname"`
// decoder message, or returns "" when the message has no quoted field.
func extractUnknownField(errMsg string) string {
parts := strings.Split(errMsg, `"`)
if len(parts) >= 2 {
return parts[1]
}
return ""
}

View File

@@ -58,11 +58,89 @@ func TestJSONBinding_BindBodyErrors(t *testing.T) {
err := JSON.BindBody(strings.NewReader(testCase.body), testCase.obj, testCase.opts...)
assert.Error(t, err)
typ, c, m, _, _, a := errors.Unwrapb(err)
typ, c, m, _, _, _ := errors.Unwrapb(err)
assert.Equal(t, errors.TypeInvalidInput, typ)
assert.Equal(t, testCase.code, c)
assert.Equal(t, testCase.message, m)
assert.ElementsMatch(t, testCase.a, a)
messages := []string{}
for _, additional := range errors.AsJSON(err).Errors {
messages = append(messages, additional.Message)
}
assert.ElementsMatch(t, testCase.a, messages)
})
}
}
type widget struct {
Name string `json:"name"`
Color string `json:"color"`
}
func TestJSONBinding_BindBody_UnknownFieldSuggestions(t *testing.T) {
testCases := []struct {
name string
body string
opts []BindBodyOption
message string
suggestions []string
}{
{
name: "NoNearMatch",
body: `{"shape":"round"}`,
opts: []BindBodyOption{WithDisallowUnknownFields(true)},
message: `unknown field "shape"`,
suggestions: []string{"valid references: `name`, `color`"},
},
{
name: "WithContext",
body: `{"shape":"round"}`,
opts: []BindBodyOption{WithDisallowUnknownFields(true), WithUnknownFieldContext("widget spec")},
message: `unknown field "shape" in widget spec`,
suggestions: []string{"valid references: `name`, `color`"},
},
{
name: "NearMatch",
body: `{"nam":"x"}`,
opts: []BindBodyOption{WithDisallowUnknownFields(true)},
message: `unknown field "nam"`,
suggestions: []string{"did you mean: `name`", "valid references: `name`, `color`"},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
err := JSON.BindBody(strings.NewReader(testCase.body), &widget{}, testCase.opts...)
assert.Error(t, err)
_, c, m, _, _, _ := errors.Unwrapb(err)
if testCase.message != "" {
assert.Equal(t, errors.CodeInvalidInput, c)
assert.Equal(t, testCase.message, m)
}
assert.Equal(t, testCase.suggestions, errors.AsJSON(err).Suggestions)
})
}
}
type structuredUnknownField struct{}
func (*structuredUnknownField) UnmarshalJSON([]byte) error {
return errors.
NewInvalidInputf(errors.CodeInvalidInput, "unknown field %q in inner spec", "foo").
WithSuggestions("did you mean: `bar`")
}
// A non-strict BindBody must pass through an already-structured "unknown field"
// error returned by a nested UnmarshalJSON, not re-wrap it with the outer field set.
func TestJSONBinding_BindBody_PassesThroughStructuredUnknownField(t *testing.T) {
err := JSON.BindBody(strings.NewReader(`{}`), &structuredUnknownField{})
assert.Error(t, err)
_, c, m, _, _, _ := errors.Unwrapb(err)
assert.Equal(t, errors.CodeInvalidInput, c)
assert.Equal(t, `unknown field "foo" in inner spec`, m)
assert.Equal(t, []string{"did you mean: `bar`"}, errors.AsJSON(err).Suggestions)
}

View File

@@ -11,6 +11,7 @@ import (
"github.com/SigNoz/signoz/pkg/analytics"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
@@ -45,7 +46,7 @@ func (handler *handler) QueryRange(rw http.ResponseWriter, req *http.Request) {
}
var queryRangeRequest qbtypes.QueryRangeRequest
if err := json.NewDecoder(req.Body).Decode(&queryRangeRequest); err != nil {
if err := binding.JSON.BindBody(req.Body, &queryRangeRequest); err != nil {
render.Error(rw, err)
return
}
@@ -186,7 +187,7 @@ func (handler *handler) QueryRawStream(rw http.ResponseWriter, req *http.Request
func (handler *handler) ReplaceVariables(rw http.ResponseWriter, req *http.Request) {
var queryRangeRequest qbtypes.QueryRangeRequest
if err := json.NewDecoder(req.Body).Decode(&queryRangeRequest); err != nil {
if err := binding.JSON.BindBody(req.Body, &queryRangeRequest); err != nil {
render.Error(rw, err)
return
}

65
pkg/querier/collect.go Normal file
View File

@@ -0,0 +1,65 @@
package querier
import (
"context"
"fmt"
"time"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/telemetrylogs"
"github.com/SigNoz/signoz/pkg/telemetrymetrics"
"github.com/SigNoz/signoz/pkg/telemetrytraces"
"github.com/SigNoz/signoz/pkg/valuer"
)
func (q *querier) Collect(ctx context.Context, _ valuer.UUID) (map[string]any, error) {
stats := make(map[string]any)
tracesTable := fmt.Sprintf("%s.%s", telemetrytraces.DBName, telemetrytraces.SpanIndexV3TableName)
logsTable := fmt.Sprintf("%s.%s", telemetrylogs.DBName, telemetrylogs.LogsV2TableName)
metricsTable := fmt.Sprintf("%s.%s", telemetrymetrics.DBName, telemetrymetrics.SamplesV4TableName)
var (
traces uint64
tracesLastSeenAt time.Time
)
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), max(timestamp) FROM %s", tracesTable)).Scan(&traces, &tracesLastSeenAt); err == nil {
stats["telemetry.traces.count"] = traces
if tracesLastSeenAt.Unix() != 0 {
stats["telemetry.traces.last_observed.time"] = tracesLastSeenAt.UTC()
stats["telemetry.traces.last_observed.time_unix"] = tracesLastSeenAt.Unix()
}
} else {
q.logger.DebugContext(ctx, "failed to collect traces stats", errors.Attr(err))
}
var (
logs uint64
logsLastSeenAt time.Time
)
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), fromUnixTimestamp64Nano(max(timestamp)) FROM %s", logsTable)).Scan(&logs, &logsLastSeenAt); err == nil {
stats["telemetry.logs.count"] = logs
if logsLastSeenAt.Unix() != 0 {
stats["telemetry.logs.last_observed.time"] = logsLastSeenAt.UTC()
stats["telemetry.logs.last_observed.time_unix"] = logsLastSeenAt.Unix()
}
} else {
q.logger.DebugContext(ctx, "failed to collect logs stats", errors.Attr(err))
}
var (
metrics uint64
metricsLastSeenAt time.Time
)
if err := q.telemetryStore.ClickhouseDB().QueryRow(ctx, fmt.Sprintf("SELECT COUNT(*), toDateTime(max(unix_milli) / 1000) FROM %s", metricsTable)).Scan(&metrics, &metricsLastSeenAt); err == nil {
stats["telemetry.metrics.count"] = metrics
if metricsLastSeenAt.Unix() != 0 {
stats["telemetry.metrics.last_observed.time"] = metricsLastSeenAt.UTC()
stats["telemetry.metrics.last_observed.time_unix"] = metricsLastSeenAt.Unix()
}
} else {
q.logger.DebugContext(ctx, "failed to collect metrics stats", errors.Attr(err))
}
return stats, nil
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"net/http"
"github.com/SigNoz/signoz/pkg/statsreporter"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -12,6 +13,7 @@ import (
type Querier interface {
QueryRange(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest) (*qbtypes.QueryRangeResponse, error)
QueryRawStream(ctx context.Context, orgID valuer.UUID, req *qbtypes.QueryRangeRequest, client *qbtypes.RawStream)
statsreporter.StatsCollector
}
// BucketCache is the interface for bucket-based caching.

View File

@@ -32,7 +32,7 @@ func CollisionHandledFinalExpr(
if requiredDataType != telemetrytypes.FieldDataTypeString &&
requiredDataType != telemetrytypes.FieldDataTypeFloat64 {
return "", nil, errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "unsupported data type %s", requiredDataType)
return "", nil, errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported data type %s", requiredDataType)
}
var dummyValue any
@@ -81,14 +81,8 @@ func CollisionHandledFinalExpr(
// - it is not a static field
// - the next best thing to do is see if there is a typo
// and suggest a correction
correction, found := telemetrytypes.SuggestCorrection(field.Name, maps.Keys(keys))
if found {
// we found a close match, in the error message send the suggestion
return "", nil, errors.WithAdditionalf(fieldForErr, "%s", correction)
} else {
// not even a close match, return an error
return "", nil, errors.WithAdditionalf(fieldForErr, "field `%s` not found", field.Name)
}
wrappedErr := errors.WithSuggestiveAdditionalf(fieldForErr, errors.SuggestionsOnLevenshteinDistance(field.Name, maps.Keys(keys)), "field `%s` not found", field.Name)
return "", nil, wrappedErr
} else {
for _, key := range keysForField {
err := addCondition(key)

View File

@@ -294,24 +294,30 @@ func (r *HavingExpressionRewriter) rewriteAndValidate(expression string) (string
validKeys = append(validKeys, k)
}
sort.Strings(validKeys)
additional := []string{"Valid references are: [" + strings.Join(validKeys, ", ") + "]"}
// Each suggestion is a self-describing string prefixed with either
// "did you mean: " (the full corrected expression) or "valid references: "
// (the set of valid references).
var suggestions []string
if len(v.invalid) == 1 {
inv := v.invalid[0]
// Only suggest for plain identifier typos, not for unresolved function
// calls: a function call will appear as "name(" in the expression, and
// the closest valid key may itself contain "(" (e.g. "sum(a)"), making
// a simple string substitution produce a corrupt expression.
isFuncCall := strings.Contains(original, inv+"(")
if match, dist := closestMatch(inv, validKeys); !isFuncCall && !strings.Contains(match, "(") && dist <= 3 {
corrected := strings.ReplaceAll(original, inv, match)
additional = append(additional, "Suggestion: `"+corrected+"`")
}
suggestions = errors.SuggestionsFromFunc(func() string {
match, ok := errors.ClosestLevenshteinMatch(inv, validKeys)
if !ok || strings.Contains(original, inv+"(") || strings.Contains(match, "(") {
return ""
}
return strings.ReplaceAll(original, inv, match)
})
}
return "", errors.NewInvalidInputf(
suggestions = append(suggestions, errors.ValidReferences(validKeys...))
havingErr := errors.NewInvalidInputf(
errors.CodeInvalidInput,
"Invalid references in `Having` expression: [%s]",
strings.Join(v.invalid, ", "),
).WithAdditional(additional...)
).WithAdditional(
"Valid references are: [" + strings.Join(validKeys, ", ") + "]",
).WithSuggestions(suggestions...)
return "", havingErr
}
// Layer 3 ANTLR syntax errors. We parse the original expression, so error messages
@@ -324,21 +330,22 @@ func (r *HavingExpressionRewriter) rewriteAndValidate(expression string) (string
msgs = append(msgs, m)
}
}
detail := strings.Join(msgs, "; ")
if detail == "" {
detail = "check the expression syntax"
}
additional := []string{detail}
// For single-error expressions, try to produce an actionable suggestion.
if len(allSyntaxErrors) == 1 {
if s := havingSuggestion(allSyntaxErrors[0], original); s != "" {
additional = append(additional, "Suggestion: `"+s+"`")
}
}
return "", errors.NewInvalidInputf(
havingErr := errors.NewInvalidInputf(
errors.CodeInvalidInput,
"Syntax error in `Having` expression",
).WithAdditional(additional...)
)
// A single syntax error can carry an actionable suggestion on the same detail;
// multiple errors are surfaced as one additional detail each. If the parser
// produced no message (rare), the top-level message stands on its own.
if len(allSyntaxErrors) == 1 && len(msgs) == 1 {
suggestions := errors.SuggestionsFromFunc(func() string {
return havingSuggestion(allSyntaxErrors[0], original)
})
return "", havingErr.WithSuggestiveAdditional(msgs[0], suggestions...)
}
return "", havingErr.WithAdditional(msgs...)
}
return result, nil
@@ -448,42 +455,6 @@ func hasUnclosedBracket(s string) bool {
return count > 0
}
// closestMatch returns the element of candidates with the smallest Levenshtein
// distance to query, along with that distance.
func closestMatch(query string, candidates []string) (string, int) {
best, bestDist := "", -1
for _, c := range candidates {
if d := levenshtein(query, c); bestDist < 0 || d < bestDist {
best, bestDist = c, d
}
}
return best, bestDist
}
// levenshtein computes the edit distance between a and b.
func levenshtein(a, b string) int {
ra, rb := []rune(a), []rune(b)
la, lb := len(ra), len(rb)
row := make([]int, lb+1)
for j := range row {
row[j] = j
}
for i := 1; i <= la; i++ {
prev := row[0]
row[0] = i
for j := 1; j <= lb; j++ {
tmp := row[j]
if ra[i-1] == rb[j-1] {
row[j] = prev
} else {
row[j] = 1 + min(prev, min(row[j], row[j-1]))
}
prev = tmp
}
}
return row[lb]
}
// endsWithComparisonOp reports whether s ends with a comparison operator token
// (longer operators are checked first to avoid ">=" being matched by ">").
func endsWithComparisonOp(s string) bool {

View File

@@ -18,14 +18,39 @@ func toTraceAggregations(logs []qbtypes.LogAggregation) []qbtypes.TraceAggregati
return out
}
// additionalMessages extracts the message of each additional detail on err, so tests
// can compare against a plain []string.
func additionalMessages(err error) []string {
var msgs []string
for _, e := range errors.AsJSON(err).Errors {
msgs = append(msgs, e.Message)
}
return msgs
}
// allSuggestions collects suggestions from both the error-wide list and every additional
// detail, so tests can assert suggestions regardless of where they are attached.
func allSuggestions(err error) []string {
j := errors.AsJSON(err)
s := append([]string{}, j.Suggestions...)
for _, e := range j.Errors {
s = append(s, e.Suggestions...)
}
if len(s) == 0 {
return nil
}
return s
}
type logsAndTracesTestCase struct {
name string
expression string
aggregations []qbtypes.LogAggregation
wantExpression string
wantErr bool
wantErrMsg string
wantAdditional []string
name string
expression string
aggregations []qbtypes.LogAggregation
wantExpression string
wantErr bool
wantErrMsg string
wantAdditional []string
wantSuggestions []string
}
func runLogsAndTracesTests(t *testing.T, tests []logsAndTracesTestCase) {
@@ -40,12 +65,12 @@ func runLogsAndTracesTests(t *testing.T, tests []logsAndTracesTestCase) {
if tt.wantErr {
require.Error(t, errLogs)
assert.ErrorContains(t, errLogs, tt.wantErrMsg)
_, _, _, _, _, additionalLogs := errors.Unwrapb(errLogs)
assert.Equal(t, tt.wantAdditional, additionalLogs)
assert.Equal(t, tt.wantAdditional, additionalMessages(errLogs))
assert.Equal(t, tt.wantSuggestions, allSuggestions(errLogs))
require.Error(t, errTraces)
assert.ErrorContains(t, errTraces, tt.wantErrMsg)
_, _, _, _, _, additionalTraces := errors.Unwrapb(errTraces)
assert.Equal(t, tt.wantAdditional, additionalTraces)
assert.Equal(t, tt.wantAdditional, additionalMessages(errTraces))
assert.Equal(t, tt.wantSuggestions, allSuggestions(errTraces))
} else {
require.NoError(t, errLogs)
assert.Equal(t, tt.wantExpression, gotLogs)
@@ -290,9 +315,10 @@ func TestRewriteForLogsAndTraces_BooleanOperators(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total_logs"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:20 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got EOF", "Suggestion: `total_logs > 100`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:20 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got EOF"},
wantSuggestions: []string{"did you mean: `total_logs > 100`"},
},
{
name: "dangling OR at start",
@@ -300,9 +326,10 @@ func TestRewriteForLogsAndTraces_BooleanOperators(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total_logs"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:0 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got 'OR'", "Suggestion: `total_logs > 100`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:0 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got 'OR'"},
wantSuggestions: []string{"did you mean: `total_logs > 100`"},
},
{
name: "dangling OR at end",
@@ -310,9 +337,10 @@ func TestRewriteForLogsAndTraces_BooleanOperators(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:14 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got EOF", "Suggestion: `total > 100`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:14 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got EOF"},
wantSuggestions: []string{"did you mean: `total > 100`"},
},
{
name: "consecutive AND operators",
@@ -562,9 +590,10 @@ func TestRewriteForLogsAndTraces_InOperator(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [ghost]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [ghost]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `count()`, `total`"},
},
{
name: "IN with end bracked missing",
@@ -572,9 +601,10 @@ func TestRewriteForLogsAndTraces_InOperator(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:19 expecting one of {]} but got EOF", "Suggestion: `count() IN [1, 2, 3]`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:19 expecting one of {]} but got EOF"},
wantSuggestions: []string{"did you mean: `count() IN [1, 2, 3]`"},
},
{
name: "IN with end paran missing",
@@ -582,9 +612,10 @@ func TestRewriteForLogsAndTraces_InOperator(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:19 expecting one of {)} but got EOF", "Suggestion: `count() IN (1, 2, 3)`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:19 expecting one of {)} but got EOF"},
wantSuggestions: []string{"did you mean: `count() IN (1, 2, 3)`"},
},
})
}
@@ -621,9 +652,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [unknown_alias]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [unknown_alias]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `count()`, `total`"},
},
{
name: "typo in identifier suggests closest match",
@@ -631,9 +663,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [totol]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]", "Suggestion: `total > 100`"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [totol]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]"},
wantSuggestions: []string{"did you mean: `total > 100`", "valid references: `__result`, `__result0`, `count()`, `total`"},
},
{
name: "expression not in column map",
@@ -641,9 +674,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [sum]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [sum]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `count()`"},
},
{
name: "one valid one invalid reference",
@@ -651,9 +685,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [ghost]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [ghost]",
wantAdditional: []string{"Valid references are: [__result, __result0, count(), total]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `count()`, `total`"},
},
{
name: "__result ambiguous with multiple aggregations",
@@ -662,9 +697,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
{Expression: "count()"},
{Expression: "sum(bytes)"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [__result]",
wantAdditional: []string{"Valid references are: [__result0, __result1, count(), sum(bytes)]", "Suggestion: `__result0 > 100`"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [__result]",
wantAdditional: []string{"Valid references are: [__result0, __result1, count(), sum(bytes)]"},
wantSuggestions: []string{"did you mean: `__result0 > 100`", "valid references: `__result0`, `__result1`, `count()`, `sum(bytes)`"},
},
{
name: "out-of-range __result_N index",
@@ -672,9 +708,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [__result_9]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]", "Suggestion: `__result > 100`"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [__result_9]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]"},
wantSuggestions: []string{"did you mean: `__result > 100`", "valid references: `__result`, `__result0`, `count()`"},
},
{
name: "__result_1 out of range for single aggregation",
@@ -682,9 +719,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [__result_1]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]", "Suggestion: `__result > 100`"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [__result_1]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]"},
wantSuggestions: []string{"did you mean: `__result > 100`", "valid references: `__result`, `__result0`, `count()`"},
},
{
name: "cascaded function calls",
@@ -692,9 +730,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [sum]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [sum]",
wantAdditional: []string{"Valid references are: [__result, __result0, count()]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `count()`"},
},
{
name: "function call with multiple args not in column map",
@@ -702,9 +741,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "sum(a)"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [sum]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(a)]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [sum]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(a)]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `sum(a)`"},
},
{
name: "unquoted string value treated as unknown identifier",
@@ -712,9 +752,10 @@ func TestRewriteForLogsAndTraces_ErrorInvalidReferences(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "sum(bytes)"},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [xyz]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(bytes)]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [xyz]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(bytes)]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `sum(bytes)`"},
},
})
}
@@ -731,9 +772,10 @@ func TestRewriteForLogsAndTraces_ErrorSyntax(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total_logs"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:7 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF", "Suggestion: `count() > 0`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:7 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF"},
wantSuggestions: []string{"did you mean: `count() > 0`"},
},
{
name: "bare identifier without comparison",
@@ -741,9 +783,10 @@ func TestRewriteForLogsAndTraces_ErrorSyntax(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total_logs"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:10 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF", "Suggestion: `total_logs > 0`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:10 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF"},
wantSuggestions: []string{"did you mean: `total_logs > 0`"},
},
// Parenthesis mismatches
{
@@ -752,9 +795,10 @@ func TestRewriteForLogsAndTraces_ErrorSyntax(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()", Alias: "total_logs"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:35 expecting one of {)} but got EOF", "Suggestion: `(total_logs > 100 AND count() < 500)`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:35 expecting one of {)} but got EOF"},
wantSuggestions: []string{"did you mean: `(total_logs > 100 AND count() < 500)`"},
},
{
name: "unexpected closing parenthesis",
@@ -805,7 +849,7 @@ func TestRewriteForLogsAndTraces_ErrorSyntax(t *testing.T) {
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:0 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got '>'; line 1:5 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF"},
wantAdditional: []string{"line 1:0 expecting one of {'*', '+', '-', (, ), AND, IDENTIFIER, NOT, number, string} but got '>'", "line 1:5 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF"},
},
{
name: "missing right operand",
@@ -813,9 +857,10 @@ func TestRewriteForLogsAndTraces_ErrorSyntax(t *testing.T) {
aggregations: []qbtypes.LogAggregation{
{Expression: "count()"},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:9 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF", "Suggestion: `count() > 0`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:9 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF"},
wantSuggestions: []string{"did you mean: `count() > 0`"},
},
{
name: "missing comparison operator",
@@ -875,13 +920,14 @@ func TestRewriteForLogsAndTraces_ErrorSyntax(t *testing.T) {
func TestRewriteForMetrics(t *testing.T) {
tests := []struct {
name string
expression string
aggregations []qbtypes.MetricAggregation
wantExpression string
wantErr bool
wantErrMsg string
wantAdditional []string
name string
expression string
aggregations []qbtypes.MetricAggregation
wantExpression string
wantErr bool
wantErrMsg string
wantAdditional []string
wantSuggestions []string
}{
// --- Happy path: reference types (time/space aggregation, __result, bare metric) ---
{
@@ -981,9 +1027,10 @@ func TestRewriteForMetrics(t *testing.T) {
SpaceAggregation: metrictypes.SpaceAggregationUnspecified,
},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [wrong_metric]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(cpu_usage)]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [wrong_metric]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(cpu_usage)]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `sum(cpu_usage)`"},
},
// --- Error: string literal (not allowed in HAVING) ---
{
@@ -1011,9 +1058,10 @@ func TestRewriteForMetrics(t *testing.T) {
SpaceAggregation: metrictypes.SpaceAggregationUnspecified,
},
},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:9 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF", "Suggestion: `cpu_usage > 0`"},
wantErr: true,
wantErrMsg: "Syntax error in `Having` expression",
wantAdditional: []string{"line 1:9 expecting one of {'*', '+', '-', (, ), IDENTIFIER, number, string} but got EOF"},
wantSuggestions: []string{"did you mean: `cpu_usage > 0`"},
},
// --- Error: aggregation not in column map ---
{
@@ -1026,9 +1074,10 @@ func TestRewriteForMetrics(t *testing.T) {
SpaceAggregation: metrictypes.SpaceAggregationUnspecified,
},
},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [count]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(cpu_usage)]"},
wantErr: true,
wantErrMsg: "Invalid references in `Having` expression: [count]",
wantAdditional: []string{"Valid references are: [__result, __result0, sum(cpu_usage)]"},
wantSuggestions: []string{"valid references: `__result`, `__result0`, `sum(cpu_usage)`"},
},
}
@@ -1039,8 +1088,8 @@ func TestRewriteForMetrics(t *testing.T) {
if tt.wantErr {
require.Error(t, err)
assert.ErrorContains(t, err, tt.wantErrMsg)
_, _, _, _, _, additional := errors.Unwrapb(err)
assert.Equal(t, tt.wantAdditional, additional)
assert.Equal(t, tt.wantAdditional, additionalMessages(err))
assert.Equal(t, tt.wantSuggestions, allSuggestions(err))
} else {
require.NoError(t, err)
assert.Equal(t, tt.wantExpression, got)

View File

@@ -49,6 +49,7 @@ import (
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/ruler/signozruler"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/zeus"
)
@@ -80,6 +81,7 @@ type Handlers struct {
TraceDetail tracedetail.Handler
RulerHandler ruler.Handler
LLMPricingRuleHandler llmpricingrule.Handler
StatsHandler statsreporter.Handler
}
func NewHandlers(
@@ -97,6 +99,7 @@ func NewHandlers(
registryHandler factory.Handler,
alertmanagerService alertmanager.Alertmanager,
rulerService ruler.Ruler,
statsAggregator statsreporter.Aggregator,
) Handlers {
return Handlers{
SavedView: implsavedview.NewHandler(modules.SavedView),
@@ -125,5 +128,6 @@ func NewHandlers(
TraceDetail: impltracedetail.NewHandler(modules.TraceDetail),
RulerHandler: signozruler.NewHandler(rulerService),
LLMPricingRuleHandler: impllmpricingrule.NewHandler(modules.LLMPricingRule),
StatsHandler: statsreporter.NewHandler(statsAggregator),
}
}

View File

@@ -63,7 +63,7 @@ func TestNewHandlers(t *testing.T) {
querierHandler := querier.NewHandler(providerSettings, nil, nil)
registryHandler := factory.NewHandler(nil)
handlers := NewHandlers(modules, providerSettings, nil, querierHandler, nil, nil, nil, nil, nil, nil, nil, registryHandler, alertmanager, nil)
handlers := NewHandlers(modules, providerSettings, nil, querierHandler, nil, nil, nil, nil, nil, nil, nil, registryHandler, alertmanager, nil, nil)
reflectVal := reflect.ValueOf(handlers)
for i := 0; i < reflectVal.NumField(); i++ {
f := reflectVal.Field(i)

View File

@@ -36,6 +36,7 @@ import (
"github.com/SigNoz/signoz/pkg/modules/user"
"github.com/SigNoz/signoz/pkg/querier"
"github.com/SigNoz/signoz/pkg/ruler"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/zeus"
"github.com/swaggest/jsonschema-go"
@@ -83,6 +84,7 @@ func NewOpenAPI(ctx context.Context, instrumentation instrumentation.Instrumenta
struct{ llmpricingrule.Handler }{},
struct{ tracedetail.Handler }{},
struct{ ruler.Handler }{},
struct{ statsreporter.Handler }{},
).New(ctx, instrumentation.ToProviderSettings(), apiserver.Config{})
if err != nil {
return nil, err

View File

@@ -265,9 +265,9 @@ func NewSharderProviderFactories() factory.NamedMap[factory.ProviderFactory[shar
)
}
func NewStatsReporterProviderFactories(telemetryStore telemetrystore.TelemetryStore, collectors []statsreporter.StatsCollector, orgGetter organization.Getter, userGetter user.Getter, tokenizer tokenizer.Tokenizer, build version.Build, analyticsConfig analytics.Config) factory.NamedMap[factory.ProviderFactory[statsreporter.StatsReporter, statsreporter.Config]] {
func NewStatsReporterProviderFactories(aggregator statsreporter.Aggregator, orgGetter organization.Getter, userGetter user.Getter, tokenizer tokenizer.Tokenizer, build version.Build, analyticsConfig analytics.Config) factory.NamedMap[factory.ProviderFactory[statsreporter.StatsReporter, statsreporter.Config]] {
return factory.MustNewNamedMap(
analyticsstatsreporter.NewFactory(telemetryStore, collectors, orgGetter, userGetter, tokenizer, build, analyticsConfig),
analyticsstatsreporter.NewFactory(aggregator, orgGetter, userGetter, tokenizer, build, analyticsConfig),
noopstatsreporter.NewFactory(),
)
}
@@ -310,6 +310,7 @@ func NewAPIServerProviderFactories(orgGetter organization.Getter, authz authz.Au
handlers.LLMPricingRuleHandler,
handlers.TraceDetail,
handlers.RulerHandler,
handlers.StatsHandler,
),
)
}

View File

@@ -85,8 +85,8 @@ func TestNewProviderFactories(t *testing.T) {
userGetter := impluser.NewGetter(impluser.NewStore(sqlstoretest.New(sqlstore.Config{Provider: "sqlite"}, sqlmock.QueryMatcherEqual), instrumentationtest.New().ToProviderSettings()), userRoleStore, flagger)
orgGetter := implorganization.NewGetter(implorganization.NewStore(sqlstoretest.New(sqlstore.Config{Provider: "sqlite"}, sqlmock.QueryMatcherEqual)), nil)
telemetryStore := telemetrystoretest.New(telemetrystore.Config{Provider: "clickhouse"}, sqlmock.QueryMatcherEqual)
NewStatsReporterProviderFactories(telemetryStore, []statsreporter.StatsCollector{}, orgGetter, userGetter, tokenizertest.NewMockTokenizer(t), version.Build{}, analytics.Config{Enabled: true})
statsAggregator := statsreporter.NewAggregator(providerSettings, []statsreporter.StatsCollector{})
NewStatsReporterProviderFactories(statsAggregator, orgGetter, userGetter, tokenizertest.NewMockTokenizer(t), version.Build{}, analytics.Config{Enabled: true})
})
assert.NotPanics(t, func() {

View File

@@ -499,14 +499,18 @@ func New(
serviceAccount,
cloudIntegrationModule,
modules.LogsPipeline,
querier,
}
// Initialize the stats aggregator (always-on, independent of whether reporting is enabled)
statsAggregator := statsreporter.NewAggregator(providerSettings, statsCollectors)
// Initialize stats reporter from the available stats reporter provider factories
statsReporter, err := factory.NewProviderFromNamedMap(
ctx,
providerSettings,
config.StatsReporter,
NewStatsReporterProviderFactories(telemetrystore, statsCollectors, orgGetter, userGetter, tokenizer, version.Info, config.Analytics),
NewStatsReporterProviderFactories(statsAggregator, orgGetter, userGetter, tokenizer, version.Info, config.Analytics),
config.StatsReporter.Provider(),
)
if err != nil {
@@ -535,7 +539,7 @@ func New(
// Initialize all handlers for the modules
registryHandler := factory.NewHandler(registry)
handlers := NewHandlers(modules, providerSettings, analytics, querierHandler, licensing, global, flagger, gateway, telemetryMetadataStore, authz, zeus, registryHandler, alertmanager, rulerInstance)
handlers := NewHandlers(modules, providerSettings, analytics, querierHandler, licensing, global, flagger, gateway, telemetryMetadataStore, authz, zeus, registryHandler, alertmanager, rulerInstance, statsAggregator)
// Initialize the API server (after registry so it can access service health)
apiserverInstance, err := factory.NewProviderFromNamedMap(

View File

@@ -0,0 +1,67 @@
package statsreporter
import (
"context"
"sync"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
// Aggregator aggregates stats from every registered StatsCollector for a single organization.
type Aggregator interface {
Aggregate(ctx context.Context, orgID valuer.UUID) (map[string]any, error)
}
type aggregator struct {
// settings
settings factory.ScopedProviderSettings
// a list of collectors, used to collect stats from across the codebase
collectors []StatsCollector
}
func NewAggregator(providerSettings factory.ProviderSettings, collectors []StatsCollector) Aggregator {
settings := factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/statsreporter")
return &aggregator{
settings: settings,
collectors: collectors,
}
}
func (aggregator *aggregator) Aggregate(ctx context.Context, orgID valuer.UUID) (map[string]any, error) {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "statsreporter",
instrumentationtypes.CodeFunctionName: "Aggregate",
})
var wg sync.WaitGroup
wg.Add(len(aggregator.collectors))
stats := make(map[string]any, 0)
mtx := sync.Mutex{}
for _, collector := range aggregator.collectors {
go func(collector StatsCollector) {
defer wg.Done()
collectorStats, err := collector.Collect(ctx, orgID)
if err != nil {
aggregator.settings.Logger().ErrorContext(ctx, "failed to collect stats", errors.Attr(err))
return
}
mtx.Lock()
for k, v := range collectorStats {
stats[k] = v
}
mtx.Unlock()
}(collector)
}
wg.Wait()
return stats, nil
}

View File

@@ -3,7 +3,6 @@ package analyticsstatsreporter
import (
"context"
"log/slog"
"sync"
"time"
"go.opentelemetry.io/otel/attribute"
@@ -16,11 +15,8 @@ import (
"github.com/SigNoz/signoz/pkg/modules/organization"
"github.com/SigNoz/signoz/pkg/modules/user"
"github.com/SigNoz/signoz/pkg/statsreporter"
"github.com/SigNoz/signoz/pkg/telemetrystore"
"github.com/SigNoz/signoz/pkg/tokenizer"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/ctxtypes"
"github.com/SigNoz/signoz/pkg/types/instrumentationtypes"
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/SigNoz/signoz/pkg/version"
)
@@ -32,11 +28,8 @@ type provider struct {
// config
config statsreporter.Config
// used to get telemetry details. srikanthcvv to move this to the querier layer
telemetryStore telemetrystore.TelemetryStore
// a list of collectors, used to collect stats from across the codebase
collectors []statsreporter.StatsCollector
// used to aggregate stats for an organization
aggregator statsreporter.Aggregator
// used to get organizations
orgGetter organization.Getter
@@ -60,9 +53,9 @@ type provider struct {
stopC chan struct{}
}
func NewFactory(telemetryStore telemetrystore.TelemetryStore, collectors []statsreporter.StatsCollector, orgGetter organization.Getter, userGetter user.Getter, tokenizer tokenizer.Tokenizer, build version.Build, analyticsConfig analytics.Config) factory.ProviderFactory[statsreporter.StatsReporter, statsreporter.Config] {
func NewFactory(aggregator statsreporter.Aggregator, orgGetter organization.Getter, userGetter user.Getter, tokenizer tokenizer.Tokenizer, build version.Build, analyticsConfig analytics.Config) factory.ProviderFactory[statsreporter.StatsReporter, statsreporter.Config] {
return factory.NewProviderFactory(factory.MustNewName("analytics"), func(ctx context.Context, settings factory.ProviderSettings, config statsreporter.Config) (statsreporter.StatsReporter, error) {
return New(ctx, settings, config, telemetryStore, collectors, orgGetter, userGetter, tokenizer, build, analyticsConfig)
return New(ctx, settings, config, aggregator, orgGetter, userGetter, tokenizer, build, analyticsConfig)
})
}
@@ -70,8 +63,7 @@ func New(
ctx context.Context,
providerSettings factory.ProviderSettings,
config statsreporter.Config,
telemetryStore telemetrystore.TelemetryStore,
collectors []statsreporter.StatsCollector,
aggregator statsreporter.Aggregator,
orgGetter organization.Getter,
userGetter user.Getter,
tokenizer tokenizer.Tokenizer,
@@ -86,17 +78,16 @@ func New(
}
return &provider{
settings: settings,
config: config,
telemetryStore: telemetryStore,
collectors: collectors,
orgGetter: orgGetter,
userGetter: userGetter,
analytics: analytics,
tokenizer: tokenizer,
build: build,
deployment: deployment,
stopC: make(chan struct{}),
settings: settings,
config: config,
aggregator: aggregator,
orgGetter: orgGetter,
userGetter: userGetter,
analytics: analytics,
tokenizer: tokenizer,
build: build,
deployment: deployment,
stopC: make(chan struct{}),
}, nil
}
@@ -134,7 +125,12 @@ func (provider *provider) Report(ctx context.Context) error {
}
for _, org := range orgs {
stats := provider.collectOrg(ctx, org.ID)
stats, err := provider.aggregator.Aggregate(ctx, org.ID)
if err != nil {
provider.settings.Logger().WarnContext(ctx, "failed to aggregate stats", errors.Attr(err), slog.Any("org_id", org.ID))
continue
}
if len(stats) == 0 {
provider.settings.Logger().WarnContext(ctx, "no stats collected", slog.Any("org_id", org.ID))
continue
@@ -204,75 +200,3 @@ func (provider *provider) Stop(ctx context.Context) error {
return nil
}
func (provider *provider) collectOrg(ctx context.Context, orgID valuer.UUID) map[string]any {
ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
instrumentationtypes.CodeNamespace: "statsreporter",
instrumentationtypes.CodeFunctionName: "collectOrg",
})
var wg sync.WaitGroup
wg.Add(len(provider.collectors))
stats := make(map[string]any, 0)
mtx := sync.Mutex{}
for _, collector := range provider.collectors {
go func(collector statsreporter.StatsCollector) {
defer wg.Done()
collectorStats, err := collector.Collect(ctx, orgID)
if err != nil {
provider.settings.Logger().ErrorContext(ctx, "failed to collect stats", errors.Attr(err))
return
}
mtx.Lock()
for k, v := range collectorStats {
stats[k] = v
}
mtx.Unlock()
}(collector)
}
wg.Wait()
var traces uint64
if err := provider.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT COUNT(*) FROM signoz_traces.distributed_signoz_index_v3").Scan(&traces); err == nil {
stats["telemetry.traces.count"] = traces
}
var logs uint64
if err := provider.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT COUNT(*) FROM signoz_logs.distributed_logs_v2").Scan(&logs); err == nil {
stats["telemetry.logs.count"] = logs
}
var metrics uint64
if err := provider.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT COUNT(*) FROM signoz_metrics.distributed_samples_v4").Scan(&metrics); err == nil {
stats["telemetry.metrics.count"] = metrics
}
var tracesLastSeenAt time.Time
if err := provider.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT max(timestamp) FROM signoz_traces.distributed_signoz_index_v3").Scan(&tracesLastSeenAt); err == nil {
if tracesLastSeenAt.Unix() != 0 {
stats["telemetry.traces.last_observed.time"] = tracesLastSeenAt.UTC()
stats["telemetry.traces.last_observed.time_unix"] = tracesLastSeenAt.Unix()
}
}
var logsLastSeenAt time.Time
if err := provider.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT fromUnixTimestamp64Nano(max(timestamp)) FROM signoz_logs.distributed_logs_v2").Scan(&logsLastSeenAt); err == nil {
if logsLastSeenAt.Unix() != 0 {
stats["telemetry.logs.last_observed.time"] = logsLastSeenAt.UTC()
stats["telemetry.logs.last_observed.time_unix"] = logsLastSeenAt.Unix()
}
}
var metricsLastSeenAt time.Time
if err := provider.telemetryStore.ClickhouseDB().QueryRow(ctx, "SELECT toDateTime(max(unix_milli) / 1000) FROM signoz_metrics.distributed_samples_v4").Scan(&metricsLastSeenAt); err == nil {
if metricsLastSeenAt.Unix() != 0 {
stats["telemetry.metrics.last_observed.time"] = metricsLastSeenAt.UTC()
stats["telemetry.metrics.last_observed.time_unix"] = metricsLastSeenAt.Unix()
}
}
return stats
}

View File

@@ -0,0 +1,46 @@
package statsreporter
import (
"context"
"net/http"
"time"
"github.com/SigNoz/signoz/pkg/http/render"
"github.com/SigNoz/signoz/pkg/types/authtypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
type Handler interface {
Get(http.ResponseWriter, *http.Request)
}
type handler struct {
aggregator Aggregator
}
func NewHandler(aggregator Aggregator) Handler {
return &handler{
aggregator: aggregator,
}
}
func (handler *handler) Get(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
claims, err := authtypes.ClaimsFromContext(ctx)
if err != nil {
render.Error(rw, err)
return
}
orgID := valuer.MustNewUUID(claims.OrgID)
stats, err := handler.aggregator.Aggregate(ctx, orgID)
if err != nil {
render.Error(rw, err)
return
}
render.Success(rw, http.StatusOK, stats)
}

View File

@@ -57,14 +57,14 @@ func (m *fieldMapper) FieldFor(ctx context.Context, _, _ uint64, key *telemetryt
return "", err
}
if len(columns) != 1 {
return "", errors.Newf(errors.TypeInternal, errors.CodeInternal, "expected exactly 1 column, got %d", len(columns))
return "", errors.NewInternalf(errors.CodeInternal, "expected exactly 1 column, got %d", len(columns))
}
column := columns[0]
switch column.Type.GetType() {
case schema.ColumnTypeEnumJSON:
if key.FieldContext != telemetrytypes.FieldContextResource {
return "", errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource context fields are supported for json columns in audit, got %s", key.FieldContext.String)
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "only resource context fields are supported for json columns in audit, got %s", key.FieldContext.String)
}
return fmt.Sprintf("%s.`%s`::String", column.Name, key.Name), nil
case schema.ColumnTypeEnumLowCardinality:
@@ -109,11 +109,8 @@ func (m *fieldMapper) ColumnExpressionFor(
field.FieldContext = telemetrytypes.FieldContextLog
fieldExpression, _ = m.FieldFor(ctx, tsStart, tsEnd, field)
} else {
correction, found := telemetrytypes.SuggestCorrection(field.Name, maps.Keys(keys))
if found {
return "", errors.Wrap(err, errors.TypeInvalidInput, errors.CodeInvalidInput, correction)
}
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name)
wrappedErr := errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.SuggestionsOnLevenshteinDistance(field.Name, maps.Keys(keys))...)
return "", wrappedErr
}
} else {
fieldExpression, _ = m.FieldFor(ctx, tsStart, tsEnd, keysForField[0])

View File

@@ -5,12 +5,12 @@ import (
"testing"
"time"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/instrumentation/instrumentationtest"
"github.com/SigNoz/signoz/pkg/querybuilder"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes/telemetrytypestest"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/stretchr/testify/require"
)

View File

@@ -183,7 +183,7 @@ func (m *fieldMapper) FieldFor(ctx context.Context, tsStart, tsEnd uint64, key *
exprs = append(exprs, expr)
default:
return "", errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "only resource/body context fields are supported for json columns, got %s", key.FieldContext.String)
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "only resource/body context fields are supported for json columns, got %s", key.FieldContext.String)
}
case schema.ColumnTypeEnumLowCardinality:
@@ -223,7 +223,7 @@ func (m *fieldMapper) FieldFor(ctx context.Context, tsStart, tsEnd uint64, key *
} else if len(exprs) > 1 {
// Ensure existExpr has the same length as exprs
if len(existExpr) != len(exprs) {
return "", errors.New(errors.TypeInternal, errors.CodeInternal, "length of exist exprs doesn't match to that of exprs")
return "", errors.NewInternalf(errors.CodeInternal, "length of exist exprs doesn't match to that of exprs")
}
finalExprs := []string{}
for i, expr := range exprs {
@@ -263,14 +263,8 @@ func (m *fieldMapper) ColumnExpressionFor(
// - it is not a static field
// - the next best thing to do is see if there is a typo
// and suggest a correction
correction, found := telemetrytypes.SuggestCorrection(field.Name, maps.Keys(keys))
if found {
// we found a close match, in the error message send the suggestion
return "", errors.Wrap(err, errors.TypeInvalidInput, errors.CodeInvalidInput, correction)
} else {
// not even a close match, return an error
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name)
}
wrappedErr := errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.SuggestionsOnLevenshteinDistance(field.Name, maps.Keys(keys))...)
return "", wrappedErr
}
} else if len(keysForField) == 1 {
// we have a single key for the field, use it
@@ -295,7 +289,7 @@ func (m *fieldMapper) ColumnExpressionFor(
func (m *fieldMapper) buildFieldForJSON(key *telemetrytypes.TelemetryFieldKey) (string, error) {
plan := key.JSONPlan
if len(plan) == 0 {
return "", errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput,
return "", errors.NewInvalidInputf(errors.CodeInvalidInput,
"Could not find any valid paths for: %s", key.Name)
}
@@ -350,7 +344,7 @@ func (m *fieldMapper) buildFieldForJSON(key *telemetrytypes.TelemetryFieldKey) (
// buildArrayConcat builds the arrayConcat pattern directly from the tree structure.
func (m *fieldMapper) buildArrayConcat(plan telemetrytypes.JSONAccessPlan) (string, error) {
if len(plan) == 0 {
return "", errors.Newf(errors.TypeInternal, CodeGroupByPlanEmpty, "group by plan is empty while building arrayConcat")
return "", errors.NewInternalf(CodeGroupByPlanEmpty, "group by plan is empty while building arrayConcat")
}
// Build arrayMap expressions for ALL available branches at the root level.
@@ -366,7 +360,7 @@ func (m *fieldMapper) buildArrayConcat(plan telemetrytypes.JSONAccessPlan) (stri
}
}
if len(arrayMapExpressions) == 0 {
return "", errors.Newf(errors.TypeInternal, CodeArrayMapExpressionsEmpty, "array map expressions are empty while building arrayConcat")
return "", errors.NewInternalf(CodeArrayMapExpressionsEmpty, "array map expressions are empty while building arrayConcat")
}
// Build the arrayConcat expression
@@ -381,12 +375,12 @@ func (m *fieldMapper) buildArrayConcat(plan telemetrytypes.JSONAccessPlan) (stri
// buildArrayMap builds the arrayMap expression for a specific branch, handling all sub-branches.
func (m *fieldMapper) buildArrayMap(currentNode *telemetrytypes.JSONAccessNode, branchType telemetrytypes.JSONAccessBranchType) (string, error) {
if currentNode == nil {
return "", errors.Newf(errors.TypeInternal, CodeCurrentNodeNil, "current node is nil while building arrayMap")
return "", errors.NewInternalf(CodeCurrentNodeNil, "current node is nil while building arrayMap")
}
childNode := currentNode.Branches[branchType]
if childNode == nil {
return "", errors.Newf(errors.TypeInternal, CodeChildNodeNil, "child node is nil while building arrayMap")
return "", errors.NewInternalf(CodeChildNodeNil, "child node is nil while building arrayMap")
}
// Build the array expression for this level
@@ -427,7 +421,7 @@ func (m *fieldMapper) buildArrayMap(currentNode *telemetrytypes.JSONAccessNode,
} else if len(nestedExpressions) > 1 {
nestedExpr = fmt.Sprintf("arrayConcat(%s)", strings.Join(nestedExpressions, ", "))
} else {
return "", errors.Newf(errors.TypeInternal, CodeNestedExpressionsEmpty, "nested expressions are empty while building arrayMap")
return "", errors.NewInternalf(CodeNestedExpressionsEmpty, "nested expressions are empty while building arrayMap")
}
return fmt.Sprintf("arrayMap(%s->%s, %s)", currentNode.Alias(), nestedExpr, arrayExpr), nil

View File

@@ -16,6 +16,22 @@ import (
"github.com/stretchr/testify/require"
)
// detailContains reports whether any additional detail on err (its message or one
// of its suggestions) contains sub.
func detailContains(err error, sub string) bool {
for _, e := range errors.AsJSON(err).Errors {
if strings.Contains(e.Message, sub) {
return true
}
for _, s := range e.Suggestions {
if strings.Contains(s, sub) {
return true
}
}
}
return false
}
// TestFilterExprLogs tests a comprehensive set of query patterns for logs search.
func TestFilterExprLogs(t *testing.T) {
fl := flaggertest.New(t)
@@ -2415,15 +2431,7 @@ func TestFilterExprLogs(t *testing.T) {
require.Equal(t, tc.expectedArgs, args)
} else {
require.Error(t, err, "Expected error for query: %s", tc.query)
_, _, _, _, _, a := errors.Unwrapb(err)
contains := false
for _, warn := range a {
if strings.Contains(warn, tc.expectedErrorContains) {
contains = true
break
}
}
require.True(t, contains)
require.True(t, detailContains(err, tc.expectedErrorContains))
}
})
}
@@ -2536,15 +2544,7 @@ func TestFilterExprLogsConflictNegation(t *testing.T) {
require.Equal(t, tc.expectedArgs, args)
} else {
require.Error(t, err, "Expected error for query: %s", tc.query)
_, _, _, _, _, a := errors.Unwrapb(err)
contains := false
for _, warn := range a {
if strings.Contains(warn, tc.expectedErrorContains) {
contains = true
break
}
}
require.True(t, contains)
require.True(t, detailContains(err, tc.expectedErrorContains))
}
})
}

View File

@@ -91,14 +91,8 @@ func (m *fieldMapper) ColumnExpressionFor(
// - it is not a static field
// - the next best thing to do is see if there is a typo
// and suggest a correction
correction, found := telemetrytypes.SuggestCorrection(field.Name, maps.Keys(keys))
if found {
// we found a close match, in the error message send the suggestion
return "", errors.Wrap(err, errors.TypeInvalidInput, errors.CodeInvalidInput, correction)
} else {
// not even a close match, return an error
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name)
}
wrappedErr := errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.SuggestionsOnLevenshteinDistance(field.Name, maps.Keys(keys))...)
return "", wrappedErr
}
} else if len(keysForField) == 1 {
// we have a single key for the field, use it

View File

@@ -324,8 +324,7 @@ func AggregationColumnForSamplesTable(
}
}
if aggregationColumn == "" {
return "", errors.Newf(
errors.TypeInvalidInput,
return "", errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid time aggregation, should be one of the following: [`latest`, `sum`, `avg`, `min`, `max`, `count`, `rate`, `increase`]",
)
@@ -335,7 +334,7 @@ func AggregationColumnForSamplesTable(
func AggregationQueryForHistogramCountWithParams(param *metrictypes.ComparisonSpaceAggregationParam) (string, error) {
if param == nil {
return "", errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "no aggregation param provided for histogram count")
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "no aggregation param provided for histogram count")
}
histogramCountThreshold := param.Threshold
@@ -345,7 +344,7 @@ func AggregationQueryForHistogramCountWithParams(param *metrictypes.ComparisonSp
case ">":
return fmt.Sprintf("argMax(value, toFloat64(le)) - (argMaxIf(value, toFloat64(le), toFloat64(le) <= %f) + (argMinIf(value, toFloat64(le), toFloat64(le) > %f) - argMaxIf(value, toFloat64(le), toFloat64(le) <= %f)) * (%f - maxIf(toFloat64(le), toFloat64(le) <= %f)) / (minIf(toFloat64(le), toFloat64(le) > %f) - maxIf(toFloat64(le), toFloat64(le) <= %f))) AS value", histogramCountThreshold, histogramCountThreshold, histogramCountThreshold, histogramCountThreshold, histogramCountThreshold, histogramCountThreshold, histogramCountThreshold), nil
default:
return "", errors.New(errors.TypeInvalidInput, errors.CodeInvalidInput, "invalid space aggregation operator, should be one of the following: [`<=`, `>`]")
return "", errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid space aggregation operator, should be one of the following: [`<=`, `>`]")
}
}

View File

@@ -368,14 +368,8 @@ func (m *defaultFieldMapper) ColumnExpressionFor(
// - it is not a static field
// - the next best thing to do is see if there is a typo
// and suggest a correction
correction, found := telemetrytypes.SuggestCorrection(field.Name, maps.Keys(keys))
if found {
// we found a close match, in the error message send the suggestion
return "", errors.Wrap(err, errors.TypeInvalidInput, errors.CodeInvalidInput, correction)
} else {
// not even a close match, return an error
return "", errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name)
}
wrappedErr := errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "field `%s` not found", field.Name).WithSuggestions(errors.SuggestionsOnLevenshteinDistance(field.Name, maps.Keys(keys))...)
return "", wrappedErr
}
} else if len(keysForField) == 1 {
// we have a single key for the field, use it

View File

@@ -85,10 +85,19 @@ var SupportedServices = map[CloudProviderType][]ServiceID{
}
func NewServiceID(provider CloudProviderType, service string) (ServiceID, error) {
for _, s := range SupportedServices[provider] {
// The valid set is provider-scoped (AWS and Azure expose different
// services), so surface it as a structured suggestion along with a
// closest-match correction for typos.
supported := SupportedServices[provider]
validServices := make([]string, 0, len(supported))
for _, s := range supported {
if s.StringValue() == service {
return s, nil
}
validServices = append(validServices, s.StringValue())
}
return ServiceID{}, errors.NewInvalidInputf(ErrCodeInvalidServiceID, "invalid service id %q for %s cloud provider", service, provider.StringValue())
return ServiceID{}, errors.NewInvalidInputf(ErrCodeInvalidServiceID,
"invalid service id %q for %s cloud provider", service, provider.StringValue()).
WithSuggestions(errors.SuggestionsOnLevenshteinDistance(service, validServices)...)
}

View File

@@ -1,9 +1,11 @@
package querybuildertypesv5
import (
"bytes"
"fmt"
"slices"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/swaggest/jsonschema-go"
@@ -161,8 +163,8 @@ func (q *QueryBuilderQuery[T]) UnmarshalJSON(data []byte) error {
type Alias QueryBuilderQuery[T]
var temp Alias
// Use UnmarshalJSONWithContext for better error messages
if err := UnmarshalJSONWithContext(data, &temp, fmt.Sprintf("query spec for %T", q)); err != nil {
// Strict-decode the alias so unknown fields surface with field-name suggestions.
if err := binding.JSON.BindBody(bytes.NewReader(data), &temp, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext(fmt.Sprintf("query spec for %T", q))); err != nil {
return err
}

View File

@@ -1,6 +1,7 @@
package querybuildertypesv5
import (
"bytes"
"fmt"
"math"
"strconv"
@@ -12,6 +13,7 @@ import (
"github.com/SigNoz/govaluate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/binding"
)
type QueryBuilderFormula struct {
@@ -66,7 +68,7 @@ func (f QueryBuilderFormula) Copy() QueryBuilderFormula {
func (f *QueryBuilderFormula) UnmarshalJSON(data []byte) error {
type Alias QueryBuilderFormula
var temp Alias
if err := UnmarshalJSONWithContext(data, &temp, "formula spec"); err != nil {
if err := binding.JSON.BindBody(bytes.NewReader(data), &temp, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext("formula spec")); err != nil {
return err
}
*f = QueryBuilderFormula(temp)

View File

@@ -1,163 +0,0 @@
package querybuildertypesv5
import (
"bytes"
"encoding/json"
"reflect"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
// UnmarshalJSONWithSuggestions unmarshals JSON data into the target struct
// and provides field name suggestions for unknown fields.
func UnmarshalJSONWithSuggestions(data []byte, target any) error {
return UnmarshalJSONWithContext(data, target, "")
}
// UnmarshalJSONWithContext unmarshals JSON with context information for better error messages.
func UnmarshalJSONWithContext(data []byte, target any, context string) error {
// First, try to unmarshal with DisallowUnknownFields to catch unknown fields
dec := json.NewDecoder(bytes.NewReader(data))
dec.DisallowUnknownFields()
err := dec.Decode(target)
if err == nil {
// No error, successful unmarshal
return nil
}
// Check if it's an unknown field error
if strings.Contains(err.Error(), "unknown field") {
// Extract the unknown field name
unknownField := extractUnknownField(err.Error())
if unknownField != "" {
// Get valid field names from the target struct
validFields := getJSONFieldNames(target)
// Build error message with context
errorMsg := "unknown field %q"
if context != "" {
errorMsg = "unknown field %q in " + context
}
// Find closest match with max distance of 3 (reasonable for typos)
if suggestion, found := telemetrytypes.SuggestCorrection(unknownField, validFields); found {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
errorMsg,
unknownField,
).WithAdditional(
suggestion,
)
}
// No good suggestion found
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
errorMsg,
unknownField,
).WithAdditional(
"Valid fields are: " + strings.Join(validFields, ", "),
)
}
}
// Return the original error if it's not an unknown field error
return errors.NewInvalidInputf(errors.CodeInvalidInput, "invalid JSON: %v", err)
}
// extractUnknownField extracts the field name from an unknown field error message.
func extractUnknownField(errMsg string) string {
// The error message format is: json: unknown field "fieldname"
parts := strings.Split(errMsg, `"`)
if len(parts) >= 2 {
return parts[1]
}
return ""
}
// getJSONFieldNames extracts all JSON field names from a struct.
func getJSONFieldNames(v any) []string {
var fields []string
t := reflect.TypeOf(v)
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return fields
}
for i := 0; i < t.NumField(); i++ {
field := t.Field(i)
jsonTag := field.Tag.Get("json")
if jsonTag == "" || jsonTag == "-" {
continue
}
// Extract the field name from the JSON tag
fieldName := strings.Split(jsonTag, ",")[0]
if fieldName != "" {
fields = append(fields, fieldName)
}
}
return fields
}
// wrapUnmarshalError wraps UnmarshalJSONWithContext errors with appropriate context
// It preserves errors that already have additional context or unknown field errors.
func wrapUnmarshalError(err error, errorFormat string, args ...interface{}) error {
if err == nil {
return nil
}
// If it's already one of our wrapped errors with additional context, return as-is
_, _, _, _, _, additionals := errors.Unwrapb(err)
if len(additionals) > 0 {
return err
}
// Preserve helpful error messages about unknown fields
if strings.Contains(err.Error(), "unknown field") {
return err
}
// Wrap with the provided error format
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
errorFormat,
args...,
)
}
// wrapValidationError rewraps validation errors with context while preserving additional hints
// It extracts the inner message from the error and creates a new error with the provided format
// The innerMsg is automatically appended to the args for formatting.
func wrapValidationError(err error, contextIdentifier string, errorFormat string) error {
if err == nil {
return nil
}
// Extract the underlying error details
_, _, innerMsg, _, _, additionals := errors.Unwrapb(err)
// Create a new error with the provided format
newErr := errors.NewInvalidInputf(
errors.CodeInvalidInput,
errorFormat,
contextIdentifier,
innerMsg,
)
// Add any additional context from the inner error
if len(additionals) > 0 {
newErr = newErr.WithAdditional(additionals...)
}
return newErr
}

View File

@@ -11,10 +11,10 @@ import (
var (
ErrColumnNotFound = errors.Newf(errors.TypeNotFound, errors.CodeNotFound, "field not found")
ErrBetweenValues = errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "(not) between operator requires two values")
ErrBetweenValuesType = errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "(not) between operator requires two values of the number type")
ErrInValues = errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "(not) in operator requires a list of values")
ErrUnsupportedOperator = errors.Newf(errors.TypeInvalidInput, errors.CodeInvalidInput, "unsupported operator")
ErrBetweenValues = errors.NewInvalidInputf(errors.CodeInvalidInput, "(not) between operator requires two values")
ErrBetweenValuesType = errors.NewInvalidInputf(errors.CodeInvalidInput, "(not) between operator requires two values of the number type")
ErrInValues = errors.NewInvalidInputf(errors.CodeInvalidInput, "(not) in operator requires a list of values")
ErrUnsupportedOperator = errors.NewInvalidInputf(errors.CodeInvalidInput, "unsupported operator")
)
type JsonKeyToFieldFunc func(context.Context, *telemetrytypes.TelemetryFieldKey, FilterOperator, any) (string, any)

View File

@@ -1,11 +1,13 @@
package querybuildertypesv5
import (
"bytes"
"encoding/json"
"strings"
"github.com/SigNoz/govaluate"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
@@ -90,7 +92,7 @@ func (q *QueryEnvelope) UnmarshalJSON(data []byte) error {
Type QueryType `json:"type"`
Spec json.RawMessage `json:"spec"`
}
if err := UnmarshalJSONWithSuggestions(data, &shadow); err != nil {
if err := binding.JSON.BindBody(bytes.NewReader(data), &shadow, binding.WithDisallowUnknownFields(true)); err != nil {
return err
}
@@ -107,40 +109,39 @@ func (q *QueryEnvelope) UnmarshalJSON(data []byte) error {
case QueryTypeFormula:
var spec QueryBuilderFormula
// TODO(srikanthccv): use json.Unmarshal here after implementing custom unmarshaler for QueryBuilderFormula
if err := UnmarshalJSONWithContext(shadow.Spec, &spec, "formula spec"); err != nil {
return wrapUnmarshalError(err, "invalid formula spec: %v", err)
if err := json.Unmarshal(shadow.Spec, &spec); err != nil {
return err
}
q.Spec = spec
case QueryTypeJoin:
var spec QueryBuilderJoin
// TODO(srikanthccv): use json.Unmarshal here after implementing custom unmarshaler for QueryBuilderJoin
if err := UnmarshalJSONWithContext(shadow.Spec, &spec, "join spec"); err != nil {
return wrapUnmarshalError(err, "invalid join spec: %v", err)
if err := binding.JSON.BindBody(bytes.NewReader(shadow.Spec), &spec, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext("join spec")); err != nil {
return err
}
q.Spec = spec
case QueryTypeTraceOperator:
var spec QueryBuilderTraceOperator
if err := json.Unmarshal(shadow.Spec, &spec); err != nil {
return wrapUnmarshalError(err, "invalid trace operator spec: %v", err)
return err
}
q.Spec = spec
case QueryTypePromQL:
var spec PromQuery
// TODO(srikanthccv): use json.Unmarshal here after implementing custom unmarshaler for PromQuery
if err := UnmarshalJSONWithContext(shadow.Spec, &spec, "PromQL spec"); err != nil {
return wrapUnmarshalError(err, "invalid PromQL spec: %v", err)
if err := binding.JSON.BindBody(bytes.NewReader(shadow.Spec), &spec, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext("PromQL spec")); err != nil {
return err
}
q.Spec = spec
case QueryTypeClickHouseSQL:
var spec ClickHouseQuery
// TODO(srikanthccv): use json.Unmarshal here after implementing custom unmarshaler for ClickHouseQuery
if err := UnmarshalJSONWithContext(shadow.Spec, &spec, "ClickHouse SQL spec"); err != nil {
return wrapUnmarshalError(err, "invalid ClickHouse SQL spec: %v", err)
if err := binding.JSON.BindBody(bytes.NewReader(shadow.Spec), &spec, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext("ClickHouse SQL spec")); err != nil {
return err
}
q.Spec = spec
@@ -151,7 +152,7 @@ func (q *QueryEnvelope) UnmarshalJSON(data []byte) error {
shadow.Type,
).WithAdditional(
"Valid query types are: builder_query, builder_sub_query, builder_formula, builder_join, builder_trace_operator, promql, clickhouse_sql",
)
).WithSuggestions(errors.ValidReferences(QueryType{}.Enum()...))
}
return nil
@@ -175,28 +176,27 @@ func UnmarshalBuilderQueryBySignal(data []byte) (any, error) {
case telemetrytypes.SignalTraces:
var spec QueryBuilderQuery[TraceAggregation]
if err := json.Unmarshal(data, &spec); err != nil {
return nil, wrapUnmarshalError(err, "invalid trace builder query spec: %v", err)
return nil, err
}
return spec, nil
case telemetrytypes.SignalLogs:
var spec QueryBuilderQuery[LogAggregation]
if err := json.Unmarshal(data, &spec); err != nil {
return nil, wrapUnmarshalError(err, "invalid log builder query spec: %v", err)
return nil, err
}
return spec, nil
case telemetrytypes.SignalMetrics:
var spec QueryBuilderQuery[MetricAggregation]
if err := json.Unmarshal(data, &spec); err != nil {
return nil, wrapUnmarshalError(err, "invalid metric builder query spec: %v", err)
return nil, err
}
return spec, nil
default:
return nil, errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid signal %q; allowed values: %v",
"invalid signal %q",
header.Signal.StringValue(),
telemetrytypes.Signal{}.Enum(),
)
).WithSuggestions(errors.ValidReferences(telemetrytypes.Signal{}.Enum()...))
}
}
@@ -227,36 +227,24 @@ func (c *CompositeQuery) UnmarshalJSON(data []byte) error {
return err
}
// Check for unknown fields at this level
validFields := map[string]bool{
"queries": true,
// Valid field names are derived from the struct itself so this stays in
// sync with the schema (and the generated OpenAPI spec) automatically.
fieldNames := binding.JSONFieldNames((*CompositeQuery)(nil))
validFields := make(map[string]bool, len(fieldNames))
for _, f := range fieldNames {
validFields[f] = true
}
for field := range check {
if !validFields[field] {
// Find closest match
var fieldNames []string
for f := range validFields {
fieldNames = append(fieldNames, f)
}
if suggestion, found := telemetrytypes.SuggestCorrection(field, fieldNames); found {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"unknown field %q in composite query",
field,
).WithAdditional(
suggestion,
)
}
return errors.NewInvalidInputf(
unknownFieldErr := errors.NewInvalidInputf(
errors.CodeInvalidInput,
"unknown field %q in composite query",
field,
).WithAdditional(
"Valid fields are: " + strings.Join(fieldNames, ", "),
)
).WithSuggestions(errors.SuggestionsOnLevenshteinDistance(field, fieldNames)...)
return unknownFieldErr
}
}
@@ -566,43 +554,24 @@ func (r *QueryRangeRequest) UnmarshalJSON(data []byte) error {
return err
}
// Check for unknown fields at the top level
validFields := map[string]bool{
"schemaVersion": true,
"start": true,
"end": true,
"requestType": true,
"compositeQuery": true,
"variables": true,
"noCache": true,
"formatOptions": true,
// Valid field names are derived from the struct itself so this stays in
// sync with the schema (and the generated OpenAPI spec) automatically.
fieldNames := binding.JSONFieldNames((*QueryRangeRequest)(nil))
validFields := make(map[string]bool, len(fieldNames))
for _, f := range fieldNames {
validFields[f] = true
}
for field := range check {
if !validFields[field] {
// Find closest match
var fieldNames []string
for f := range validFields {
fieldNames = append(fieldNames, f)
}
if suggestion, found := telemetrytypes.SuggestCorrection(field, fieldNames); found {
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"unknown field %q",
field,
).WithAdditional(
suggestion,
)
}
return errors.NewInvalidInputf(
unknownFieldErr := errors.NewInvalidInputf(
errors.CodeInvalidInput,
"unknown field %q",
field,
).WithAdditional(
"Valid fields are: " + strings.Join(fieldNames, ", "),
)
).WithSuggestions(errors.SuggestionsOnLevenshteinDistance(field, fieldNames)...)
return unknownFieldErr
}
}

View File

@@ -16,6 +16,7 @@ func TestQueryRangeRequest_UnmarshalJSON_ErrorMessages(t *testing.T) {
jsonData string
wantErrMsg string
wantAdditionalHints []string
wantSuggestions []string
}{
{
name: "unknown field 'function' in query spec",
@@ -42,10 +43,8 @@ func TestQueryRangeRequest_UnmarshalJSON_ErrorMessages(t *testing.T) {
}]
}
}`,
wantErrMsg: `unknown field "function" in query spec`,
wantAdditionalHints: []string{
"did you mean: 'functions'?",
},
wantErrMsg: `unknown field "function" in query spec`,
wantSuggestions: []string{"did you mean: `functions`"},
},
{
name: "unknown field 'filters' in query spec",
@@ -70,10 +69,8 @@ func TestQueryRangeRequest_UnmarshalJSON_ErrorMessages(t *testing.T) {
}]
}
}`,
wantErrMsg: `unknown field "filters" in query spec`,
wantAdditionalHints: []string{
"did you mean: 'filter'?",
},
wantErrMsg: `unknown field "filters" in query spec`,
wantSuggestions: []string{"did you mean: `filter`"},
},
{
name: "unknown field at top level",
@@ -86,10 +83,9 @@ func TestQueryRangeRequest_UnmarshalJSON_ErrorMessages(t *testing.T) {
"queries": []
}
}`,
wantErrMsg: `unknown field "compositeQueries"`,
wantAdditionalHints: []string{
"did you mean: 'compositeQuery'?",
},
wantErrMsg: `unknown field "compositeQueries"`,
wantAdditionalHints: []string{"Valid fields are:"},
wantSuggestions: []string{"did you mean: `compositeQuery`"},
},
{
name: "unknown field with no good suggestion",
@@ -113,9 +109,6 @@ func TestQueryRangeRequest_UnmarshalJSON_ErrorMessages(t *testing.T) {
}
}`,
wantErrMsg: `unknown field "randomField" in query spec`,
wantAdditionalHints: []string{
"Valid fields are:",
},
},
}
@@ -129,20 +122,28 @@ func TestQueryRangeRequest_UnmarshalJSON_ErrorMessages(t *testing.T) {
// Check main error message
assert.Contains(t, err.Error(), tt.wantErrMsg)
// Check if it's an error from our package using Unwrapb
_, _, _, _, _, additionals := errors.Unwrapb(err)
// Inspect the structured error via its JSON representation.
j := errors.AsJSON(err)
// Check additional hints if we have any
if len(additionals) > 0 {
// Check additional hints (the messages on the errors array) if we have any.
if len(j.Errors) > 0 {
for _, hint := range tt.wantAdditionalHints {
found := false
for _, additional := range additionals {
if strings.Contains(additional, hint) {
for _, e := range j.Errors {
if strings.Contains(e.Message, hint) {
found = true
break
}
}
assert.True(t, found, "Expected to find hint '%s' in additionals: %v", hint, additionals)
assert.True(t, found, "Expected to find hint '%s' in additionals: %v", hint, j.Errors)
}
}
// Typo suggestions are surfaced as structured (machine-consumable)
// suggestions, not in the human-facing additional hints.
if len(tt.wantSuggestions) > 0 {
for _, want := range tt.wantSuggestions {
assert.Contains(t, j.Suggestions, want, "Expected suggestion %q in %v", want, j.Suggestions)
}
}
})

View File

@@ -1,10 +1,12 @@
package querybuildertypesv5
import (
"bytes"
"regexp"
"strings"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/http/binding"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
)
@@ -372,8 +374,8 @@ func (q *QueryBuilderTraceOperator) UnmarshalJSON(data []byte) error {
type Alias QueryBuilderTraceOperator
var temp Alias
// Use UnmarshalJSONWithContext for better error messages
if err := UnmarshalJSONWithContext(data, &temp, "query spec"); err != nil {
// Strict-decode the alias so unknown fields surface with field-name suggestions.
if err := binding.JSON.BindBody(bytes.NewReader(data), &temp, binding.WithDisallowUnknownFields(true), binding.WithUnknownFieldContext("query spec")); err != nil {
return err
}

View File

@@ -38,6 +38,30 @@ func getQueryIdentifier(envelope QueryEnvelope, index int) string {
return fmt.Sprintf("%s at position %d", typeLabel, index+1)
}
// wrapValidationError rewraps a validation failure as errorFormat % (contextIdentifier,
// innerMsg), carrying the inner error's additionals and suggestions onto the new error so
// the structured hints survive the rewrap.
func wrapValidationError(cause error, contextIdentifier string, errorFormat string) error {
if cause == nil {
return nil
}
_, _, innerMsg, _, _, additionals := errors.Unwrapb(cause)
inner := errors.AsJSON(cause)
newErr := errors.NewInvalidInputf(errors.CodeInvalidInput, errorFormat, contextIdentifier, innerMsg)
if len(additionals) > 0 {
newErr = newErr.WithAdditionals(additionals...)
}
if len(inner.Suggestions) > 0 {
newErr = newErr.WithSuggestions(inner.Suggestions...)
}
return newErr
}
const (
// Maximum limit for query results.
MaxQueryLimit = 10000
@@ -484,6 +508,9 @@ func (q *QueryBuilderQuery[T]) validateOrderByForAggregation() error {
}
slices.Sort(validKeys)
// Aggregation order-by keys are a small, exhaustive set (group-by keys,
// aggregation aliases/expressions, indices, __result), so a "valid references"
// list — unlike free-form field suggestions — is genuinely useful here.
return errors.NewInvalidInputf(
errors.CodeInvalidInput,
"invalid order by key '%s' for %s",
@@ -491,7 +518,7 @@ func (q *QueryBuilderQuery[T]) validateOrderByForAggregation() error {
orderId,
).WithAdditional(
fmt.Sprintf("For aggregation queries, order by can only reference group by keys, aggregation aliases/expressions, or aggregation indices. Valid keys are: %s", strings.Join(validKeys, ", ")),
)
).WithSuggestions(errors.SuggestionsOnLevenshteinDistance(orderKey, validKeys)...)
}
}
@@ -685,7 +712,7 @@ func validateQueryEnvelope(envelope QueryEnvelope, opts ...ValidationOption) err
envelope.Type,
).WithAdditional(
"Valid query types are: builder_query, builder_sub_query, builder_formula, builder_join, promql, clickhouse_sql, trace_operator",
)
).WithSuggestions(errors.ValidReferences(QueryType{}.Enum()...))
}
}

View File

@@ -1,111 +0,0 @@
package telemetrytypes
import (
"fmt"
"strings"
)
const (
typoSuggestionThreshold = 0.75
)
// levenshteinDistance calculates the edit distance between two strings.
func levenshteinDistance(s1, s2 string) int {
s1 = strings.ToLower(s1)
s2 = strings.ToLower(s2)
if len(s1) == 0 {
return len(s2)
}
if len(s2) == 0 {
return len(s1)
}
// Create two work vectors of integer distances
v0 := make([]int, len(s2)+1)
v1 := make([]int, len(s2)+1)
// Initialize v0 (the previous row of distances)
for i := 0; i <= len(s2); i++ {
v0[i] = i
}
// Calculate each row in the matrix
for i := range len(s1) {
v1[0] = i + 1
for j := range len(s2) {
deletionCost := v0[j+1] + 1
insertionCost := v1[j] + 1
var substitutionCost int
if s1[i] == s2[j] {
substitutionCost = v0[j]
} else {
substitutionCost = v0[j] + 1
}
v1[j+1] = min(deletionCost, insertionCost, substitutionCost)
}
// Copy v1 to v0 for next iteration
for j := 0; j <= len(s2); j++ {
v0[j] = v1[j]
}
}
return v1[len(s2)]
}
// similarity returns a value between 0 and 1, where 1 means perfect match.
func similarity(s1, s2 string) float64 {
maxLen := max(len(s1), len(s2))
if maxLen == 0 {
return 1.0 // Both strings are empty
}
distance := levenshteinDistance(s1, s2)
return 1.0 - float64(distance)/float64(maxLen)
}
func min(a, b, c int) int {
if a < b {
if a < c {
return a
}
return c
}
if b < c {
return b
}
return c
}
func max(a, b int) int {
if a > b {
return a
}
return b
}
// SuggestCorrection checks if there are any column names similar to the input
// and returns a suggestion if there's at least 75% similarity.
func SuggestCorrection(input string, knownFieldKeys []string) (string, bool) {
var bestMatch string
bestSimilarity := 0.0
for _, columnName := range knownFieldKeys {
sim := similarity(input, columnName)
if sim > bestSimilarity && sim >= typoSuggestionThreshold {
bestSimilarity = sim
bestMatch = columnName
}
}
if bestSimilarity >= typoSuggestionThreshold {
return fmt.Sprintf("did you mean: '%s'?", bestMatch), true
}
return "", false
}