Compare commits

..

2 Commits

Author SHA1 Message Date
Naman Verma
63514beed0 Merge branch 'main' into fga/dashboards 2026-08-10 16:47:09 +05:30
Naman Verma
67d158056f chore: add dashboard and public dashboard as authz resources 2026-08-06 13:33:50 +05:30
28 changed files with 138 additions and 719 deletions

View File

@@ -96,6 +96,8 @@ func runGenerateAuthz(_ context.Context) error {
coretypes.NewResourceRef(coretypes.ResourceServiceAccount).String(): true,
coretypes.NewResourceRef(coretypes.ResourceRole).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourceFactorAPIKey).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourceDashboard).String(): true,
coretypes.NewResourceRef(coretypes.ResourceMetaResourcePublicDashboard).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceLogs).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceTraces).String(): true,
coretypes.NewResourceRef(coretypes.ResourceTelemetryResourceMetrics).String(): true,

View File

@@ -349,7 +349,7 @@ func (Step) JSONSchema() (jsonschema.Schema, error) {
### `oneOf` with a discriminator
For a sum type whose variants are keyed by a property (e.g. `kind`), expose the variants via `JSONSchemaOneOf()` and add a discriminator. Without it, code generators intersect the variants (`A & B & C`) instead of producing a clean discriminated union (`A | B | C`). How to model the sum type itself is covered in [types.md](types.md#sum-types-the-kindspec-envelope) — this section is only about its schema.
For a sum type whose variants are keyed by a property (e.g. `kind`), expose the variants via `JSONSchemaOneOf()` and add a discriminator. Without it, code generators intersect the variants (`A & B & C`) instead of producing a clean discriminated union (`A | B | C`).
The parent keeps its `JSONSchemaOneOf()` (the `oneOf` itself) and *additionally* tags it via `PrepareJSONSchema` with the `x-signoz-discriminator` extension; `signoz.attachDiscriminators` then promotes that marker to a real OpenAPI 3 `discriminator` (and strips the duplicate parent properties) after reflection.

View File

@@ -99,69 +99,6 @@ Each flavor exists for a concrete reason:
The core `AuthDomain` holds the two live halves — `storableAuthDomain` and `authDomainConfig` — and owns business methods such as `Update(config)`. Conversions use the `New<Output>From<Input>` form: `NewAuthDomainFromConfig`, `NewAuthDomainFromStorableAuthDomain`, `NewGettableAuthDomainFromAuthDomain`.
## Sum types: the kind/spec envelope
When a domain type is a *sum type* — exactly one of several variants, selected by a discriminator — model it as an envelope with a `kind` and a `spec`:
```go
type FooConfig struct {
Kind FooKind `json:"kind" required:"true"`
Spec any `json:"spec" required:"true"`
}
```
```json
{ "kind": "bar", "spec": { "url": "...", "timeout": "30s" } }
```
`Kind` is a `valuer.String` enum implementing `Enum()`; `Spec` holds exactly one concrete variant type (`BarSpec`, `BazSpec`, …). `RuleThresholdData` and `EvaluationEnvelope` in `pkg/types/ruletypes/` are the canonical in-tree examples; the dashboard panel/query/variable plugins in `pkg/types/dashboardtypes/` are the same pattern behind generics. (`QueryEnvelope` in querybuildertypes uses `type` as the discriminator key for historical reasons; new envelopes use `kind`.)
### The envelope goes at the point of variance, not the resource root
Put the envelope on the field that actually varies. The resource root is almost never a sum type — a `Foo` has a `name` and an `enabled` flag regardless of which kind it is configured with; only its configuration varies, so the envelope is the `config` field:
```json
{ "name": "my-foo", "enabled": true, "config": { "kind": "bar", "spec": { "...": "..." } } }
```
Hoisting `kind`/`spec` to the root would turn the whole resource into a `oneOf`: every flavor (`PostableFoo`, `UpdatableFoo`, `GettableFoo`) then needs one variant schema per kind, each repeating the common fields; every new common field has to be added to all of them; and generated clients get unions of large objects instead of one small union that narrows on `config.kind`. A root-level `kind` also collides with the resource-model meaning of the word — root `kind` conventionally answers "what resource is this" (`Dashboard`), never "which flavor of config does it hold".
The existing domains already follow this placement:
- **Rules** — plain root; envelopes on the varying fields: `thresholds: {kind, spec}` and `evaluation: {kind, spec}`.
- **Dashboards** — metadata at the root plus one typed `spec`; the unions sit deep inside, at each panel/query/variable plugin (`{kind, spec}` in `perses_plugin_wrappers.go`).
- **Saved views** — root `{schemaVersion, spec}`, where `spec` is a *versioning* envelope holding one fixed type, not a union; the unions are inside it (`spec.queries: [{type, spec}]`). Same word, different job — a versioned body is not a discriminated union.
### Why this tagging style
Of the union encodings in common use, the envelope is the *adjacently tagged* one — tag and payload side by side. Variant payloads stay collision-free, and each kind maps to a named wrapper schema that carries the discriminator, which is exactly what OpenAPI generators need. The alternatives lose on those points: *internally tagged* (`{"kind": "bar", ...fields flattened}`) mixes common and variant fields, admits cross-variant key collisions, and forces every variant schema to redeclare the discriminator; *sibling optional fields* (`{"kind": "bar", "barConfig": {}, "bazConfig": {}}`) is the anti-pattern the first rule below exists to prevent.
The rules that make the envelope work:
- **Never model variants as sibling fields.** A struct with `Bar *BarSpec`, `Baz *BazSpec` next to a discriminator cannot be expressed as an OpenAPI discriminated union, forces nilability checks on every consumer, and silently admits contradictory payloads (kind=bar with a baz spec). The chosen variant *is* the payload.
- **The envelope owns `UnmarshalJSON`.** Decode `kind` first, then switch on it to decode and validate the matching concrete type into `Spec`. Unknown kinds and missing specs are rejected at the boundary:
```go
func (typ *FooConfig) UnmarshalJSON(data []byte) error {
var raw map[string]json.RawMessage
// ... unmarshal raw, decode raw["kind"] ...
switch kind {
case FooKindBar:
spec := BarSpec{}
if err := json.Unmarshal(raw["spec"], &spec); err != nil {
return err
}
typ.Spec = spec
// ... one case per kind, default rejects ...
}
typ.Kind = kind
return nil
}
```
- **Consumers type-assert on `Spec`** (`config.Spec.(BarSpec)`) after switching on `Kind`. If assertion sites multiply, add typed accessors on the envelope (see `EvaluationEnvelope.GetEvaluation()`).
- **OpenAPI needs one unexported variant struct per kind** (`fooConfigBar{Kind; Spec BarSpec}`), exposed via `JSONSchemaOneOf()` and mapped via `PrepareJSONSchema` with the `x-signoz-discriminator` extension. The schema mechanics are covered in [handler.md](handler.md#oneof-with-a-discriminator).
- **A legacy persisted shape gets a data migration or a `StorableX`.** When rows were written before the envelope existed, prefer an idempotent `sqlmigration` that rewrites them into the new shape, so the storable type simply nests the envelope. Only when the old shape must keep being written (external writers, rollback windows) keep it in a storable twin and convert at the type boundary.
## Conventions that tie the flavors together
- **Conversions** use either a `New<Output>From<Input>` constructor — e.g. `NewChannelFromReceiver`, `NewGettableAuthDomainFromAuthDomain` — or a receiver-style `ToY()` method. Both forms coexist in the codebase; use whichever fits the call site.
@@ -202,8 +139,6 @@ Both are optional. Do not introduce them if `PostableX` already covers the case.
- Every domain package defines the core type `X`. Only `X` is mandatory.
- Add `PostableX` / `GettableX` / `UpdatableX` / `StorableX` one at a time, only when the shape actually diverges from `X`.
- Model sum types as a `{kind, spec}` envelope with a validating `UnmarshalJSON` — never as sibling variant fields next to a discriminator.
- The envelope goes on the field that varies, never at the resource root — common fields stay on the resource, outside the union.
- Domain logic lives on `X`, not on the flavor types.
- Conversions can be a `New<Output>From<Input>` constructor or a receiver-style `ToY()` method — pick whichever reads best at the call site.
- Use a type alias when two shapes are truly identical.

View File

@@ -376,19 +376,7 @@ function App(): JSX.Element {
tracesSampleRate: 0, // Ref: https://github.com/SigNoz/platform-pod/issues/2393#issuecomment-4603658055
replaysSessionSampleRate: 0.1, // This sets the sample rate at 10%. You may want to change it to 100% while in development and then sample at a lower rate in production.
replaysOnErrorSampleRate: 1.0, // If you're not already sampling the entire session, change the sample rate to 100% when sampling sessions where errors occur.
beforeSend(event, hint) {
const error = hint?.originalException as
| { name?: string; code?: string | number }
| undefined;
// Ignore benign aborted/cancelled requests (axios + fetch).
if (error?.code === 'ERR_CANCELED' || error?.code === 'ECONNABORTED') {
return null;
}
if (error?.name === 'AbortError') {
return null;
}
beforeSend(event) {
// Drop the event if its level is 'warning' or 'info'
if (event.level === 'warning' || event.level === 'info') {
return null;

View File

@@ -0,0 +1,31 @@
import axios from 'api';
import { ErrorResponseHandlerV2 } from 'api/ErrorResponseHandlerV2';
import { AxiosError } from 'axios';
import { ErrorV2Resp, SuccessResponseV2 } from 'types/api';
import { PayloadProps, Props } from 'types/api/user/resetPassword';
/**
* @deprecated Use the generated `useResetPassword` hook (or `resetPassword` fetcher) from
* `api/generated/services/users` instead. This hand-written client targets the
* same endpoint and will be removed once call sites migrate.
*
* Part of https://github.com/SigNoz/engineering-pod/issues/5289, add a comment or update when removing this method.
*/
const resetPassword = async (
props: Props,
): Promise<SuccessResponseV2<PayloadProps>> => {
try {
const response = await axios.post<PayloadProps>(`/resetPassword`, {
...props,
});
return {
httpStatusCode: response.status,
data: response.data,
};
} catch (error) {
ErrorResponseHandlerV2(error as AxiosError<ErrorV2Resp>);
}
};
export default resetPassword;

View File

@@ -1,46 +0,0 @@
.highlights {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 12px 16px;
padding: 12px 0;
// Constrain each KeyValueLabel (the grid items) to its cell.
:global(.key-value-label) {
width: auto;
min-width: 0;
overflow: hidden;
}
}
.valueBadge {
--badge-font-size: 13px;
box-sizing: border-box;
max-width: 100%;
min-width: 0;
}
// Truncating text inside a badge
.badgeText {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.serviceDot {
width: 6px;
height: 6px;
border-radius: 50%;
background: var(--accent-forest);
flex-shrink: 0;
margin-right: 4px;
}
.traceLink {
display: inline-block;
max-width: 100%;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--accent-primary);
}

View File

@@ -1,36 +0,0 @@
import KeyValueLabel from 'periscope/components/KeyValueLabel';
import { ILog } from 'types/api/logs/log';
import { LOG_HIGHLIGHTS } from './config';
import styles from './LogHighlights.module.scss';
interface LogHighlightsProps {
log: ILog;
}
function LogHighlights({ log }: LogHighlightsProps): JSX.Element | null {
const fields = LOG_HIGHLIGHTS.map((field) => ({
key: field.key,
label: field.label,
value: field.render(log),
})).filter((field) => field.value != null);
if (fields.length === 0) {
return null;
}
return (
<div className={styles.highlights} data-testid="log-details-highlights">
{fields.map((field) => (
<KeyValueLabel
key={field.key}
badgeKey={field.label}
badgeValue={field.value}
direction="column"
/>
))}
</div>
);
}
export default LogHighlights;

View File

@@ -1,23 +0,0 @@
import { Link } from 'react-router-dom';
import styles from './LogHighlights.module.scss';
interface TraceIdFieldProps {
traceId: string;
}
function TraceIdField({ traceId }: TraceIdFieldProps): JSX.Element {
return (
<Link
to={{ pathname: `/trace/${traceId}` }}
target="_blank"
rel="noreferrer"
className={styles.traceLink}
title={traceId}
>
{traceId}
</Link>
);
}
export default TraceIdField;

View File

@@ -1,102 +0,0 @@
import { ReactNode } from 'react';
import { Badge, BadgeColor } from '@signozhq/ui/badge';
import { LogType } from 'components/Logs/LogStateIndicator/LogStateIndicator';
import { getLogIndicatorType } from 'components/Logs/LogStateIndicator/utils';
import { ILog } from 'types/api/logs/log';
import styles from './LogHighlights.module.scss';
import TraceIdField from './TraceIdField';
// Severity badge color mirrors the LogStateIndicator bar
const SEVERITY_COLOR: Record<string, BadgeColor> = {
[LogType.TRACE]: 'forest',
[LogType.DEBUG]: 'aqua',
[LogType.INFO]: 'robin',
[LogType.WARN]: 'amber',
[LogType.ERROR]: 'cherry',
[LogType.FATAL]: 'sakura',
};
export interface LogHighlightConfig {
key: string;
label: string;
render: (log: ILog) => ReactNode | null;
}
// Resource/attribute lookup (keys like `service.name` live in resources_string,
// occasionally attributes_string). Typed loosely as these are string maps.
const getAttr = (log: ILog, key: string): string =>
(log.resources_string as unknown as Record<string, string>)?.[key] ||
(log.attributes_string as unknown as Record<string, string>)?.[key] ||
'';
const valueBadge = (
value: string,
options?: { prefix?: ReactNode; color?: BadgeColor },
): ReactNode => (
<Badge color={options?.color ?? 'vanilla'} className={styles.valueBadge}>
{options?.prefix}
<span className={styles.badgeText} title={value}>
{value}
</span>
</Badge>
);
export const LOG_HIGHLIGHTS: LogHighlightConfig[] = [
{
key: 'service',
label: 'SERVICE',
render: (log): ReactNode | null => {
const value = getAttr(log, 'service.name');
return value
? valueBadge(value, {
prefix: <span className={styles.serviceDot} />,
})
: null;
},
},
{
key: 'severity',
label: 'SEVERITY',
render: (log): ReactNode | null => {
if (!log.severity_text) {
return null;
}
return valueBadge(log.severity_text, {
color: SEVERITY_COLOR[getLogIndicatorType(log)] ?? 'vanilla',
});
},
},
{
key: 'namespace',
label: 'NAMESPACE',
render: (log): ReactNode | null => {
const value = getAttr(log, 'service.namespace');
return value ? valueBadge(value) : null;
},
},
{
key: 'environment',
label: 'ENVIRONMENT',
render: (log): ReactNode | null => {
const value = getAttr(log, 'deployment.environment');
return value ? valueBadge(value) : null;
},
},
{
key: 'traceId',
label: 'TRACE ID',
render: (log): ReactNode | null => {
const traceId = log.trace_id || log.traceId;
return traceId ? <TraceIdField traceId={traceId} /> : null;
},
},
{
key: 'spanId',
label: 'SPAN ID',
render: (log): ReactNode | null => {
const spanId = log.span_id || log.spanID;
return spanId ? valueBadge(spanId) : null;
},
},
];

View File

@@ -115,45 +115,6 @@ describe('LogDetail drawer — header (isLogDetailsV2)', () => {
expect(screen.queryByText('Open in Explorer')).not.toBeInTheDocument();
});
it('renders Highlights for fields present on the log, omitting absent ones', () => {
const logWithMeta = {
...mockLog,
severity_text: 'ERROR',
trace_id: 'trace-abc',
resources_string: {
'service.name': 'checkout',
'deployment.environment': 'production',
},
} as unknown as ILog;
renderDrawer({ log: logWithMeta });
const highlights = screen.getByTestId('log-details-highlights');
expect(highlights).toHaveTextContent('SEVERITY');
expect(highlights).toHaveTextContent('ERROR');
expect(highlights).toHaveTextContent('SERVICE');
expect(highlights).toHaveTextContent('checkout');
expect(highlights).toHaveTextContent('ENVIRONMENT');
expect(highlights).toHaveTextContent('production');
expect(highlights).toHaveTextContent('TRACE ID');
// Absent fields are omitted (no namespace / span id on this log).
expect(highlights).not.toHaveTextContent('NAMESPACE');
expect(highlights).not.toHaveTextContent('SPAN ID');
});
it('links the trace id highlight to the trace detail in a new tab', () => {
const logWithTrace = {
...mockLog,
trace_id: 'trace-abc',
} as unknown as ILog;
renderDrawer({ log: logWithTrace });
const link = screen.getByRole('link', { name: 'trace-abc' });
expect(link).toHaveAttribute('target', '_blank');
expect(link.getAttribute('href')).toContain('/trace/trace-abc');
});
it('navigates to the next / previous log with the Down / Up arrow keys', async () => {
const user = userEvent.setup({ pointerEventsCheck: 0 });
const logs = [makeLog('log-0'), makeLog('log-1'), makeLog('log-2')];

View File

@@ -55,7 +55,6 @@ import { isLogDetailsV2, RESOURCE_KEYS, VIEW_TYPES, VIEWS } from './constants';
import { LogDetailInnerProps, LogDetailProps } from './LogDetail.interfaces';
import LogDetailsHeader from './LogDetailsHeader/LogDetailsHeader';
import { useLogNavigation } from './LogDetailsHeader/useLogNavigation';
import LogHighlights from './LogHighlights/LogHighlights';
import './LogDetails.styles.scss';
@@ -400,8 +399,6 @@ function LogDetailInner({
<div className="log-overflow-shadow">&nbsp;</div>
</div>
{isLogDetailsV2 && <LogHighlights log={log} />}
<div className="tabs-and-search">
<ToggleGroupSimple
type="single"

View File

@@ -183,14 +183,15 @@ function QuerySearch({
isProgrammaticChangeRef.current = true;
}
const changes = view.state.changes({
from: 0,
to: currentValue.length,
insert: value,
});
view.dispatch({
changes,
selection: { anchor: changes.newLength },
changes: {
from: 0,
to: currentValue.length,
insert: value,
},
selection: {
anchor: value.length,
},
});
},
[],

View File

@@ -301,66 +301,6 @@ describe('QuerySearch (Integration with Real CodeMirror)', () => {
dispatchSpy.mockRestore();
});
it('does not crash when the expression contains CRLF line breaks (issue #5869)', async () => {
const dispatchSpy = jest.spyOn(EditorView.prototype, 'dispatch');
const onChange = jest.fn() as jest.MockedFunction<(v: string) => void>;
const initialExpression = "service.name = 'frontend'";
// Filtering on a multi-line log value (CRLF) used to throw
// "RangeError: Selection points outside of document".
const crlfExpression = "body CONTAINS 'line1\r\nline2\r\nline3'";
const baseQueryData = {
...initialQueriesMap.logs.builder.queryData[0],
filter: { expression: initialExpression },
};
const { rerender } = render(
<QuerySearch
onChange={onChange}
queryData={baseQueryData}
dataSource={DataSource.LOGS}
/>,
);
await waitFor(
() => {
const editorContent = document.querySelector(
CM_EDITOR_SELECTOR,
) as HTMLElement;
expect(editorContent.textContent || '').toBe(initialExpression);
},
{ timeout: 3000 },
);
rerender(
<QuerySearch
onChange={onChange}
queryData={{ ...baseQueryData, filter: { expression: crlfExpression } }}
dataSource={DataSource.LOGS}
/>,
);
// The programmatic replace dispatched without throwing, and the selection anchor
// stayed within the CRLF-normalized document (the bug set it past the end).
await waitFor(() => {
const spec = dispatchSpy.mock.calls
.map(
(call) =>
call[0] as {
selection?: { anchor?: number };
changes?: { newLength?: number };
},
)
.find((s) => s?.selection?.anchor != null && s?.changes?.newLength != null);
expect(spec).toBeDefined();
expect(spec?.selection?.anchor).toBeLessThanOrEqual(
spec?.changes?.newLength as number,
);
});
dispatchSpy.mockRestore();
});
it('fetches key suggestions for metrics even without aggregateAttribute.key when showFilterSuggestionsWithoutMetric is true', async () => {
const mockedGetKeys = getKeySuggestions as jest.MockedFunction<
typeof getKeySuggestions

View File

@@ -33,7 +33,7 @@ jest.mock('hooks/useNotifications', () => ({
}),
}));
const RESET_PASSWORD_ENDPOINT = '*/api/v2/factor_password/reset';
const RESET_PASSWORD_ENDPOINT = '*/resetPassword';
const mockHistoryPush = history.push as jest.MockedFunction<
typeof history.push

View File

@@ -1,12 +1,11 @@
import { useMemo, useState } from 'react';
import { useState } from 'react';
import { useTranslation } from 'react-i18next';
import { useLocation } from 'react-use';
import { Button } from '@signozhq/ui/button';
import { Callout } from '@signozhq/ui/callout';
import { Form, Input as AntdInput } from 'antd';
import { Typography } from '@signozhq/ui/typography';
import { convertToApiError } from 'api/ErrorResponseHandlerForGeneratedAPIs';
import { useResetPassword } from 'api/generated/services/users';
import resetPasswordApi from 'api/v1/factor_password/resetPassword';
import AuthError from 'components/AuthError/AuthError';
import AuthPageContainer from 'components/AuthPageContainer';
import ROUTES from 'constants/routes';
@@ -15,6 +14,7 @@ import { useNotifications } from 'hooks/useNotifications';
import history from 'lib/history';
import { ArrowRight, CircleAlert, KeyRound } from '@signozhq/icons';
import { Label } from 'pages/SignUp/styles';
import APIError from 'types/api/error';
import { FormContainer } from './styles';
@@ -26,41 +26,40 @@ function ResetPassword({ version }: ResetPasswordProps): JSX.Element {
const [confirmPasswordError, setConfirmPasswordError] =
useState<boolean>(false);
const [errorMessage, setErrorMessage] = useState<APIError | null>();
const [isValidPassword, setIsValidPassword] = useState(false);
const [loading, setLoading] = useState(false);
const { t } = useTranslation(['common']);
const { search } = useLocation();
const params = new URLSearchParams(search);
const token = params.get('token');
const { notifications } = useNotifications();
const {
mutate: resetPassword,
isLoading,
error: mutationError,
} = useResetPassword();
const errorMessage = useMemo(
() => convertToApiError(mutationError),
[mutationError],
);
const [form] = Form.useForm<FormValues>();
const handleFormSubmit = (): void => {
const { password } = form.getFieldsValue();
const handleFormSubmit: () => Promise<void> = async () => {
try {
setLoading(true);
setErrorMessage(null);
const { password } = form.getFieldsValue();
resetPassword(
{ data: { password, token: token || '' } },
{
onSuccess: (): void => {
notifications.success({
message: t('success', {
ns: 'common',
}),
});
history.push(ROUTES.LOGIN);
},
},
);
await resetPasswordApi({
password,
token: token || '',
});
notifications.success({
message: t('success', {
ns: 'common',
}),
});
history.push(ROUTES.LOGIN);
setLoading(false);
} catch (error) {
setLoading(false);
setErrorMessage(error as APIError);
}
};
const validatePassword = (): boolean => {
@@ -223,7 +222,7 @@ function ResetPassword({ version }: ResetPasswordProps): JSX.Element {
color="primary"
type="submit"
data-attr="reset-password"
disabled={!isValidPassword || isLoading}
disabled={!isValidPassword || loading}
className="reset-password-submit-button"
suffix={<ArrowRight size={16} />}
>

View File

@@ -3,11 +3,29 @@ export default {
status: 'success',
data: {
resources: [
{
kind: 'dashboard',
type: 'metaresource',
allowedVerbs: ['create', 'delete', 'list', 'read', 'update'],
},
{
kind: 'factor-api-key',
type: 'metaresource',
allowedVerbs: ['create', 'delete', 'list', 'read', 'update'],
},
{
kind: 'public-dashboard',
type: 'metaresource',
allowedVerbs: [
'attach',
'create',
'delete',
'detach',
'list',
'read',
'update',
],
},
{
kind: 'role',
type: 'role',

View File

@@ -0,0 +1,9 @@
export interface Props {
token: string;
password: string;
}
export interface PayloadProps {
data: string;
status: string;
}

View File

@@ -469,8 +469,6 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
expectedErr: nil,
},
{
// The `[*]` path is extracted per value, not as an Array(String) compared to a
// scalar — ClickHouse rejects that outright (code 130).
name: "IN operator with json search",
requestType: qbtypes.RequestTypeRaw,
query: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
@@ -481,7 +479,7 @@ func TestStatementBuilderListQueryResourceTests(t *testing.T) {
Limit: 10,
},
expected: qbtypes.Statement{
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE ((JSON_VALUE(body, '$.\"user_names\"[*]') = ?) AND JSON_EXISTS(body, '$.\"user_names\"[*]')) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Query: "SELECT timestamp, id, trace_id, span_id, trace_flags, severity_text, severity_number, scope_name, scope_version, body, attributes_string, attributes_number, attributes_bool, resources_string, scope_string FROM signoz_logs.distributed_logs_v2 WHERE ((JSONExtract(JSON_QUERY(body, '$.\"user_names\"[*]'), 'Array(String)') = ?) AND JSON_EXISTS(body, '$.\"user_names\"[*]')) AND timestamp >= ? AND ts_bucket_start >= ? AND timestamp < ? AND ts_bucket_start <= ? LIMIT ?",
Args: []any{"john_doe", "1747947419000000000", uint64(1747945619), "1747983448000000000", uint64(1747983448), 10},
Warnings: []string{querybuilder.NewKeyNotFoundWarning("user_names[*]")},
},

View File

@@ -180,16 +180,20 @@ func (b *defaultConditionBuilder) conditionForKey(
if !ok {
return "", qbtypes.ErrInValues
}
// each value carries its own index filter, since `=` derives one from the value
inConditions := make([]string, 0, len(values))
for _, v := range values {
cond, err := b.conditionForKey(ctx, startNs, endNs, key, qbtypes.FilterOperatorEqual, v, sb)
if err != nil {
return "", err
}
inConditions = append(inConditions, cond)
inConditions = append(inConditions, sb.E(fieldName, querybuilder.FormatValueForContains(v)))
}
return sb.Or(inConditions...), nil
mainCondition := sb.Or(inConditions...)
valConditions := make([]string, 0, len(values))
if valuesForIndexFilter, ok := valueForIndexFilter.([]string); ok {
for _, v := range valuesForIndexFilter {
valConditions = append(valConditions, sb.Like(column.Name, v))
}
}
mainCondition = sb.And(mainCondition, keyIdxFilter, sb.Or(valConditions...))
return mainCondition, nil
case qbtypes.FilterOperatorNotIn:
values, ok := value.([]any)
if !ok {
@@ -197,13 +201,17 @@ func (b *defaultConditionBuilder) conditionForKey(
}
notInConditions := make([]string, 0, len(values))
for _, v := range values {
cond, err := b.conditionForKey(ctx, startNs, endNs, key, qbtypes.FilterOperatorNotEqual, v, sb)
if err != nil {
return "", err
}
notInConditions = append(notInConditions, cond)
notInConditions = append(notInConditions, sb.NE(fieldName, querybuilder.FormatValueForContains(v)))
}
return sb.And(notInConditions...), nil
mainCondition := sb.And(notInConditions...)
valConditions := make([]string, 0, len(values))
if valuesForIndexFilter, ok := valueForIndexFilter.([]string); ok {
for _, v := range valuesForIndexFilter {
valConditions = append(valConditions, sb.NotLike(column.Name, v))
}
}
mainCondition = sb.And(mainCondition, sb.And(valConditions...))
return mainCondition, nil
case qbtypes.FilterOperatorExists:
return sb.And(

View File

@@ -109,8 +109,8 @@ func TestConditionBuilder(t *testing.T) {
},
op: qbtypes.FilterOperatorIn,
value: []any{"watch", "redis"},
expected: "((simpleJSONExtractString(labels, 'k8s.namespace.name') = ? AND labels LIKE ? AND labels LIKE ?) OR (simpleJSONExtractString(labels, 'k8s.namespace.name') = ? AND labels LIKE ? AND labels LIKE ?))",
expectedArgs: []any{"watch", "%k8s.namespace.name%", "%k8s.namespace.name\":\"watch%", "redis", "%k8s.namespace.name%", "%k8s.namespace.name\":\"redis%"},
expected: "(simpleJSONExtractString(labels, 'k8s.namespace.name') = ? OR simpleJSONExtractString(labels, 'k8s.namespace.name') = ?) AND labels LIKE ? AND (labels LIKE ? OR labels LIKE ?)",
expectedArgs: []any{"watch", "redis", "%k8s.namespace.name%", "%k8s.namespace.name\":\"watch%", "%k8s.namespace.name\":\"redis%"},
},
{
name: "string_not_in",
@@ -120,8 +120,8 @@ func TestConditionBuilder(t *testing.T) {
},
op: qbtypes.FilterOperatorNotIn,
value: []any{"watch", "redis"},
expected: "((simpleJSONExtractString(labels, 'k8s.namespace.name') <> ? AND labels NOT LIKE ?) AND (simpleJSONExtractString(labels, 'k8s.namespace.name') <> ? AND labels NOT LIKE ?))",
expectedArgs: []any{"watch", "%k8s.namespace.name\":\"watch%", "redis", "%k8s.namespace.name\":\"redis%"},
expected: "(simpleJSONExtractString(labels, 'k8s.namespace.name') <> ? AND simpleJSONExtractString(labels, 'k8s.namespace.name') <> ?) AND (labels NOT LIKE ? AND labels NOT LIKE ?)",
expectedArgs: []any{"watch", "redis", "%k8s.namespace.name\":\"watch%", "%k8s.namespace.name\":\"redis%"},
},
{
name: "string_exists",
@@ -173,8 +173,8 @@ func TestConditionBuilder(t *testing.T) {
},
op: qbtypes.FilterOperatorIn,
value: []any{1, 2},
expected: "((simpleJSONExtractString(labels, 'test_num') = ? AND labels LIKE ? AND labels LIKE ?) OR (simpleJSONExtractString(labels, 'test_num') = ? AND labels LIKE ? AND labels LIKE ?))",
expectedArgs: []any{"1", "%test_num%", "%test_num\":\"1%", "2", "%test_num%", "%test_num\":\"2%"},
expected: "(simpleJSONExtractString(labels, 'test_num') = ? OR simpleJSONExtractString(labels, 'test_num') = ?) AND labels LIKE ? AND (labels LIKE ? OR labels LIKE ?)",
expectedArgs: []any{"1", "2", "%test_num%", "%test_num\":\"1%", "%test_num\":\"2%"},
},
{
name: "number_between",

View File

@@ -229,8 +229,8 @@ func TestResourceFilterStatementBuilder_Traces(t *testing.T) {
start: testStartNs,
end: testEndNs,
expected: &qbtypes.Statement{
Query: "SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE ((simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?) OR (simpleJSONExtractString(labels, 'service.name') = ? AND labels LIKE ? AND labels LIKE ?)) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint",
Args: []any{"redis", "%service.name%", "%service.name\":\"redis%", "postgres", "%service.name%", "%service.name\":\"postgres%", expectedBucketStart, expectedBucketEnd},
Query: "SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE ((simpleJSONExtractString(labels, 'service.name') = ? OR simpleJSONExtractString(labels, 'service.name') = ?) AND labels LIKE ? AND (labels LIKE ? OR labels LIKE ?)) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint",
Args: []any{"redis", "postgres", "%service.name%", "%service.name\":\"redis%", "%service.name\":\"postgres%", expectedBucketStart, expectedBucketEnd},
},
},
{
@@ -244,8 +244,8 @@ func TestResourceFilterStatementBuilder_Traces(t *testing.T) {
start: testStartNs,
end: testEndNs,
expected: &qbtypes.Statement{
Query: "SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE ((simpleJSONExtractString(labels, 'service.name') <> ? AND labels NOT LIKE ?) AND (simpleJSONExtractString(labels, 'service.name') <> ? AND labels NOT LIKE ?)) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint",
Args: []any{"redis", "%service.name\":\"redis%", "postgres", "%service.name\":\"postgres%", expectedBucketStart, expectedBucketEnd},
Query: "SELECT fingerprint FROM signoz_traces.distributed_traces_v3_resource WHERE ((simpleJSONExtractString(labels, 'service.name') <> ? AND simpleJSONExtractString(labels, 'service.name') <> ?) AND (labels NOT LIKE ? AND labels NOT LIKE ?)) AND seen_at_ts_bucket_start >= ? AND seen_at_ts_bucket_start <= ? GROUP BY fingerprint",
Args: []any{"redis", "postgres", "%service.name\":\"redis%", "%service.name\":\"postgres%", expectedBucketStart, expectedBucketEnd},
},
},
{

View File

@@ -94,11 +94,7 @@ func (c *conditionBuilder) conditionFor(
}
conditions := []string{}
for _, value := range values {
cond, err := c.conditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorEqual, value, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
conditions = append(conditions, sb.E(fieldExpression, value))
}
return sb.Or(conditions...), nil
case qbtypes.FilterOperatorNotIn:
@@ -108,11 +104,7 @@ func (c *conditionBuilder) conditionFor(
}
conditions := []string{}
for _, value := range values {
cond, err := c.conditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorNotEqual, value, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
conditions = append(conditions, sb.NE(fieldExpression, value))
}
return sb.And(conditions...), nil
case qbtypes.FilterOperatorExists, qbtypes.FilterOperatorNotExists:

View File

@@ -410,11 +410,7 @@ func (c *conditionBuilder) conditionForResolvedKey(
// instead of using IN, we use `=` + `OR` to make use of index
conditions := []string{}
for _, value := range values {
cond, err := c.conditionForResolvedKey(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorEqual, value, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
conditions = append(conditions, sb.E(fieldExpression, value))
}
return sb.Or(conditions...), nil
case qbtypes.FilterOperatorNotIn:
@@ -425,11 +421,7 @@ func (c *conditionBuilder) conditionForResolvedKey(
// instead of using NOT IN, we use `!=` + `AND` to make use of index
conditions := []string{}
for _, value := range values {
cond, err := c.conditionForResolvedKey(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorNotEqual, value, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
conditions = append(conditions, sb.NE(fieldExpression, value))
}
return sb.And(conditions...), nil

View File

@@ -905,52 +905,3 @@ func TestConditionForJSONBodySearch(t *testing.T) {
})
}
}
// IN on the body column routes each value back through the `=` path; the SQL it produces
// must stay what the shared IN handling produced before, including for a mixed-type list.
func TestConditionForBodyIn(t *testing.T) {
testCases := []struct {
name string
values []any
expectedSQL string
expectedArgs []any
}{
{
name: "strings",
values: []any{"alpha", "beta"},
expectedSQL: "(body = ? OR body = ?)",
expectedArgs: []any{"alpha", "beta"},
},
{
name: "mixed types are stringified before they reach the column",
values: []any{"alpha", float64(1), true},
expectedSQL: "(body = ? OR body = ? OR body = ?)",
expectedArgs: []any{"alpha", "1", "true"},
},
}
fl := flaggertest.New(t)
fm := NewFieldMapper(fl)
conditionBuilder := NewConditionBuilder(fm, fl)
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
key := telemetrytypes.TelemetryFieldKey{
Name: "body",
FieldContext: telemetrytypes.FieldContextLog,
FieldDataType: telemetrytypes.FieldDataTypeString,
}
sb := sqlbuilder.NewSelectBuilder()
sb.Select("1").From("t")
cond, _, err := conditionBuilder.ConditionFor(context.Background(), valuer.UUID{}, 0, 0, &key,
map[string][]*telemetrytypes.TelemetryFieldKey{key.Name: {&key}}, qbtypes.ConditionBuilderOptions{},
qbtypes.FilterOperatorIn, tc.values, sb)
require.NoError(t, err)
sb.Where(cond...)
sql, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
assert.Contains(t, sql, tc.expectedSQL)
assert.Equal(t, tc.expectedArgs, args)
})
}
}

View File

@@ -135,11 +135,7 @@ func (c *conditionBuilder) conditionFor(
// instead of using IN, we use `=` + `OR` to make use of index
conditions := []string{}
for _, value := range values {
cond, err := c.conditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorEqual, value, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
conditions = append(conditions, sb.E(fieldExpression, value))
}
return sb.Or(conditions...), nil
case qbtypes.FilterOperatorNotIn:
@@ -150,11 +146,7 @@ func (c *conditionBuilder) conditionFor(
// instead of using NOT IN, we use `!=` + `AND` to make use of index
conditions := []string{}
for _, value := range values {
cond, err := c.conditionFor(ctx, orgID, startNs, endNs, key, qbtypes.FilterOperatorNotEqual, value, sb)
if err != nil {
return "", err
}
conditions = append(conditions, cond)
conditions = append(conditions, sb.NE(fieldExpression, value))
}
return sb.And(conditions...), nil

View File

@@ -1,78 +0,0 @@
package telemetrytypes
import "strings"
// LogicalField is one queryable field. Its Name is the spelling that the
// request used. Its Members are the physical keys that store the field.
// LogicalField is the output type of name resolution: resolution changes a
// referenced name into logical fields, and compilers make SQL from them.
//
// A []*LogicalField shows ambiguity. Ambiguity means that possibly different
// fields have the same name. Each logical field in the slice gets its own
// condition. The operator tells the compiler how to connect the conditions.
//
// One LogicalField with more than one member shows a semantic-convention
// family. A family is one field that has more than one spelling. The members
// are in current-first order. The compiler merges the members into one
// expression, and the current name wins.
//
// Members always has one entry or more. A field that is not a family has
// exactly one member. The members point to the metadata map entries. Do not
// change the members.
type LogicalField struct {
// Name is the spelling that the request used. Aliases, series labels,
// and warnings use this spelling. Because of this, the response shows
// the same spelling as the request.
Name string
// Signal, FieldContext, and FieldDataType are the identity that all
// members share. Members with a different signal, field context, or
// data type are parts of different logical fields.
Signal Signal
FieldContext FieldContext
FieldDataType FieldDataType
// Members are the physical keys that store this field, in current-first
// order. Each member has its own physical data (Materialized,
// Evolutions, JSONPlan, ...). A per-member accessor does not need data
// from the other members.
Members []*TelemetryFieldKey
}
// SingleLogicalField makes a logical field that has one physical key.
func SingleLogicalField(name string, key *TelemetryFieldKey) *LogicalField {
return &LogicalField{
Name: name,
Signal: key.Signal,
FieldContext: key.FieldContext,
FieldDataType: key.FieldDataType,
Members: []*TelemetryFieldKey{key},
}
}
// Single returns the only member of a single-member field. A decision that
// uses only the shared identity can also use Single on a family. This is
// safe because all members have the same signal, context, and data type.
func (l *LogicalField) Single() *TelemetryFieldKey {
return l.Members[0]
}
// IsFamily returns true when the field has more than one physical member.
func (l *LogicalField) IsFamily() bool {
return len(l.Members) > 1
}
// String implements fmt.Stringer. A single-member field prints as its
// member. Because of this, a message made from the field and a message made
// from the key are the same. A family prints its shared identity and its
// member spellings.
func (l *LogicalField) String() string {
if len(l.Members) == 1 {
return l.Members[0].String()
}
names := make([]string, 0, len(l.Members))
for _, member := range l.Members {
names = append(names, member.Name)
}
return l.Name + "(" + l.FieldContext.StringValue() + ", " + l.FieldDataType.StringValue() + ", members: " + strings.Join(names, ", ") + ")"
}

View File

@@ -1,50 +0,0 @@
package telemetrytypes
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestSingleLogicalFieldSharesIdentityAndAliasesKey(t *testing.T) {
key := &TelemetryFieldKey{
Name: "service.name",
Signal: SignalTraces,
FieldContext: FieldContextResource,
FieldDataType: FieldDataTypeString,
}
logical := SingleLogicalField("resource.service.name", key)
assert.Equal(t, "resource.service.name", logical.Name, "the identity is the spelling that the request used, not the stored spelling")
assert.Equal(t, key.Signal, logical.Signal)
assert.Equal(t, key.FieldContext, logical.FieldContext)
assert.Equal(t, key.FieldDataType, logical.FieldDataType)
assert.False(t, logical.IsFamily())
assert.Same(t, key, logical.Single(), "the member points to the key; there is no copy")
}
func TestStringDelegatesForSingleMember(t *testing.T) {
key := &TelemetryFieldKey{
Name: "service.name",
FieldContext: FieldContextResource,
FieldDataType: FieldDataTypeString,
}
assert.Equal(t, key.String(), SingleLogicalField(key.Name, key).String(),
"a message made from a single-member field must be the same as a message made from the key")
}
func TestStringListsFamilyMembers(t *testing.T) {
logical := &LogicalField{
Name: "deployment.environment.name",
Signal: SignalTraces,
FieldContext: FieldContextResource,
FieldDataType: FieldDataTypeString,
Members: []*TelemetryFieldKey{
{Name: "deployment.environment.name"},
{Name: "deployment.environment"},
},
}
assert.True(t, logical.IsFamily())
assert.Equal(t, "deployment.environment.name(resource, string, members: deployment.environment.name, deployment.environment)", logical.String())
}

View File

@@ -3,13 +3,11 @@ from collections.abc import Callable
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
import pytest
import requests
from fixtures import types
from fixtures.auth import USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD
from fixtures.logs import Logs
from fixtures.querier import build_order_by, build_raw_query, get_rows, make_query_request
def test_logs_json_body_simple_searches(
@@ -913,61 +911,3 @@ def test_logs_json_body_listing(
assert len(results) == 1
count = results[0]["data"][0][0]
assert count == 4 # 4 logs have status="success"
@pytest.mark.parametrize(
"expression,expected_services",
[
pytest.param("body.service IN ['auth', 'payment']", {"auth", "payment"}, id="in_scalar_path"),
pytest.param("body.status IN [200, 500]", {"auth", "payment"}, id="in_number_path"),
pytest.param("body.service NOT IN ['auth']", {"payment", "search"}, id="not_in_scalar_path"),
# An `[]` path is extracted as an array. Comparing that array to each scalar in the
# list is something ClickHouse rejects outright (code 130), so this shape used to
# fail the whole query; per-value extraction reads the first element instead.
pytest.param("body.user_names[*] IN ['alpha', 'gamma']", {"auth", "payment"}, id="in_array_path"),
],
)
def test_logs_json_body_in_operator(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_logs: Callable[[list[Logs]], None],
expression: str,
expected_services: set[str],
) -> None:
"""IN over a body JSON path fans out to one comparison per value."""
now = datetime.now(tz=UTC).replace(second=0, microsecond=0)
specs = [("auth", 200, ["alpha", "beta"]), ("payment", 500, ["gamma"]), ("search", 404, ["beta", "alpha"])]
insert_logs(
[
Logs(
timestamp=now - timedelta(seconds=i + 1),
resources={"service.name": "api"},
body=json.dumps({"service": service, "status": status, "user_names": user_names}),
)
for i, (service, status, user_names) in enumerate(specs)
]
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
response = make_query_request(
signoz,
token,
start_ms=int((now - timedelta(minutes=5)).timestamp() * 1000),
end_ms=int(now.timestamp() * 1000),
request_type="raw",
queries=[
build_raw_query(
"A",
"logs",
filter_expression=expression,
order=[build_order_by("timestamp", "desc")],
limit=100,
)
],
)
assert response.status_code == HTTPStatus.OK, response.text
assert response.json()["status"] == "success"
# flag off: the body comes back as the raw JSON string
assert {json.loads(row["data"]["body"])["service"] for row in get_rows(response)} == expected_services