Compare commits

..

23 Commits

Author SHA1 Message Date
Abhi Kumar
e83a24ba59 refactor(charts): trim scatter chart layer comments and dead code 2026-10-13 13:10:12 +05:30
Abhi Kumar
784edafe23 fix(charts): pick the scatter auto scale by how the points spread
Max/min decades let one outlier decide and never left linear with zeros.
Compare middle-half coverage instead, falling back to symlog.
2026-10-13 13:10:10 +05:30
Abhi Kumar
bb9dc59b90 Merge remote-tracking branch 'origin/feat/scatter-plot-panel-spec' into feat/scatter-chart-layer 2026-10-12 19:33:57 +05:30
Abhi Kumar
b8a7db2ea3 feat(dashboards): add an axis to scatter plot thresholds 2026-10-12 19:27:35 +05:30
Naman Verma
3991224508 chore: add more validations for scatter plot 2026-10-12 17:11:20 +05:30
Naman Verma
a64aadcd81 Merge branch 'main' into feat/scatter-plot-panel-spec 2026-10-12 13:12:15 +05:30
Abhi Kumar
44c807cacc fix(charts): restore the default log axis grid
Assisted-by: Claude Opus 5.5
2026-10-08 17:52:49 +05:30
Abhi Kumar
62107fbb14 fix(dashboard): honour scatter plot soft limits on every axis scale
Only Auto weighs the soft limits when choosing a scale; an explicit scale keeps its choice.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:49 +05:30
Abhi Kumar
526c2c7d29 feat(charts): draw axis lines in the grid's style by default
Padding the scatter range moved the plot edges off the outer grid lines, which
had stood in for the axes.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:49 +05:30
Abhi Kumar
927b46a5b7 fix(charts): keep scatter dots whole at the plot edges
uPlot clips series to the plot area, so a dot at an extreme value lost the half
past the edge. Each scatter scale now pads by the largest dot's radius in
pixels. Log axes are no longer snapped to powers of ten, since snapping after
the pad could add a whole empty decade; they place 1/2/5 ticks themselves.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:49 +05:30
Abhi Kumar
3cbc0f4601 fix(charts): draw log axis grid lines at powers of ten only
uPlot splits a log axis at every 1–9 × 10ⁿ, and a line on each buried the
points under a mesh.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:49 +05:30
Abhi Kumar
47d38289f4 feat(charts): report the scatter point a click lands on
The shared click data finds its series by x position, which a faceted plot has
no axis for; the hit is read off the cursor instead.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:49 +05:30
Abhi Kumar
e334c48735 fix(charts): forward plotRef and skip x lookups on faceted clicks
ChartWrapper dropped its plotRef prop, and a click on a faceted plot threw in
posToIdx, which reads the null data[0].

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:49 +05:30
Abhi Kumar
a7e8a9e19c fix(charts): outline the hovered scatter point
Snap the ring to whole pixels so uPlot's ceil on left/top doesn't shift it
off the disc.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:48 +05:30
Abhi Kumar
dfcad79e79 feat(charts): scatter axis scales and titles
auto picks log only for positive values spanning three decades; log falls
back to symlog when given values at or below zero.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:48 +05:30
Abhi Kumar
befac14f47 chore: scatter plot base work 2026-10-08 17:52:48 +05:30
Abhi Kumar
3de4d6eaea feat(dashboards): rename scatter plot dimensions to colorBy and sizeBy 2026-10-08 17:52:05 +05:30
Abhi Kumar
e18afc0178 chore(openapi): regenerate specs for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-08 17:52:05 +05:30
Abhi Kumar
2bac73ee5c feat(dashboards): add point size and opacity to scatter plot spec
The fixed size and the min/max range are both stored so binding or unbinding
the size dimension restores the other's last setting.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:05 +05:30
Abhi Kumar
552f35e5bb chore(openapi): regenerate specs for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-08 17:52:05 +05:30
Abhi Kumar
9eba4a794f feat(dashboards): add axis labels and multi-key colour to scatter plot spec
An empty colour list colours by every group-by label.

Assisted-by: Claude Opus 5.5
2026-10-08 17:52:05 +05:30
Abhi Kumar
8a0c6c9911 chore(openapi): regenerate specs for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-08 17:52:05 +05:30
Abhi Kumar
d3662e66c7 feat(dashboards): add plugin schema for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-08 17:52:05 +05:30
74 changed files with 4184 additions and 1070 deletions

View File

@@ -3425,6 +3425,13 @@ components:
nullable: true
type: number
type: object
DashboardtypesAxisScale:
enum:
- auto
- linear
- log
- symlog
type: string
DashboardtypesBarChartPanelSpec:
properties:
axes:
@@ -4167,6 +4174,7 @@ components:
signoz/ListPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec'
signoz/NumberPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesNumberPanelSpec'
signoz/PieChartPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesPieChartPanelSpec'
signoz/ScatterPlotPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesScatterPlotPanelSpec'
signoz/TablePanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTablePanelSpec'
signoz/TextPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTextPanelSpec'
signoz/TimeSeriesPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTimeSeriesPanelSpec'
@@ -4182,6 +4190,7 @@ components:
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTextPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesScatterPlotPanelSpec'
type: object
DashboardtypesPanelPluginKind:
enum:
@@ -4195,6 +4204,7 @@ components:
- signoz/ListPanel
- signoz/TextPanel
- signoz/HeatmapPanel
- signoz/ScatterPlotPanel
type: string
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAreaChartPanelSpec:
properties:
@@ -4280,6 +4290,18 @@ components:
- kind
- spec
type: object
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesScatterPlotPanelSpec:
properties:
kind:
enum:
- signoz/ScatterPlotPanel
type: string
spec:
$ref: '#/components/schemas/DashboardtypesScatterPlotPanelSpec'
required:
- kind
- spec
type: object
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTablePanelSpec:
properties:
kind:
@@ -4359,6 +4381,16 @@ components:
visualization:
$ref: '#/components/schemas/DashboardtypesBasicVisualization'
type: object
DashboardtypesPointDiameter:
maximum: 40
minimum: 2
nullable: true
type: integer
DashboardtypesPointOpacity:
maximum: 1
minimum: 0.1
nullable: true
type: number
DashboardtypesPostableDashboardV2:
properties:
generateName:
@@ -4532,6 +4564,104 @@ components:
required:
- queryValue
type: object
DashboardtypesScatterPlotAxes:
properties:
x:
$ref: '#/components/schemas/DashboardtypesScatterPlotAxis'
"y":
$ref: '#/components/schemas/DashboardtypesScatterPlotAxis'
type: object
DashboardtypesScatterPlotAxis:
properties:
label:
description: Axis title. Empty draws none.
type: string
scale:
$ref: '#/components/schemas/DashboardtypesAxisScale'
softMax:
nullable: true
type: number
softMin:
nullable: true
type: number
type: object
DashboardtypesScatterPlotChartAppearance:
properties:
points:
$ref: '#/components/schemas/DashboardtypesScatterPlotPoints'
type: object
DashboardtypesScatterPlotDimensions:
properties:
colorBy:
description: Group-by label names (e.g. k8s.namespace.name) whose combined
values colour dots and drive the legend. Empty colours by every group-by
label.
items:
type: string
nullable: true
type: array
sizeBy:
description: Value column key that scales dot size. Empty draws every dot
at the default size.
type: string
x:
description: Value column key (queryName, or queryName.expression for a
multi-aggregation query) plotted on the x axis. Empty uses the first value
column.
type: string
"y":
description: Value column key plotted on the y axis. Empty uses the second
value column.
type: string
type: object
DashboardtypesScatterPlotPanelSpec:
properties:
axes:
$ref: '#/components/schemas/DashboardtypesScatterPlotAxes'
chartAppearance:
$ref: '#/components/schemas/DashboardtypesScatterPlotChartAppearance'
dimensions:
$ref: '#/components/schemas/DashboardtypesScatterPlotDimensions'
formatting:
$ref: '#/components/schemas/DashboardtypesTableFormatting'
legend:
$ref: '#/components/schemas/DashboardtypesLegend'
thresholds:
items:
$ref: '#/components/schemas/DashboardtypesScatterPlotThreshold'
nullable: true
type: array
visualization:
$ref: '#/components/schemas/DashboardtypesBasicVisualization'
type: object
DashboardtypesScatterPlotPoints:
properties:
maxSize:
$ref: '#/components/schemas/DashboardtypesPointDiameter'
minSize:
$ref: '#/components/schemas/DashboardtypesPointDiameter'
opacity:
$ref: '#/components/schemas/DashboardtypesPointOpacity'
size:
$ref: '#/components/schemas/DashboardtypesPointDiameter'
type: object
DashboardtypesScatterPlotThreshold:
properties:
axis:
$ref: '#/components/schemas/DashboardtypesThresholdAxis'
color:
type: string
label:
type: string
unit:
type: string
value:
format: double
type: number
required:
- value
- color
type: object
DashboardtypesSource:
enum:
- user
@@ -4645,6 +4775,11 @@ components:
- value
- name
type: object
DashboardtypesThresholdAxis:
enum:
- x
- "y"
type: string
DashboardtypesThresholdFormat:
enum:
- text

View File

@@ -57,6 +57,7 @@ const mockPaths = {
const mockTzDate = jest.fn(
(date: Date, _timezone: string) => new Date(date.getTime()),
);
const mockOrient = jest.fn();
// Mock uPlot constructor - this needs to be a proper constructor function
function MockUPlot(
@@ -70,6 +71,7 @@ function MockUPlot(
// Add static methods to the constructor
MockUPlot.tzDate = mockTzDate;
MockUPlot.paths = mockPaths;
MockUPlot.orient = mockOrient;
// Pinned so canvas-space maths in draw hooks is deterministic under jsdom.
MockUPlot.pxRatio = 1;

View File

Before

Width:  |  Height:  |  Size: 1.5 KiB

After

Width:  |  Height:  |  Size: 1.5 KiB

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" viewBox="0 0 60 60"><linearGradient id="a" x1="15.959" x2="44.307" y1="-13.014" y2="15.333" gradientTransform="matrix(1 0 0 -1 0 61)" gradientUnits="userSpaceOnUse"><stop offset=".097" style="stop-color:#0095d5"/><stop offset=".301" style="stop-color:#238ad9"/><stop offset=".621" style="stop-color:#557bde"/><stop offset=".864" style="stop-color:#7472e2"/><stop offset="1" style="stop-color:#806ee3"/></linearGradient><path d="m0 60 30.1-30.1L60 60z" style="fill:url(#a)"/><linearGradient id="b" x1="4.209" x2="20.673" y1="48.941" y2="65.405" gradientTransform="matrix(1 0 0 -1 0 61)" gradientUnits="userSpaceOnUse"><stop offset=".118" style="stop-color:#0095d5"/><stop offset=".418" style="stop-color:#3c83dc"/><stop offset=".696" style="stop-color:#6d74e1"/><stop offset=".833" style="stop-color:#806ee3"/></linearGradient><path d="M0 0h30.1L0 32.5z" style="fill:url(#b)"/><linearGradient id="c" x1="-10.102" x2="45.731" y1="5.836" y2="61.669" gradientTransform="matrix(1 0 0 -1 0 61)" gradientUnits="userSpaceOnUse"><stop offset=".107" style="stop-color:#c757bc"/><stop offset=".214" style="stop-color:#d0609a"/><stop offset=".425" style="stop-color:#e1725c"/><stop offset=".605" style="stop-color:#ee7e2f"/><stop offset=".743" style="stop-color:#f58613"/><stop offset=".823" style="stop-color:#f88909"/></linearGradient><path d="M30.1 0 0 31.7V60l30.1-30.1L60 0z" style="fill:url(#c)"/></svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

View File

@@ -1 +0,0 @@
<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path fill="#9CA3AF" d="M12 22.596c6.628 0 12-4.338 12-9.688 0-3.318-2.057-6.248-5.219-7.986-1.286-.715-2.297-1.357-3.139-1.89C14.058 2.025 13.08 1.404 12 1.404c-1.097 0-2.334.785-3.966 1.821a49.92 49.92 0 0 1-2.816 1.697C2.057 6.66 0 9.59 0 12.908c0 5.35 5.372 9.687 12 9.687v.001ZM10.599 4.715c.334-.759.503-1.58.498-2.409 0-.145.202-.187.23-.029.658 2.783-.902 4.162-2.057 4.624-.124.048-.199-.121-.103-.209a5.763 5.763 0 0 0 1.432-1.977Zm2.058-.102a5.82 5.82 0 0 0-.782-2.306v-.016c-.069-.123.086-.263.185-.172 1.962 2.111 1.307 4.067.556 5.051-.082.103-.23-.003-.189-.126a5.85 5.85 0 0 0 .23-2.431Zm1.776-.561a5.727 5.727 0 0 0-1.612-1.806v-.014c-.112-.085-.024-.274.114-.218 2.595 1.087 2.774 3.18 2.459 4.407a.116.116 0 0 1-.049.071.11.11 0 0 1-.153-.026.122.122 0 0 1-.022-.083 5.891 5.891 0 0 0-.737-2.331Zm-5.087.561c-.617.546-1.282.76-2.063 1-.117 0-.195-.078-.156-.181 1.752-.909 2.376-1.649 2.999-2.778 0 0 .155-.118.188.085 0 .304-.349 1.329-.968 1.874Zm4.945 11.237a2.957 2.957 0 0 1-.937 1.553c-.346.346-.8.565-1.286.62a2.178 2.178 0 0 1-1.327-.62 2.955 2.955 0 0 1-.925-1.553.244.244 0 0 1 .064-.198.234.234 0 0 1 .193-.069h3.965a.226.226 0 0 1 .19.07c.05.053.073.125.063.197Zm-5.458-2.176a1.862 1.862 0 0 1-2.384-.245 1.98 1.98 0 0 1-.233-2.447c.207-.319.503-.566.848-.713a1.84 1.84 0 0 1 1.092-.11c.366.075.703.261.967.531a1.98 1.98 0 0 1 .408 2.114 1.931 1.931 0 0 1-.698.869v.001Zm8.495.005a1.86 1.86 0 0 1-2.381-.253 1.964 1.964 0 0 1-.547-1.366c0-.384.11-.76.32-1.079.207-.319.503-.567.849-.713a1.844 1.844 0 0 1 1.093-.108c.367.076.704.262.968.534a1.98 1.98 0 0 1 .4 2.117 1.932 1.932 0 0 1-.702.868Z"/></svg>

Before

Width:  |  Height:  |  Size: 1.7 KiB

View File

@@ -1 +0,0 @@
<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path fill="#F25022" d="M1 1h10.5v10.5H1z"/><path fill="#7FBA00" d="M12.5 1H23v10.5H12.5z"/><path fill="#00A4EF" d="M1 12.5h10.5V23H1z"/><path fill="#FFB900" d="M12.5 12.5H23V23H12.5z"/></svg>

Before

Width:  |  Height:  |  Size: 264 B

View File

@@ -6,6 +6,7 @@ import {
useCallback,
useEffect,
useMemo,
useState,
} from 'react';
import { QueryKey } from 'react-query';
// eslint-disable-next-line no-restricted-imports
@@ -33,8 +34,6 @@ import { Warning } from 'types/api';
import { DataSource } from 'types/common/queryBuilder';
import { GlobalReducer } from 'types/reducer/globalTime';
import { getListOrderBy, setListOrderBy } from 'utils/explorerUtils';
import TraceExplorerControls from '../Controls';
import { getListViewQuery } from '../explorerUtils';
import {
@@ -59,16 +58,12 @@ function ListView({
setIsLoadingQueries,
queryKeyRef,
}: ListViewProps): JSX.Element {
const {
stagedQuery,
currentQuery,
redirectWithQueryBuilderData,
panelType: panelTypeFromQueryBuilder,
} = useQueryBuilder();
const { stagedQuery, panelType: panelTypeFromQueryBuilder } =
useQueryBuilder();
const panelType = panelTypeFromQueryBuilder || PANEL_TYPES.LIST;
const orderBy = useMemo(() => getListOrderBy(stagedQuery), [stagedQuery]);
const [orderBy, setOrderBy] = useState<string>('timestamp:desc');
const {
selectedTime: globalSelectedTime,
@@ -97,6 +92,7 @@ function ListView({
stagedQuery,
panelType,
paginationConfig,
orderBy,
],
[
stagedQuery,
@@ -105,6 +101,7 @@ function ListView({
paginationConfig,
maxTime,
minTime,
orderBy,
],
);
@@ -170,12 +167,9 @@ function ListView({
[queryTableData],
);
const handleOrderChange = useCallback(
(value: string) => {
redirectWithQueryBuilderData(setListOrderBy(currentQuery, value));
},
[currentQuery, redirectWithQueryBuilderData],
);
const handleOrderChange = useCallback((value: string) => {
setOrderBy(value);
}, []);
useEffect(() => {
if (!isLoading && !isFetching && !isError && rows.length !== 0) {

View File

@@ -6,6 +6,7 @@ import {
useCallback,
useEffect,
useMemo,
useState,
} from 'react';
import { QueryKey } from 'react-query';
// eslint-disable-next-line no-restricted-imports
@@ -22,7 +23,6 @@ import { TracesTableRow } from '../TracesTable/getFieldColumn';
import TracesTable from '../TracesTable/TracesTable';
import { useGetQueryRange } from 'hooks/queryBuilder/useGetQueryRange';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { getListOrderBy, setListOrderBy } from 'utils/explorerUtils';
import { Pagination } from 'hooks/queryPagination';
import useUrlQueryData from 'hooks/useUrlQueryData';
import { ArrowUp10, Minus } from '@signozhq/icons';
@@ -57,13 +57,9 @@ function TracesView({
setIsLoadingQueries,
queryKeyRef,
}: TracesViewProps): JSX.Element {
const { stagedQuery, currentQuery, redirectWithQueryBuilderData, panelType } =
useQueryBuilder();
const { stagedQuery, panelType } = useQueryBuilder();
const orderBy = useMemo(
() => getListOrderBy(stagedQuery, TRACE_VIEW_DEFAULT_ORDER_BY),
[stagedQuery],
);
const [orderBy, setOrderBy] = useState<string>(TRACE_VIEW_DEFAULT_ORDER_BY);
const {
columns,
@@ -98,6 +94,7 @@ function TracesView({
stagedQuery,
panelType,
paginationQueryData,
orderBy,
],
[
globalSelectedTime,
@@ -106,6 +103,7 @@ function TracesView({
stagedQuery,
panelType,
paginationQueryData,
orderBy,
],
);
@@ -167,12 +165,9 @@ function TracesView({
}
}, [isLoading, isFetching, isError, rows.length]);
const handleOrderChange = useCallback(
(value: string): void => {
redirectWithQueryBuilderData(setListOrderBy(currentQuery, value));
},
[currentQuery, redirectWithQueryBuilderData],
);
const handleOrderChange = useCallback((value: string): void => {
setOrderBy(value);
}, []);
// Without the full column set there is no pool to pick from, so the control is dropped.
const fieldsSelectorConfig = useMemo(

View File

@@ -1,7 +1,6 @@
import { initialQueriesMap } from 'constants/queryBuilder';
import { cloneDeep } from 'lodash-es';
import { OrderByPayload, Query } from 'types/api/queryBuilder/queryBuilderData';
import { parseListOrderBy } from 'utils/explorerUtils';
export const getListViewQuery = (
stagedQuery: Query,
@@ -12,7 +11,12 @@ export const getListViewQuery = (
: cloneDeep(initialQueriesMap.traces);
const orderByPayload: OrderByPayload[] = orderBy
? [parseListOrderBy(orderBy)]
? [
{
columnName: orderBy.split(':')[0],
order: orderBy.split(':')[1] as 'asc' | 'desc',
},
]
: [];
for (let i = 0; i < query.builder.queryData.length; i++) {

View File

@@ -16,14 +16,14 @@ function LogsActionsContainer({
showFrequencyChart,
handleToggleFrequencyChart,
orderBy,
onOrderChange,
setOrderBy,
explorerActions,
}: {
listQuery: any;
showFrequencyChart: boolean;
handleToggleFrequencyChart: () => void;
orderBy: string;
onOrderChange: (value: string) => void;
setOrderBy: (value: string) => void;
explorerActions: ReactNode;
}): JSX.Element {
const { options, config } = useOptionsMenu({
@@ -78,7 +78,7 @@ function LogsActionsContainer({
<ListViewOrderBy
value={orderBy}
onChange={onOrderChange}
onChange={(value): void => setOrderBy(value)}
dataSource={DataSource.LOGS}
/>
</div>

View File

@@ -59,7 +59,6 @@ import { QueryDataV3 } from 'types/api/widgets/getQuery';
import { DataSource } from 'types/common/queryBuilder';
import { GlobalReducer } from 'types/reducer/globalTime';
import { v4 } from 'uuid';
import { getListOrderBy, setListOrderBy } from 'utils/explorerUtils';
import LogsActionsContainer from './LogsActionsContainer';
@@ -102,8 +101,7 @@ function LogsExplorerViewsContainer({
const currentMinTimeRef = useRef<number>(minTime);
// Context
const { stagedQuery, currentQuery, redirectWithQueryBuilderData, panelType } =
useQueryBuilder();
const { stagedQuery, panelType } = useQueryBuilder();
const selectedPanelType = panelType || PANEL_TYPES.LIST;
@@ -113,7 +111,7 @@ function LogsExplorerViewsContainer({
const [requestData, setRequestData] = useState<Query | null>(null);
const [listChartQuery, setListChartQuery] = useState<Query | null>(null);
const orderBy = useMemo(() => getListOrderBy(stagedQuery), [stagedQuery]);
const [orderBy, setOrderBy] = useState<string>('timestamp:desc');
const { yAxisUnit, onUnitChange } = useUrlYAxisUnit('');
@@ -194,13 +192,6 @@ function LogsExplorerViewsContainer({
'custom',
);
const handleOrderChange = useCallback(
(value: string): void => {
redirectWithQueryBuilderData(setListOrderBy(currentQuery, value));
},
[currentQuery, redirectWithQueryBuilderData],
);
const getRequestData = useCallback(
(
query: Query | null,
@@ -332,10 +323,18 @@ function LogsExplorerViewsContainer({
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [data]);
// Store previous orderDirection to detect changes
const prevOrderByRef = useRef(orderBy);
useEffect(() => {
const orderByChanged =
prevOrderByRef.current !== orderBy && selectedPanelType === PANEL_TYPES.LIST;
prevOrderByRef.current = orderBy;
if (
requestData?.id !== stagedQuery?.id ||
currentMinTimeRef.current !== minTime
currentMinTimeRef.current !== minTime ||
orderByChanged
) {
const newRequestData = getRequestData(stagedQuery, {
filters: listQuery?.filters || initialFilters,
@@ -359,6 +358,7 @@ function LogsExplorerViewsContainer({
activeLogId,
selectedPanelType,
maxTime,
orderBy,
]);
const chartData = useMemo(() => {
@@ -430,7 +430,7 @@ function LogsExplorerViewsContainer({
showFrequencyChart={showFrequencyChart}
handleToggleFrequencyChart={handleToggleFrequencyChart}
orderBy={orderBy}
onOrderChange={handleOrderChange}
setOrderBy={setOrderBy}
explorerActions={explorerActions}
/>
)}

View File

@@ -1,7 +1,8 @@
import agnoUrl from '@/assets/Logos/agno.svg';
import amazonBedrockUrl from '@/assets/Logos/amazon-bedrock.svg';
import amazonMskUrl from '@/assets/Logos/amazon-msk.svg';
import androidUrl from '@/assets/Logos/android.svg';
import androidJavaMonitoringUrl from '@/assets/Logos/android-java-monitoring.svg';
import androidKotlinMonitoringUrl from '@/assets/Logos/android-kotlin-monitoring.svg';
import anthropicApiMonitoringUrl from '@/assets/Logos/anthropic-api-monitoring.svg';
import antigravityUrl from '@/assets/Logos/antigravity.svg';
import apacheDruidUrl from '@/assets/Logos/apache-druid.svg';
@@ -24,7 +25,6 @@ import azureSqlDatabaseMetricsUrl from '@/assets/Logos/azure-sql-database-metric
import azureVmUrl from '@/assets/Logos/azure-vm.svg';
import azureUrl from '@/assets/Logos/azure.svg';
import basetenUrl from '@/assets/Logos/baseten.svg';
import bunUrl from '@/assets/Logos/bun.svg';
import cassandraUrl from '@/assets/Logos/cassandra.svg';
import celeryUrl from '@/assets/Logos/celery.svg';
import certManagerUrl from '@/assets/Logos/cert-manager.svg';
@@ -126,7 +126,6 @@ import logstashUrl from '@/assets/Logos/logstash.svg';
import mastraUrl from '@/assets/Logos/mastra.svg';
import metaUrl from '@/assets/Logos/meta.svg';
import metricsUrl from '@/assets/Logos/metrics.svg';
import microsoftUrl from '@/assets/Logos/microsoft.svg';
import microsoftSqlServerUrl from '@/assets/Logos/microsoft-sql-server.svg';
import mistralUrl from '@/assets/Logos/mistral.svg';
import modalUrl from '@/assets/Logos/modal.svg';
@@ -4713,40 +4712,70 @@ const onboardingConfigWithLinks = [
link: '/docs/messaging-queues/celery-setup/',
},
{
dataSource: 'android',
label: 'Android',
imgUrl: androidUrl,
dataSource: 'android-java',
label: 'Android Java',
imgUrl: androidJavaMonitoringUrl,
tags: ['mobile app monitoring'],
module: 'apm',
relatedSearchKeywords: [
'android',
'android app tracing',
'android java',
'android kotlin',
'android java instrumentation',
'android java monitoring',
'android java to signoz',
'android logs',
'android metrics',
'android observability',
'android performance monitoring',
'android to signoz',
'android-java',
'apm',
'app',
'application performance monitoring',
'java',
'kotlin',
'logs',
'metrics',
'mobile',
'mobile app monitoring',
'monitor android apps',
'monitoring',
'observability',
'opentelemetry android',
'otel android agent',
'opentelemetry android java',
'otel android sdk',
'traces',
'tracing',
],
id: 'android',
link: '/docs/instrumentation/mobile-instrumentation/opentelemetry-android/',
link: '/docs/instrumentation/mobile-instrumentation/opentelemetry-java/',
},
{
dataSource: 'android-kotlin',
label: 'Android Kotlin',
imgUrl: androidKotlinMonitoringUrl,
tags: ['mobile app monitoring'],
module: 'apm',
relatedSearchKeywords: [
'android',
'android app performance kotlin',
'android kotlin',
'android kotlin monitoring',
'android kotlin to signoz',
'android observability kotlin',
'android telemetry sdk',
'android-kotlin',
'apm',
'app',
'application performance monitoring',
'kotlin',
'kotlin app logs',
'kotlin mobile tracing',
'mobile',
'mobile app monitoring',
'monitor kotlin apps',
'monitoring',
'observability',
'opentelemetry android kotlin',
'otel kotlin sdk',
'traces',
'tracing',
],
link: '/docs/instrumentation/mobile-instrumentation/opentelemetry-kotlin/',
},
{
dataSource: 'flutter',
@@ -7958,112 +7987,5 @@ const onboardingConfigWithLinks = [
id: 'aws-lambda-microvms',
link: '/docs/aws-monitoring/lambda-microvms/',
},
{
dataSource: 'bun-elysia',
label: 'Bun / Elysia',
imgUrl: bunUrl,
tags: ['apm/traces', 'metrics'],
module: 'apm',
relatedSearchKeywords: [
'apm',
'application performance monitoring',
'bun',
'bun elysia',
'bun metrics',
'bun monitoring',
'bun observability',
'bun traces',
'bunjs',
'elysia',
'elysiajs',
'javascript',
'metrics',
'observability',
'opentelemetry bun',
'traces',
'tracing',
'typescript',
],
id: 'bun-elysia',
link: '/docs/instrumentation/opentelemetry-bun/',
},
{
dataSource: 'snmp',
label: 'SNMP',
imgUrl: metricsUrl,
tags: ['metrics'],
module: 'metrics',
relatedSearchKeywords: [
'firewall',
'metrics',
'monitoring',
'network',
'network devices',
'network monitoring',
'observability',
'opentelemetry snmp',
'router',
'snmp',
'snmp metrics',
'snmp receiver',
'switch',
],
id: 'snmp',
link: '/docs/metrics-management/opentelemetry-snmp-metrics/',
},
{
dataSource: 'vercel-eve',
label: 'Vercel Eve',
imgUrl: vercelUrl,
tags: ['LLM Monitoring'],
module: 'apm',
relatedSearchKeywords: [
'agent',
'agents',
'ai agent',
'eve',
'eve dev',
'eve observability',
'llm',
'llm monitoring',
'monitoring',
'observability',
'token usage',
'traces',
'tracing',
'vercel',
'vercel agents',
'vercel eve',
],
id: 'vercel-eve',
link: '/docs/eve-observability/',
},
{
dataSource: 'microsoft-agent-framework',
label: 'Microsoft Agent Framework',
imgUrl: microsoftUrl,
tags: ['LLM Monitoring'],
module: 'apm',
relatedSearchKeywords: [
'agent',
'agent framework',
'agents',
'ai agent',
'llm',
'llm monitoring',
'maf',
'microsoft',
'microsoft agent framework',
'microsoft agent framework observability',
'monitoring',
'observability',
'token usage',
'traces',
'tracing',
'workflows',
],
id: 'microsoft-agent-framework',
link: '/docs/microsoft-agent-framework-observability/',
},
];
export default onboardingConfigWithLinks;

View File

@@ -7,6 +7,7 @@ import {
useCallback,
useEffect,
useMemo,
useState,
} from 'react';
// eslint-disable-next-line no-restricted-imports
import { useSelector } from 'react-redux';
@@ -33,7 +34,6 @@ import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { Pagination } from 'hooks/queryPagination';
import { getDefaultPaginationConfig } from 'hooks/queryPagination/utils';
import useUrlQueryData from 'hooks/useUrlQueryData';
import { getListOrderBy, setListOrderBy } from 'utils/explorerUtils';
import { ArrowUp10, Minus } from '@signozhq/icons';
import { AppState } from 'store/reducers';
import { Warning } from 'types/api';
@@ -66,16 +66,12 @@ function ListView({
queryKeyRef,
headerActions,
}: ListViewProps): JSX.Element {
const {
stagedQuery,
currentQuery,
redirectWithQueryBuilderData,
panelType: panelTypeFromQueryBuilder,
} = useQueryBuilder();
const { stagedQuery, panelType: panelTypeFromQueryBuilder } =
useQueryBuilder();
const panelType = panelTypeFromQueryBuilder || PANEL_TYPES.LIST;
const orderBy = useMemo(() => getListOrderBy(stagedQuery), [stagedQuery]);
const [orderBy, setOrderBy] = useState<string>('timestamp:desc');
const {
selectedTime: globalSelectedTime,
@@ -127,6 +123,7 @@ function ListView({
panelType,
paginationConfig,
selectColumnsSignature,
orderBy,
],
[
stagedQuery,
@@ -136,6 +133,7 @@ function ListView({
selectColumnsSignature,
maxTime,
minTime,
orderBy,
],
);
@@ -217,12 +215,9 @@ function ListView({
[config],
);
const handleOrderChange = useCallback(
(value: string) => {
redirectWithQueryBuilderData(setListOrderBy(currentQuery, value));
},
[currentQuery, redirectWithQueryBuilderData],
);
const handleOrderChange = useCallback((value: string) => {
setOrderBy(value);
}, []);
useEffect(() => {
if (!isLoading && !isFetching && !isError && rows.length !== 0) {

View File

@@ -2,7 +2,6 @@ import { initialQueriesMap, PANEL_TYPES } from 'constants/queryBuilder';
import { OptionsQuery } from 'container/OptionsMenu/types';
import { cloneDeep, set } from 'lodash-es';
import { OrderByPayload, Query } from 'types/api/queryBuilder/queryBuilderData';
import { parseListOrderBy } from 'utils/explorerUtils';
export const getListViewQuery = (
stagedQuery: Query,
@@ -13,7 +12,12 @@ export const getListViewQuery = (
: cloneDeep(initialQueriesMap.traces);
const orderByPayload: OrderByPayload[] = orderBy
? [parseListOrderBy(orderBy)]
? [
{
columnName: orderBy.split(':')[0],
order: orderBy.split(':')[1] as 'asc' | 'desc',
},
]
: [];
for (let i = 0; i < query.builder.queryData.length; i++) {

View File

@@ -1,108 +0,0 @@
import { renderHook } from '@testing-library/react';
import { initialQueriesMap, PANEL_TYPES } from 'constants/queryBuilder';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import { useHandleExplorerTabChange } from 'hooks/useHandleExplorerTabChange';
import { IBuilderQuery, Query } from 'types/api/queryBuilder/queryBuilderData';
jest.mock('hooks/queryBuilder/useQueryBuilder');
jest.mock('hooks/saveViews/useGetSavedViewParams', () => ({
useGetSavedViewParams: (): { viewName: string; viewKey: string } => ({
viewName: '',
viewKey: '',
}),
}));
const mockedUseQueryBuilder = jest.mocked(useQueryBuilder);
const countOrderBy = [{ columnName: 'count()', order: 'desc' }];
const queryWithOrder = (orderBy: typeof countOrderBy): Query => ({
...initialQueriesMap.traces,
builder: {
...initialQueriesMap.traces.builder,
queryData: initialQueriesMap.traces.builder.queryData.map((item) => ({
...item,
orderBy,
})),
},
});
const setUpQueryBuilder = (
from: PANEL_TYPES,
currentQuery: Query,
): jest.Mock => {
const redirectWithQueryBuilderData = jest.fn();
mockedUseQueryBuilder.mockReturnValue({
currentQuery,
panelType: from,
redirectWithQueryBuilderData,
updateAllQueriesOperators: (query: Query): Query => query,
updateQueriesData: (
query: Query,
type: 'queryData',
updateCallback: (item: IBuilderQuery, index: number) => IBuilderQuery,
): Query => ({
...query,
builder: {
...query.builder,
[type]: query.builder[type].map(updateCallback),
},
}),
} as unknown as ReturnType<typeof useQueryBuilder>);
return redirectWithQueryBuilderData;
};
const switchTab = (from: PANEL_TYPES, to: PANEL_TYPES, query: Query): Query => {
const redirect = setUpQueryBuilder(from, query);
const { result } = renderHook(() => useHandleExplorerTabChange());
result.current.handleExplorerTabChange(to);
return redirect.mock.calls[0][0] as Query;
};
describe('explorer tab change clears the order for raw panels', () => {
it.each([
[PANEL_TYPES.TIME_SERIES, PANEL_TYPES.LIST],
[PANEL_TYPES.TABLE, PANEL_TYPES.LIST],
[PANEL_TYPES.TRACE, PANEL_TYPES.LIST],
[PANEL_TYPES.LIST, PANEL_TYPES.TRACE],
])('clears it on %s -> %s', (from, to) => {
const query = switchTab(from, to, queryWithOrder(countOrderBy));
query.builder.queryData.forEach((item) => {
expect(item.orderBy).toStrictEqual([]);
});
});
it.each([
[PANEL_TYPES.LIST, PANEL_TYPES.TIME_SERIES],
[PANEL_TYPES.LIST, PANEL_TYPES.TABLE],
])('keeps it on %s -> %s, where the sanitizer validates it', (from, to) => {
const query = switchTab(from, to, queryWithOrder(countOrderBy));
query.builder.queryData.forEach((item) => {
expect(item.orderBy).toStrictEqual(countOrderBy);
});
});
it('leaves the order alone when the caller supplies its own query', () => {
const saved = queryWithOrder([{ columnName: 'timestamp', order: 'asc' }]);
const redirect = setUpQueryBuilder(
PANEL_TYPES.TIME_SERIES,
queryWithOrder(countOrderBy),
);
const { result } = renderHook(() => useHandleExplorerTabChange());
result.current.handleExplorerTabChange(PANEL_TYPES.LIST, { query: saved });
const sent = redirect.mock.calls[0][0] as Query;
sent.builder.queryData.forEach((item) => {
expect(item.orderBy).toStrictEqual([
{ columnName: 'timestamp', order: 'asc' },
]);
});
});
});

View File

@@ -2,6 +2,7 @@ import { useCallback } from 'react';
import { QueryParams } from 'constants/query';
import { initialAutocompleteData, PANEL_TYPES } from 'constants/queryBuilder';
import ROUTES from 'constants/routes';
import { SIGNOZ_VALUE } from 'container/QueryBuilder/filters/OrderByFilter/constants';
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import { DataSource } from 'types/common/queryBuilder';
@@ -46,7 +47,7 @@ export const useHandleExplorerTabChange = (): {
) {
query = updateQueriesData(query, 'queryData', (item) => ({
...item,
orderBy: [],
orderBy: item.orderBy.filter((item) => item.columnName !== SIGNOZ_VALUE),
aggregateAttribute: initialAutocompleteData,
}));
}

View File

@@ -0,0 +1,98 @@
// Sections own the padding so the footer spans edge to edge.
.container {
font-family: 'Inter';
font-size: 12px;
background: var(--l2-background);
-webkit-font-smoothing: antialiased;
color: var(--l2-foreground);
border-radius: 6px;
border: 1px solid var(--l2-border);
display: flex;
flex-direction: column;
min-width: 200px;
&.pinned {
border-color: var(--ring);
}
}
.divider {
display: block;
width: 100%;
height: 1px;
background-color: var(--l2-border);
}
.header {
display: flex;
align-items: center;
gap: var(--spacing-2);
padding: var(--spacing-3) var(--spacing-4);
}
// Matches the legend row's marker.
.marker {
width: 10px;
height: 10px;
border-radius: var(--radius);
flex-shrink: 0;
}
.title {
flex: 1;
min-width: 0;
font-family: var(--font-mono);
font-size: var(--font-size-xs);
font-weight: 600;
color: var(--text-vanilla-100);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.status {
display: flex;
align-items: center;
gap: var(--spacing-1);
font-size: 11px;
color: var(--text-vanilla-400);
flex-shrink: 0;
}
.rows {
display: flex;
flex-direction: column;
gap: var(--spacing-1);
padding: var(--spacing-3) var(--spacing-4);
}
.row {
display: flex;
align-items: baseline;
justify-content: space-between;
gap: var(--spacing-4);
font-family: var(--font-mono);
font-size: var(--font-size-xs);
font-variant-numeric: tabular-nums;
color: var(--text-vanilla-100);
}
.rowMuted {
color: var(--text-vanilla-400);
}
.rowLabel {
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.rowValue {
flex: 0 0 auto;
max-width: 60%;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}

View File

@@ -0,0 +1,107 @@
import { useMemo } from 'react';
import cx from 'classnames';
import { Pin } from '@signozhq/icons';
import { ScatterTooltipProps } from '../types';
import { buildChannelRows, resolveHoveredPoint } from './scatterTooltipContent';
import Styles from './ScatterTooltip.module.scss';
/** Own component: the shared Tooltip lists one row per series at a shared x. */
export default function ScatterTooltip({
uPlotInstance,
dataIndexes,
seriesIndex,
channels,
resolvePointLabels,
decimalPrecision,
isPinned,
dismiss,
renderTooltipFooter,
}: ScatterTooltipProps): JSX.Element | null {
const point = useMemo(
() => resolveHoveredPoint(uPlotInstance, seriesIndex, dataIndexes),
[uPlotInstance, seriesIndex, dataIndexes],
);
const rows = useMemo(
() => (point ? buildChannelRows(point, channels, decimalPrecision) : []),
[point, channels, decimalPrecision],
);
const labels = useMemo(
() =>
point
? (resolvePointLabels?.(point.seriesIndex, point.dataIndex) ?? [])
: [],
[point, resolvePointLabels],
);
if (!point) {
return null;
}
return (
<div
className={cx(Styles.container, { [Styles.pinned]: isPinned })}
data-pinned={isPinned}
data-testid="scatter-tooltip"
>
<div className={Styles.header}>
<span className={Styles.marker} style={{ backgroundColor: point.color }} />
<span
className={Styles.title}
title={point.label}
data-testid="scatter-tooltip-title"
>
{point.label}
</span>
{isPinned && (
<span className={Styles.status} data-testid="scatter-tooltip-status">
<Pin size={12} />
<span>Pinned</span>
</span>
)}
</div>
<span className={Styles.divider} />
<div className={Styles.rows}>
{rows.map((row) => (
<div
key={row.label}
className={Styles.row}
data-testid="scatter-tooltip-row"
>
<span className={Styles.rowLabel}>{row.label}</span>
<span className={Styles.rowValue}>{row.value}</span>
</div>
))}
</div>
{labels.length > 0 && (
<>
<span className={Styles.divider} />
<div className={Styles.rows}>
{labels.map((label) => (
<div
key={label.key}
className={cx(Styles.row, Styles.rowMuted)}
data-testid="scatter-tooltip-label"
>
<span className={Styles.rowLabel} title={label.key}>
{label.key}
</span>
<span className={Styles.rowValue} title={label.value}>
{label.value}
</span>
</div>
))}
</div>
</>
)}
{renderTooltipFooter?.({ isPinned, dismiss })}
</div>
);
}

View File

@@ -0,0 +1,94 @@
import type uPlot from 'uplot';
import {
buildChannelRows,
resolveHoveredPoint,
ScatterHoveredPoint,
} from '../scatterTooltipContent';
jest.mock('components/Graph/yAxisConfig', () => ({
getToolTipValue: jest.fn((value: number | string, unit?: string) =>
`${value} ${unit ?? ''}`.trim(),
),
}));
const plot = {
data: [
null,
[
[10, 20],
[100, 200],
[5, null],
],
[[30], [300]],
],
series: [
{},
{ label: 'cart', stroke: '#ff0000' },
{ label: 'checkout', stroke: (): string => '#00ff00' },
],
} as unknown as uPlot;
describe('resolveHoveredPoint', () => {
it('reads the focused series at its own data index', () => {
expect(resolveHoveredPoint(plot, 1, [null, 1, null])).toStrictEqual({
seriesIndex: 1,
dataIndex: 1,
label: 'cart',
color: '#ff0000',
x: 20,
y: 200,
size: null,
});
});
it('carries the size column when present and resolves function strokes', () => {
expect(resolveHoveredPoint(plot, 1, [null, 0, null])?.size).toBe(5);
expect(resolveHoveredPoint(plot, 2, [null, null, 0])).toMatchObject({
label: 'checkout',
color: '#00ff00',
size: null,
});
});
it('is null without a focused series or an index for it', () => {
expect(resolveHoveredPoint(plot, null, [null, 0, null])).toBeNull();
expect(resolveHoveredPoint(plot, 0, [0, 0, null])).toBeNull();
expect(resolveHoveredPoint(plot, 1, [null, null, null])).toBeNull();
});
});
describe('buildChannelRows', () => {
const point: ScatterHoveredPoint = {
seriesIndex: 1,
dataIndex: 0,
label: 'cart',
color: '#f00',
x: 12,
y: 340,
size: 7,
};
it('formats x and y with their own units', () => {
const rows = buildChannelRows(point, {
x: { label: 'Throughput', unit: 'reqps' },
y: { label: 'p99', unit: 'ms' },
});
expect(rows).toStrictEqual([
{ label: 'Throughput', value: '12 reqps' },
{ label: 'p99', value: '340 ms' },
]);
});
it('adds the size row only when the channel is mapped and the point has one', () => {
const channels = {
x: { label: 'x' },
y: { label: 'y' },
size: { label: 'Errors' },
};
expect(buildChannelRows(point, channels)).toHaveLength(3);
expect(buildChannelRows({ ...point, size: null }, channels)).toHaveLength(2);
});
});

View File

@@ -0,0 +1,89 @@
import { PrecisionOption } from 'components/Graph/types';
import { getToolTipValue } from 'components/Graph/yAxisConfig';
import type uPlot from 'uplot';
import type {
ScatterChannel,
ScatterChannels,
ScatterSeriesData,
} from '../../plugins/ScatterPlugin/types';
import { resolveSeriesColor } from './utils';
export interface ScatterHoveredPoint {
seriesIndex: number;
dataIndex: number;
label: string;
color: string;
x: number;
y: number;
size: number | null;
}
export interface ScatterTooltipRow {
label: string;
value: string;
}
export function resolveHoveredPoint(
u: uPlot,
seriesIndex: number | null,
dataIndexes: Array<number | null>,
): ScatterHoveredPoint | null {
if (seriesIndex == null || seriesIndex < 1) {
return null;
}
const dataIndex = dataIndexes[seriesIndex];
const series = u.series[seriesIndex];
const columns = u.data[seriesIndex] as unknown as
| ScatterSeriesData
| undefined;
if (dataIndex == null || !series || !columns) {
return null;
}
const x = columns[0][dataIndex];
const y = columns[1][dataIndex];
if (x == null || y == null) {
return null;
}
return {
seriesIndex,
dataIndex,
label: String(series.label ?? ''),
color: resolveSeriesColor(series.stroke, u, seriesIndex),
x,
y,
size: columns[2]?.[dataIndex] ?? null,
};
}
function formatChannel(
value: number,
channel: ScatterChannel,
decimalPrecision?: PrecisionOption,
): string {
return getToolTipValue(value, channel.unit, decimalPrecision);
}
export function buildChannelRows(
point: ScatterHoveredPoint,
channels: ScatterChannels,
decimalPrecision?: PrecisionOption,
): ScatterTooltipRow[] {
const rows: ScatterTooltipRow[] = [
{
label: channels.x.label,
value: formatChannel(point.x, channels.x, decimalPrecision),
},
{
label: channels.y.label,
value: formatChannel(point.y, channels.y, decimalPrecision),
},
];
if (channels.size && point.size != null) {
rows.push({
label: channels.size.label,
value: formatChannel(point.size, channels.size, decimalPrecision),
});
}
return rows;
}

View File

@@ -6,6 +6,10 @@ import uPlot from 'uplot';
import { UPlotConfigBuilder } from '../config/UPlotConfigBuilder';
import { LegendItem } from '../config/types';
import { HeatmapSeries, HeatmapYAxis } from '../plugins/HeatmapPlugin/types';
import type {
ScatterChannels,
ScatterPointLabel,
} from '../plugins/ScatterPlugin/types';
import { SyncTooltipFilterMode } from '../plugins/TooltipPlugin/types';
/**
@@ -119,6 +123,16 @@ export interface HeatmapTooltipProps
groupColor: string;
}
export interface ScatterTooltipProps
extends BaseTooltipProps, TooltipRenderArgs {
channels: ScatterChannels;
/** The group values behind a point, e.g. `service.name` → `cart`. */
resolvePointLabels?: (
seriesIndex: number,
dataIndex: number,
) => ScatterPointLabel[];
}
export type TooltipProps =
| TimeSeriesTooltipProps
| BarTooltipProps

View File

@@ -39,6 +39,16 @@ export class UPlotAxisBuilder extends ConfigBuilder<AxisProps, Axis> {
};
}
/** Styled like the grid. */
private buildBorderConfig(
grid: uPlot.Axis.Grid | undefined,
): uPlot.Axis.Border | undefined {
if (this.props.showBorder === false || !grid || grid.show === false) {
return undefined;
}
return { show: true, stroke: grid.stroke, width: grid.width };
}
/**
* Build ticks configuration
*/
@@ -57,33 +67,43 @@ export class UPlotAxisBuilder extends ConfigBuilder<AxisProps, Axis> {
};
}
/**
* Build values formatter for X-axis (time)
*/
/** Time ticks, or unit-formatted values when a unit/precision is set. */
private buildXAxisValuesFormatter(): uPlot.Axis.Values | undefined {
const { isTimeAxis } = this.props;
const { isTimeAxis, yAxisUnit, decimalPrecision } = this.props;
if (isTimeAxis) {
return uPlotXAxisValuesFormat as uPlot.Axis.Values;
}
if (yAxisUnit !== undefined || decimalPrecision !== undefined) {
return this.buildValueAxisFormatter();
}
return undefined;
}
/**
* Build values formatter for Y-axis (values with units)
*/
private buildYAxisValuesFormatter(): uPlot.Axis.Values {
const { yAxisUnit, decimalPrecision } = this.props;
/** Blanks splits outside the scale; uPlot's asinh splits always include ±threshold. */
private buildValueAxisFormatter(): uPlot.Axis.Values {
const { yAxisUnit, decimalPrecision, scaleKey } = this.props;
return (_, t): string[] =>
t.map((v) => {
if (v === null || v === undefined || Number.isNaN(v)) {
return (u, t): string[] => {
const scale = u?.scales?.[scaleKey];
const min = scale?.min ?? -Infinity;
const max = scale?.max ?? Infinity;
return t.map((v) => {
if (
v === null ||
v === undefined ||
Number.isNaN(v) ||
v < min ||
v > max
) {
return '';
}
const value = getToolTipValue(v.toString(), yAxisUnit, decimalPrecision);
return `${value}`;
});
};
}
/**
@@ -101,7 +121,7 @@ export class UPlotAxisBuilder extends ConfigBuilder<AxisProps, Axis> {
return scaleKey === 'x'
? this.buildXAxisValuesFormatter()
: scaleKey === 'y'
? this.buildYAxisValuesFormatter()
? this.buildValueAxisFormatter()
: undefined;
}
@@ -149,10 +169,12 @@ export class UPlotAxisBuilder extends ConfigBuilder<AxisProps, Axis> {
side = 2, // bottom by default
space,
splits,
filter,
gap = 5, // default gap is 5
} = this.props;
const grid = this.buildGridConfig();
const border = this.buildBorderConfig(grid);
const ticks = this.buildTicksConfig();
const values = this.buildValuesFormatter();
const size = this.buildSizeCalculator();
@@ -177,6 +199,9 @@ export class UPlotAxisBuilder extends ConfigBuilder<AxisProps, Axis> {
if (ticks) {
axisConfig.ticks = ticks;
}
if (border) {
axisConfig.border = border;
}
if (values) {
axisConfig.values = values;
}
@@ -189,6 +214,9 @@ export class UPlotAxisBuilder extends ConfigBuilder<AxisProps, Axis> {
if (space !== undefined) {
axisConfig.space = space;
}
if (filter) {
axisConfig.filter = filter;
}
if (size) {
axisConfig.size = size;
}

View File

@@ -19,6 +19,7 @@ import {
ConfigBuilder,
ConfigBuilderProps,
LegendItem,
PlotMode,
SelectionPreferencesSource,
StackMode,
} from './types';
@@ -65,6 +66,8 @@ export class UPlotConfigBuilder extends ConfigBuilder<
private stackMode: StackMode = StackMode.None;
private mode: PlotMode = PlotMode.Aligned;
private cursor: Cursor | undefined;
private hooks: Hooks.Arrays = {};
@@ -160,6 +163,10 @@ export class UPlotConfigBuilder extends ConfigBuilder<
return this.stackMode;
}
setMode(mode: PlotMode): void {
this.mode = mode;
}
/**
* Add or merge a scale configuration
*/
@@ -512,6 +519,10 @@ export class UPlotConfigBuilder extends ConfigBuilder<
{} as Record<string, uPlot.Scale>,
);
if (this.mode === PlotMode.Faceted) {
config.mode = this.mode as number as uPlot.Mode;
}
config.hooks = this.hooks;
config.select = this.select;

View File

@@ -40,6 +40,7 @@ export class UPlotScaleBuilder extends ConfigBuilder<
range,
thresholds,
logBase = 10,
asinhThreshold,
padMinBy = 0,
padMaxBy = 0.05,
} = this.props;
@@ -99,6 +100,7 @@ export class UPlotScaleBuilder extends ConfigBuilder<
time,
distr,
logBase,
asinhThreshold,
});
const { rangeConfig, hardMinOnly, hardMaxOnly, hasFixedRange } =

View File

@@ -87,6 +87,8 @@ export class UPlotSeriesBuilder extends ConfigBuilder<
lineConfig.fill = finalFillColor;
} else if (this.props.drawStyle === DrawStyle.Histogram) {
lineConfig.fill = `${finalFillColor}40`;
} else if (this.props.drawStyle === DrawStyle.Scatter) {
lineConfig.fill = `${finalFillColor}${toAlphaHex(resolveFillOpacity(fillOpacity))}`;
} else if (fillMode && fillMode !== FillMode.None) {
const resolvedOpacity = resolveFillOpacity(fillOpacity);
if (fillMode === FillMode.Solid) {
@@ -122,7 +124,7 @@ export class UPlotSeriesBuilder extends ConfigBuilder<
return { paths: pathBuilder };
}
if (drawStyle === DrawStyle.Points) {
if (drawStyle === DrawStyle.Points || drawStyle === DrawStyle.Scatter) {
return { paths: (): null => null };
}
@@ -194,6 +196,10 @@ export class UPlotSeriesBuilder extends ConfigBuilder<
if (drawStyle === DrawStyle.Points) {
return true;
}
// The discs are the series path; uPlot's own points would double-draw them.
if (drawStyle === DrawStyle.Scatter) {
return false;
}
return !!showPoints;
}
@@ -218,7 +224,7 @@ export class UPlotSeriesBuilder extends ConfigBuilder<
}
getConfig(): ExtendedSeries {
const { scaleKey, label, spanGaps, show = true, metric } = this.props;
const { scaleKey, label, spanGaps, show = true, metric, facets } = this.props;
const resolvedLineColor = this.getLineColor();
@@ -246,6 +252,7 @@ export class UPlotSeriesBuilder extends ConfigBuilder<
...pathConfig,
points: Object.keys(pointsConfig).length > 0 ? pointsConfig : undefined,
metric,
...(facets && { facets }),
};
}
}

View File

@@ -120,6 +120,41 @@ describe('UPlotAxisBuilder', () => {
});
});
it('draws the axis line by default, styled like the grid', () => {
const config = new UPlotAxisBuilder(
createAxisProps({ isDarkMode: true, grid: { width: 0.5 } }),
).getConfig();
expect(config.border).toStrictEqual({
show: true,
stroke: config.grid?.stroke,
width: 0.5,
});
});
it('leaves the axis line off when showBorder is false', () => {
expect(
new UPlotAxisBuilder(createAxisProps({ showBorder: false })).getConfig(),
).not.toHaveProperty('border');
});
it('leaves the axis line off when the grid is hidden', () => {
expect(
new UPlotAxisBuilder(createAxisProps({ grid: { show: false } })).getConfig(),
).not.toHaveProperty('border');
});
it('passes explicit splits and a label filter through to uPlot', () => {
const splits = jest.fn();
const filter = jest.fn();
const config = new UPlotAxisBuilder(
createAxisProps({ splits, filter }),
).getConfig();
expect(config.splits).toBe(splits);
expect(config.filter).toBe(filter);
});
it('uses provided ticks config when present and falls back to defaults otherwise', () => {
const customTicks = { width: 1, show: false };
const withTicks = new UPlotAxisBuilder(
@@ -376,3 +411,51 @@ describe('UPlotAxisBuilder', () => {
expect(config.values).toBeDefined();
});
});
describe('UPlotAxisBuilder value x axis', () => {
it('formats a non-time x axis with its unit', () => {
(getToolTipValue as jest.Mock).mockReturnValue('1.2K req/s');
const config = new UPlotAxisBuilder(
createAxisProps({ scaleKey: 'x', isTimeAxis: false, yAxisUnit: 'reqps' }),
).getConfig();
const values = (config.values as uPlot.Axis.DynamicValues)(
{} as uPlot,
[1200],
0,
0,
0,
);
expect(values).toStrictEqual(['1.2K req/s']);
expect(getToolTipValue).toHaveBeenCalledWith('1200', 'reqps', undefined);
});
it('leaves a non-time x axis to uPlot when nothing says how to format it', () => {
const config = new UPlotAxisBuilder(
createAxisProps({ scaleKey: 'x', isTimeAxis: false }),
).getConfig();
expect(config.values).toBeUndefined();
});
});
describe('UPlotAxisBuilder out-of-range splits', () => {
it('leaves a split the scale cannot place unlabelled', () => {
(getToolTipValue as jest.Mock).mockImplementation((v: string) => `${v} ms`);
const config = new UPlotAxisBuilder(
createAxisProps({ scaleKey: 'y', yAxisUnit: 'ms' }),
).getConfig();
const u = { scales: { y: { min: 0, max: 1000 } } } as unknown as uPlot;
const values = (config.values as uPlot.Axis.DynamicValues)(
u,
[-10, 0, 500, 5000],
0,
0,
0,
);
expect(values).toStrictEqual(['', '0 ms', '500 ms', '']);
});
});

View File

@@ -5,7 +5,12 @@ import {
STEP_INTERVAL_MULTIPLIER,
} from '../../constants';
import type { SeriesProps } from '../types';
import { DrawStyle, SelectionPreferencesSource, StackMode } from '../types';
import {
DrawStyle,
PlotMode,
SelectionPreferencesSource,
StackMode,
} from '../types';
import { UPlotConfigBuilder } from '../UPlotConfigBuilder';
// Mock only the real boundary that hits localStorage
@@ -651,3 +656,14 @@ describe('UPlotConfigBuilder stacking', () => {
expect(builder.getConfig().bands).toStrictEqual([{ series: [1, 3] }]);
});
});
describe('UPlotConfigBuilder plot mode', () => {
it('leaves mode unset for aligned data and emits 2 when faceted', () => {
const aligned = new UPlotConfigBuilder({ id: 'aligned' });
expect(aligned.getConfig().mode).toBeUndefined();
const faceted = new UPlotConfigBuilder({ id: 'faceted' });
faceted.setMode(PlotMode.Faceted);
expect(faceted.getConfig().mode).toBe(2);
});
});

View File

@@ -399,3 +399,39 @@ describe('UPlotSeriesBuilder', () => {
expect(builder.getConfig().fill).toBeUndefined();
});
});
describe('UPlotSeriesBuilder scatter', () => {
it('draws through the given path builder and hides uPlot points', () => {
const pathBuilder = jest.fn();
const config = new UPlotSeriesBuilder(
createBaseProps({
drawStyle: DrawStyle.Scatter,
pathBuilder,
facets: [{ scale: 'x' }, { scale: 'y' }],
lineColor: '#ff0000',
fillOpacity: 0.5,
lineWidth: 1,
pointSize: 8,
}),
).getConfig();
expect(config.paths).toBe(pathBuilder);
expect(config.facets).toStrictEqual([{ scale: 'x' }, { scale: 'y' }]);
expect(config.points?.show).toBe(false);
expect(config.points?.size).toBe(8);
expect(config.stroke).toBe('#ff0000');
expect(config.width).toBe(1);
expect(config.fill).toBe('#ff000080');
});
it('draws nothing without a path builder', () => {
const config = new UPlotSeriesBuilder(
createBaseProps({ drawStyle: DrawStyle.Scatter }),
).getConfig();
expect(
(config.paths as uPlot.Series.PathBuilder)({} as uPlot, 1, 0, 0),
).toBeNull();
expect(config.facets).toBeUndefined();
});
});

View File

@@ -78,8 +78,12 @@ export interface AxisProps {
};
/** Explicit tick formatter, replacing the scale's default (time / unit-formatted). */
values?: uPlot.Axis.Values;
/** Explicit axis splits, overriding the default tick calculation. */
/** Explicit tick positions, replacing uPlot's per-distribution default. */
splits?: uPlot.Axis.Splits;
/** Which splits get a label, replacing uPlot's per-distribution default. */
filter?: uPlot.Axis.Filter;
/** Draw the axis line along the plot edge, styled like the grid. Default true. */
showBorder?: boolean;
/** Pixels between the ticks and their labels; also feeds the y axis width calculation. */
gap?: number;
/** Explicit axis thickness. Left unset, the y axis sizes itself to its widest label. */
@@ -90,7 +94,7 @@ export interface AxisProps {
isDarkMode?: boolean;
/** Axis is on a log scale — thins the grid lines to keep dense decades readable. */
isLogScale?: boolean;
/** Unit the y axis ticks are formatted in (`spec.formatting.unit`). */
/** Unit the value ticks are formatted in (`spec.formatting.unit`); a non-time x axis reads it too. */
yAxisUnit?: string;
/**
* X axis carries timestamps, so its ticks format as dates/times. Declared by the caller
@@ -109,6 +113,14 @@ export interface AxisProps {
export enum DistributionType {
Linear = 'linear',
Logarithmic = 'logarithmic',
/** arcsinh: linear within ±`asinhThreshold`, log beyond; places zero and negatives. */
SymmetricLog = 'symlog',
}
/** uPlot's data layout: one shared x per chart, or per-series x/y columns. */
export enum PlotMode {
Aligned = 1,
Faceted = 2,
}
export interface ScaleProps {
@@ -125,6 +137,8 @@ export interface ScaleProps {
auto?: boolean;
logBase?: uPlot.Scale.LogBase;
distribution?: DistributionType;
/** Half-width of a `SymmetricLog` scale's linear band around zero. Default 1. */
asinhThreshold?: number;
}
export enum DisconnectedValuesMode {
@@ -146,6 +160,8 @@ export enum DrawStyle {
Points = 'points',
Bar = 'bar',
Histogram = 'histogram',
/** Mode 2; drawn by the caller's `pathBuilder`. */
Scatter = 'scatter',
}
export enum LineInterpolation {
@@ -229,6 +245,8 @@ export interface SeriesProps extends LineConfig, PointsConfig, BarConfig {
isDarkMode?: boolean;
stepInterval?: number;
metric?: { [key: string]: string };
/** Mode 2 only: the scales the series' own x and y columns are read against. */
facets?: Series.Facet[];
}
export interface LegendItem {

View File

@@ -0,0 +1,114 @@
import type uPlot from 'uplot';
import { Quadtree } from '../../../utils/quadtree';
import {
resolveHit,
resolvePointDiameter,
resolveSizeDomain,
} from '../geometry';
import { ScatterHit, ScatterPointSize } from '../types';
const POINT_SIZE: ScatterPointSize = { fixed: 6, min: 4, max: 20 };
const asData = (columns: unknown[]): uPlot.AlignedData =>
columns as unknown as uPlot.AlignedData;
describe('resolveSizeDomain', () => {
it('spans the size columns of every series, skipping nulls', () => {
const data = asData([
null,
[
[1, 2],
[1, 2],
[10, null],
],
[[3], [3], [40]],
]);
expect(resolveSizeDomain(data)).toStrictEqual({ min: 10, max: 40 });
});
it('is null when no series carries sizes', () => {
expect(resolveSizeDomain(asData([null, [[1], [1]]]))).toBeNull();
expect(resolveSizeDomain(asData([null, [[1], [1], [null]]]))).toBeNull();
});
});
describe('resolvePointDiameter', () => {
it('uses the fixed diameter without a size or a domain', () => {
expect(resolvePointDiameter(null, { min: 0, max: 10 }, POINT_SIZE)).toBe(6);
expect(resolvePointDiameter(5, null, POINT_SIZE)).toBe(6);
});
it('maps the domain ends to min and max', () => {
const domain = { min: 0, max: 100 };
expect(resolvePointDiameter(0, domain, POINT_SIZE)).toBe(4);
expect(resolvePointDiameter(100, domain, POINT_SIZE)).toBe(20);
});
it('scales by area, not diameter', () => {
const midArea = (4 ** 2 + 20 ** 2) / 2;
expect(
resolvePointDiameter(50, { min: 0, max: 100 }, POINT_SIZE),
).toBeCloseTo(Math.sqrt(midArea));
});
it('clamps values outside the domain', () => {
const domain = { min: 10, max: 20 };
expect(resolvePointDiameter(-5, domain, POINT_SIZE)).toBe(4);
expect(resolvePointDiameter(500, domain, POINT_SIZE)).toBe(20);
});
it('uses the midpoint when every size is the same', () => {
expect(resolvePointDiameter(7, { min: 7, max: 7 }, POINT_SIZE)).toBe(12);
});
});
describe('resolveHit', () => {
const hit = (
seriesIndex: number,
dataIndex: number,
x: number,
y: number,
d: number,
): ScatterHit => ({ seriesIndex, dataIndex, x, y, w: d, h: d });
it('returns the disc under the cursor', () => {
const tree = new Quadtree<ScatterHit>(0, 0, 100, 100);
tree.add(hit(1, 0, 10, 10, 6));
tree.add(hit(2, 3, 50, 50, 6));
expect(resolveHit(tree, 13, 13, 0)).toMatchObject({
seriesIndex: 1,
dataIndex: 0,
});
expect(resolveHit(tree, 52, 52, 0)).toMatchObject({
seriesIndex: 2,
dataIndex: 3,
});
});
it('is null when the cursor is off every disc', () => {
const tree = new Quadtree<ScatterHit>(0, 0, 100, 100);
tree.add(hit(1, 0, 10, 10, 6));
expect(resolveHit(tree, 30, 30, 0)).toBeNull();
});
it('tolerance widens each disc', () => {
const tree = new Quadtree<ScatterHit>(0, 0, 100, 100);
tree.add(hit(1, 0, 10, 10, 6));
expect(resolveHit(tree, 18, 13, 0)).toBeNull();
expect(resolveHit(tree, 18, 13, 3)).not.toBeNull();
});
it('prefers the disc whose centre is nearest when they overlap', () => {
const tree = new Quadtree<ScatterHit>(0, 0, 100, 100);
tree.add(hit(1, 0, 10, 10, 10));
tree.add(hit(1, 1, 14, 10, 10));
expect(resolveHit(tree, 13, 15, 0)?.dataIndex).toBe(0);
expect(resolveHit(tree, 21, 15, 0)?.dataIndex).toBe(1);
});
});

View File

@@ -0,0 +1,286 @@
import uPlot from 'uplot';
import { UPlotConfigBuilder } from '../../../config/UPlotConfigBuilder';
import {
applyScatterPlugin,
createScatterPlugin,
SCATTER_FACETS,
} from '../scatterPlugin';
jest.mock('lib/visualization/panels/utils/legendVisibilityUtils', () => ({
getStoredSeriesVisibility: jest.fn(),
}));
/** jsdom has no Path2D; the builder only needs something that takes the calls. */
class FakePath2D {
moveTo = jest.fn();
arc = jest.fn();
}
type OrientCallback = Parameters<typeof uPlot.orient>[2];
interface FakePlotArgs {
series: Array<{ xs: number[]; ys: number[]; sizes?: Array<number | null> }>;
cursor?: { left: number; top: number };
scaleX?: { min: number; max: number };
scaleY?: { min: number; max: number };
}
/**
* A 100×100 plot at the canvas origin with identity scales: value 10 draws at
* pixel 10 on x, and at 100 − 10 on y (uPlot's y grows downward).
*/
function createFakePlot({
series,
cursor = { left: -1, top: -1 },
scaleX = { min: 0, max: 100 },
scaleY = { min: 0, max: 100 },
}: FakePlotArgs): uPlot {
const data = [
null,
...series.map((entry) =>
entry.sizes ? [entry.xs, entry.ys, entry.sizes] : [entry.xs, entry.ys],
),
];
return {
data,
series: [{}, ...series.map((_, index) => ({ label: `s${index + 1}` }))],
bbox: { left: 0, top: 0, width: 100, height: 100 },
cursor,
scales: { x: scaleX, y: scaleY },
} as unknown as uPlot;
}
/** Stands in for `uPlot.orient`: identity x, flipped y, an `arc` that records. */
function orientWithIdentityScales(
u: uPlot,
seriesIdx: number,
cb: OrientCallback,
): void {
const columns = (u.data as unknown as Array<number[][] | null>)[seriesIdx];
if (!columns) {
return;
}
const scaleX = (u.scales as unknown as Record<string, uPlot.Scale>).x;
const scaleY = (u.scales as unknown as Record<string, uPlot.Scale>).y;
const valToPosX = (value: number): number => value;
const valToPosY = (value: number): number => 100 - value;
// Real uPlot's `arc` helper forwards to the path; the test counts those calls.
const arc = (path: FakePath2D, ...args: number[]): void => {
path.arc(...args);
};
cb(
u.series[seriesIdx],
columns[0],
columns[1],
scaleX,
scaleY,
valToPosX as unknown as uPlot.ValToPos,
valToPosY as unknown as uPlot.ValToPos,
0,
0,
100,
100,
jest.fn() as never,
jest.fn() as never,
jest.fn() as never,
arc as never,
jest.fn() as never,
);
}
describe('createScatterPlugin', () => {
beforeAll(() => {
(globalThis as { Path2D?: unknown }).Path2D = FakePath2D;
});
beforeEach(() => {
(uPlot.orient as jest.Mock).mockImplementation(orientWithIdentityScales);
});
afterEach(() => {
(uPlot.orient as jest.Mock).mockReset();
});
function drawAll(
u: uPlot,
plugin: ReturnType<typeof createScatterPlugin>,
): void {
plugin.hooks.drawClear(u);
for (let seriesIdx = 1; seriesIdx < u.series.length; seriesIdx++) {
const columns = (u.data as unknown as number[][][])[seriesIdx];
plugin.pathBuilder(u, seriesIdx, 0, columns[0].length - 1);
}
}
/** Runs the cursor scan the way uPlot does: every data series, in order. */
function scan(
u: uPlot,
plugin: ReturnType<typeof createScatterPlugin>,
): Array<number | null> {
const dataIdx = plugin.cursor.dataIdx as NonNullable<uPlot.Cursor['dataIdx']>;
const indexes: Array<number | null> = [null];
for (let seriesIdx = 1; seriesIdx < u.series.length; seriesIdx++) {
indexes.push(dataIdx(u, seriesIdx, 0, 0));
}
return indexes;
}
it('returns one path that strokes and fills the same discs', () => {
const plugin = createScatterPlugin();
const u = createFakePlot({ series: [{ xs: [10, 20], ys: [10, 20] }] });
plugin.hooks.drawClear(u);
const paths = plugin.pathBuilder(u, 1, 0, 1) as uPlot.Series.Paths;
expect(paths.stroke).toBeInstanceOf(FakePath2D);
expect(paths.fill).toBe(paths.stroke);
expect((paths.fill as unknown as FakePath2D).arc).toHaveBeenCalledTimes(2);
});
it('resolves the hovered point to its own series and index', () => {
const plugin = createScatterPlugin({
pointSize: { fixed: 6, min: 4, max: 20 },
});
const u = createFakePlot({
series: [
{ xs: [10, 50], ys: [10, 50] },
{ xs: [80], ys: [80] },
],
// Over the second series' only point: x 80, y drawn at 100 − 80.
cursor: { left: 80, top: 20 },
});
drawAll(u, plugin);
expect(scan(u, plugin)).toStrictEqual([null, null, 0]);
expect(plugin.getHit()).toMatchObject({ seriesIndex: 2, dataIndex: 0 });
});
it('returns null for every series when the cursor is off the plot or off any disc', () => {
const plugin = createScatterPlugin();
const u = createFakePlot({
series: [{ xs: [10], ys: [10] }],
cursor: { left: -1, top: -1 },
});
drawAll(u, plugin);
expect(scan(u, plugin)).toStrictEqual([null, null]);
(u.cursor as { left: number; top: number }).left = 50;
(u.cursor as { left: number; top: number }).top = 50;
expect(scan(u, plugin)).toStrictEqual([null, null]);
});
it('skips points outside the visible scale range', () => {
const plugin = createScatterPlugin();
const u = createFakePlot({
series: [{ xs: [10, 500], ys: [10, 10] }],
cursor: { left: 10, top: 90 },
});
drawAll(u, plugin);
const paths = plugin.pathBuilder(u, 1, 0, 1) as uPlot.Series.Paths;
expect((paths.fill as unknown as FakePath2D).arc).toHaveBeenCalledTimes(1);
expect(scan(u, plugin)).toStrictEqual([null, 0]);
});
it('rings the hit disc, in CSS pixels, with a gap around it', () => {
const plugin = createScatterPlugin({
pointSize: { fixed: 8, min: 4, max: 20 },
});
const u = createFakePlot({
series: [{ xs: [10], ys: [10] }],
cursor: { left: 10, top: 90 },
});
drawAll(u, plugin);
scan(u, plugin);
const bbox = plugin.cursor.points?.bbox;
// The 8px disc spans 6..14; the ring sits 3px out and is 2px wide.
expect(bbox?.(u, 1)).toStrictEqual({
left: 1,
top: 81,
width: 18,
height: 18,
});
expect(bbox?.(u, 2)).toMatchObject({ width: 0, height: 0 });
});
it('snaps the ring to whole pixels without moving its centre', () => {
const plugin = createScatterPlugin({
pointSize: { fixed: 7, min: 4, max: 20 },
});
const u = createFakePlot({
series: [{ xs: [10], ys: [10] }],
cursor: { left: 10, top: 90 },
});
drawAll(u, plugin);
scan(u, plugin);
// The 7px disc starts at 6.5, so the ring's edge would land on 1.5.
const ring = plugin.cursor.points?.bbox?.(u, 1);
expect(ring).toMatchObject({ left: 1, width: 18 });
expect((ring?.left ?? 0) + (ring?.width ?? 0) / 2).toBe(10);
});
it('draws the hover marker as an outline in the series colour', () => {
const plugin = createScatterPlugin();
const u = {
series: [{}, { stroke: (): string => '#E5484D' }, { stroke: '#30A46C' }],
} as unknown as uPlot;
const points = plugin.cursor.points as {
fill: (u: uPlot, seriesIdx: number) => string;
stroke: (u: uPlot, seriesIdx: number) => string;
width: (u: uPlot, seriesIdx: number, size: number) => number;
};
expect(points.fill(u, 1)).toBe('transparent');
expect(points.stroke(u, 1)).toBe('#E5484D');
expect(points.stroke(u, 2)).toBe('#30A46C');
expect(points.width(u, 1, 0)).toBe(2);
});
it('drawClear drops cached paths on data series only', () => {
const plugin = createScatterPlugin();
const u = createFakePlot({ series: [{ xs: [1], ys: [1] }] });
const [xSeries, dataSeries] = u.series as Array<{ _paths?: unknown }>;
xSeries._paths = 'x';
dataSeries._paths = 'cached';
plugin.hooks.drawClear(u);
expect(xSeries._paths).toBe('x');
expect(dataSeries._paths).toBeNull();
});
it('focus distance is zero, so the hit series wins focus', () => {
const plugin = createScatterPlugin();
expect(plugin.cursor.focus?.dist?.({} as uPlot, 1, 0, 0, 0)).toBe(0);
});
});
describe('applyScatterPlugin', () => {
it('switches the builder to faceted mode and disables drag selection', () => {
const builder = new UPlotConfigBuilder({ id: 'scatter' });
const plugin = createScatterPlugin();
applyScatterPlugin(builder, plugin);
const config = builder.getConfig();
expect(config.mode).toBe(2);
expect(config.cursor?.drag).toMatchObject({
x: false,
y: false,
setScale: false,
});
expect(config.hooks?.drawClear).toHaveLength(1);
expect(config.hooks?.destroy).toHaveLength(1);
});
it('facets read x and y against the shared scales', () => {
expect(SCATTER_FACETS).toStrictEqual([
{ scale: 'x', auto: true },
{ scale: 'y', auto: true },
]);
});
});

View File

@@ -0,0 +1,86 @@
import type uPlot from 'uplot';
import type { Quadtree } from '../../utils/quadtree';
import type { ScatterHit, ScatterPointSize, ScatterSeriesData } from './types';
export interface SizeDomain {
min: number;
max: number;
}
/** Size extent across all series, so equal values draw equal discs. */
export function resolveSizeDomain(data: uPlot.AlignedData): SizeDomain | null {
let min = Infinity;
let max = -Infinity;
for (let seriesIndex = 1; seriesIndex < data.length; seriesIndex++) {
const sizes = (data[seriesIndex] as unknown as ScatterSeriesData)[2];
if (!sizes) {
continue;
}
for (const size of sizes) {
if (size == null || !Number.isFinite(size)) {
continue;
}
min = Math.min(min, size);
max = Math.max(max, size);
}
}
return min <= max ? { min, max } : null;
}
/** Area, not diameter, scales with the value. */
export function resolvePointDiameter(
size: number | null | undefined,
domain: SizeDomain | null,
pointSize: ScatterPointSize,
): number {
if (size == null || domain == null || !Number.isFinite(size)) {
return pointSize.fixed;
}
if (domain.max === domain.min) {
return (pointSize.min + pointSize.max) / 2;
}
const t = Math.min(
1,
Math.max(0, (size - domain.min) / (domain.max - domain.min)),
);
const minArea = pointSize.min ** 2;
const maxArea = pointSize.max ** 2;
return Math.sqrt(minArea + (maxArea - minArea) * t);
}
/** Disc whose centre is closest, within `tolerance` px; `null` if none. */
export function resolveHit(
tree: Quadtree<ScatterHit>,
cx: number,
cy: number,
tolerance: number,
): ScatterHit | null {
let best: ScatterHit | null = null;
let bestDistance = Infinity;
tree.get(
cx - tolerance,
cy - tolerance,
tolerance * 2,
tolerance * 2,
(hit) => {
const left = hit.x - tolerance;
const top = hit.y - tolerance;
const right = hit.x + hit.w + tolerance;
const bottom = hit.y + hit.h + tolerance;
if (cx < left || cx > right || cy < top || cy > bottom) {
return;
}
const dx = cx - (hit.x + hit.w / 2);
const dy = cy - (hit.y + hit.h / 2);
const distance = dx * dx + dy * dy;
if (distance < bestDistance) {
best = hit;
bestDistance = distance;
}
},
);
return best;
}

View File

@@ -0,0 +1,225 @@
import uPlot, { Series } from 'uplot';
import { DEFAULT_FOCUS_PROXIMITY_VALUE } from '../../constants';
import { PlotMode } from '../../config/types';
import type { UPlotConfigBuilder } from '../../config/UPlotConfigBuilder';
import { Quadtree } from '../../utils/quadtree';
import {
resolveHit,
resolvePointDiameter,
resolveSizeDomain,
SizeDomain,
} from './geometry';
import {
DEFAULT_HOVER_TOLERANCE_PX,
DEFAULT_SCATTER_POINT_SIZE,
ScatterHit,
ScatterPluginOptions,
ScatterSeriesData,
} from './types';
export const SCATTER_FACETS: Series.Facet[] = [
{ scale: 'x', auto: true },
{ scale: 'y', auto: true },
];
const HIDDEN_BBOX: uPlot.BBox = { left: -10, top: -10, width: 0, height: 0 };
const TWO_PI = 2 * Math.PI;
/** CSS pixels between a hovered disc and the outline ring drawn around it. */
const HOVER_RING_GAP_PX = 3;
const HOVER_RING_WIDTH_PX = 2;
/** uPlot ceils a cursor point's left/top but not its size; snap so the ring stays centred. */
function snapToPixel(start: number, length: number): [number, number] {
const snapped = Math.floor(start);
return [snapped, length + (start - snapped) * 2];
}
export function getSeriesStroke(u: uPlot, seriesIdx: number): string {
const { stroke } = u.series[seriesIdx];
const color = typeof stroke === 'function' ? stroke(u, seriesIdx) : stroke;
return typeof color === 'string' ? color : '';
}
/** uPlot caches built paths on the series; the field is internal to it. */
type SeriesWithPaths = Series & { _paths?: Series.Paths | null };
export interface ScatterPlugin {
pathBuilder: Series.PathBuilder;
/** Hover by disc rather than by nearest x: mode 2 has no shared x to scan. */
cursor: uPlot.Cursor;
hooks: {
drawClear: (u: uPlot) => void;
destroy: (u: uPlot) => void;
};
getHit: () => ScatterHit | null;
}
export function createScatterPlugin({
pointSize = DEFAULT_SCATTER_POINT_SIZE,
hoverTolerance = DEFAULT_HOVER_TOLERANCE_PX,
}: ScatterPluginOptions = {}): ScatterPlugin {
let tree: Quadtree<ScatterHit> | null = null;
let hit: ScatterHit | null = null;
// Spans every series; cached per dataset.
let cachedData: uPlot.AlignedData | null = null;
let cachedDomain: SizeDomain | null = null;
function getSizeDomain(u: uPlot): SizeDomain | null {
if (cachedData !== u.data) {
cachedDomain = resolveSizeDomain(u.data);
cachedData = u.data;
}
return cachedDomain;
}
const pathBuilder: Series.PathBuilder = (u, seriesIdx, idx0, idx1) => {
const path = new Path2D();
const sizes = (u.data[seriesIdx] as unknown as ScatterSeriesData)[2];
const domain = getSizeDomain(u);
const { pxRatio } = uPlot;
uPlot.orient(
u,
seriesIdx,
(
_series,
dataX,
dataY,
scaleX,
scaleY,
valToPosX,
valToPosY,
xOff,
yOff,
xDim,
yDim,
_moveTo,
_lineTo,
_rect,
arc,
) => {
const xMin = scaleX.min ?? -Infinity;
const xMax = scaleX.max ?? Infinity;
const yMin = scaleY.min ?? -Infinity;
const yMax = scaleY.max ?? Infinity;
for (let i = idx0; i <= idx1; i++) {
const x = dataX[i];
const y = dataY[i];
if (
x == null ||
y == null ||
x < xMin ||
x > xMax ||
y < yMin ||
y > yMax
) {
continue;
}
const diameter =
resolvePointDiameter(sizes?.[i], domain, pointSize) * pxRatio;
const radius = diameter / 2;
const cx = valToPosX(x, scaleX, xDim, xOff);
const cy = valToPosY(y, scaleY, yDim, yOff);
path.moveTo(cx + radius, cy);
arc(path, cx, cy, radius, 0, TWO_PI);
tree?.add({
x: cx - radius - u.bbox.left,
y: cy - radius - u.bbox.top,
w: diameter,
h: diameter,
seriesIndex: seriesIdx,
dataIndex: i,
});
}
},
);
return { stroke: path, fill: path, clip: null };
};
const cursor: uPlot.Cursor = {
// Selection would set the dashboard time range; neither axis is time here.
drag: { x: false, y: false, setScale: false },
dataIdx: (u, seriesIdx): number | null => {
// uPlot asks series 1..n in order on every cursor move; resolve once.
if (seriesIdx === 1) {
const { left = -1, top = -1 } = u.cursor;
const { pxRatio } = uPlot;
hit =
tree && left >= 0 && top >= 0
? resolveHit(
tree,
left * pxRatio,
top * pxRatio,
hoverTolerance * pxRatio,
)
: null;
}
return hit?.seriesIndex === seriesIdx ? hit.dataIndex : null;
},
points: {
bbox: (_u, seriesIdx): uPlot.BBox => {
if (hit?.seriesIndex !== seriesIdx) {
return HIDDEN_BBOX;
}
const { pxRatio } = uPlot;
const inset = HOVER_RING_GAP_PX + HOVER_RING_WIDTH_PX;
const [left, width] = snapToPixel(
hit.x / pxRatio - inset,
hit.w / pxRatio + inset * 2,
);
const [top, height] = snapToPixel(
hit.y / pxRatio - inset,
hit.h / pxRatio + inset * 2,
);
return { left, top, width, height };
},
fill: (): string => 'transparent',
stroke: getSeriesStroke,
width: (): number => HOVER_RING_WIDTH_PX,
},
// uPlot only measures series that returned a data index, i.e. the hit one.
focus: { prox: DEFAULT_FOCUS_PROXIMITY_VALUE, dist: (): number => 0 },
};
return {
pathBuilder,
cursor,
hooks: {
drawClear: (u: uPlot): void => {
tree = new Quadtree<ScatterHit>(0, 0, u.bbox.width, u.bbox.height);
// The tree holds only what the path builder drew, so force a rebuild.
u.series.forEach((series, index) => {
if (index > 0) {
(series as SeriesWithPaths)._paths = null;
}
});
},
destroy: (): void => {
tree = null;
hit = null;
cachedData = null;
cachedDomain = null;
},
},
getHit: (): ScatterHit | null => hit,
};
}
export function applyScatterPlugin(
builder: UPlotConfigBuilder,
plugin: ScatterPlugin,
): void {
builder.setMode(PlotMode.Faceted);
builder.setCursor(plugin.cursor);
builder.addHook('drawClear', plugin.hooks.drawClear);
builder.addHook('destroy', plugin.hooks.destroy);
}

View File

@@ -0,0 +1,54 @@
import type { QuadtreeRect } from '../../utils/quadtree';
/** Diameters in CSS pixels. `min`/`max` bound the area scale when a size column is mapped. */
export interface ScatterPointSize {
fixed: number;
min: number;
max: number;
}
export const DEFAULT_SCATTER_POINT_SIZE: ScatterPointSize = {
fixed: 6,
min: 4,
max: 24,
};
/** CSS pixels around a point's disc that still register as a hover. */
export const DEFAULT_HOVER_TOLERANCE_PX = 3;
export interface ScatterPluginOptions {
pointSize?: ScatterPointSize;
hoverTolerance?: number;
}
/** Parallel columns; a `null` size draws at `fixed`. */
export type ScatterSeriesData = [
xs: number[],
ys: number[],
sizes?: Array<number | null>,
];
/** Mode-2 data: series 0 is uPlot's x placeholder and carries nothing. */
export type ScatterChartData = [null, ...ScatterSeriesData[]];
/** A drawn point's disc, in canvas pixels relative to the plot area. */
export interface ScatterHit extends QuadtreeRect {
seriesIndex: number;
dataIndex: number;
}
export interface ScatterChannel {
label: string;
unit?: string;
}
export interface ScatterChannels {
x: ScatterChannel;
y: ScatterChannel;
size?: ScatterChannel;
}
export interface ScatterPointLabel {
key: string;
value: string;
}

View File

@@ -0,0 +1,52 @@
import type uPlot from 'uplot';
import { buildClickData } from '../utils';
jest.mock('lib/uPlotLib/plugins/onClickPlugin', () => ({
getFocusedSeriesAtPosition: jest.fn(() => ({ seriesIndex: 1 })),
}));
const event = {
offsetX: 10,
offsetY: 20,
clientX: 110,
clientY: 220,
} as MouseEvent;
describe('buildClickData', () => {
it('skips the x-index lookups on a faceted plot, which has no shared x series', () => {
const posToIdx = jest.fn(() => {
throw new Error('faceted data[0] is null');
});
const plot = {
data: [null],
posToVal: (pos: number): number => pos * 2,
posToIdx,
} as unknown as uPlot;
expect(buildClickData(event, plot)).toStrictEqual({
xValue: 20,
yValue: 40,
focusedSeries: null,
clickedDataTimestamp: 20,
mouseX: 10,
mouseY: 20,
absoluteMouseX: 110,
absoluteMouseY: 220,
});
expect(posToIdx).not.toHaveBeenCalled();
});
it('resolves the focused series and clicked timestamp on an aligned plot', () => {
const plot = {
data: [[100, 200, 300]],
posToVal: (pos: number): number => pos,
posToIdx: (): number => 1,
} as unknown as uPlot;
expect(buildClickData(event, plot)).toMatchObject({
focusedSeries: { seriesIndex: 1 },
clickedDataTimestamp: 200,
});
});
});

View File

@@ -174,6 +174,24 @@ export function createLayoutObserver(
export function buildClickData(event: MouseEvent, plot: uPlot): ChartClickData {
const xValue = plot.posToVal(event.offsetX, 'x');
const yValue = plot.posToVal(event.offsetY, 'y');
const pointer = {
mouseX: event.offsetX,
mouseY: event.offsetY,
absoluteMouseX: event.clientX,
absoluteMouseY: event.clientY,
};
// Faceted (scatter): no shared x, so `posToIdx` would throw.
if (plot.data[0] == null) {
return {
xValue,
yValue,
focusedSeries: null,
clickedDataTimestamp: xValue,
...pointer,
};
}
const focusedSeries = getFocusedSeriesAtPosition(event, plot);
const dataIndex = plot.posToIdx(event.offsetX);
@@ -193,9 +211,6 @@ export function buildClickData(event: MouseEvent, plot: uPlot): ChartClickData {
yValue,
focusedSeries,
clickedDataTimestamp,
mouseX: event.offsetX,
mouseY: event.offsetY,
absoluteMouseX: event.clientX,
absoluteMouseY: event.clientY,
...pointer,
};
}

View File

@@ -0,0 +1,59 @@
import type uPlot from 'uplot';
import { logScaleSplits, spacedLogLabels } from '../logGridSplits';
describe('logScaleSplits', () => {
const splitsFor = (min: number, max: number): number[] =>
(
logScaleSplits as (
u: uPlot,
axisIdx: number,
min: number,
max: number,
) => number[]
)({} as uPlot, 0, min, max);
it('splits at 1, 2 and 5 × 10ⁿ inside an unsnapped range', () => {
expect(splitsFor(10.6, 990)).toStrictEqual([20, 50, 100, 200, 500]);
});
it('includes the ends when they fall on a split', () => {
expect(splitsFor(0.1, 10)).toStrictEqual([0.1, 0.2, 0.5, 1, 2, 5, 10]);
});
it('is empty for a range a log scale cannot hold', () => {
expect(splitsFor(0, 10)).toStrictEqual([]);
expect(splitsFor(5, 5)).toStrictEqual([]);
});
});
describe('spacedLogLabels', () => {
// 100px per decade.
const plot = {
axes: [{ scale: 'x' }],
valToPos: (value: number): number => Math.log10(value) * 100,
} as unknown as uPlot;
const label = (splits: number[], space: number): (number | null)[] =>
spacedLogLabels(plot, splits, 0, space, 0) as (number | null)[];
it('always labels the powers of ten', () => {
expect(label([10, 20, 50, 100], 200)).toStrictEqual([10, null, null, 100]);
});
it('adds a 2 or 5 that clears the labels already kept', () => {
// 20 is 30px from 10; 50 is 70px from 10 and 30px from 100.
expect(label([10, 20, 50, 100], 25)).toStrictEqual([10, 20, 50, 100]);
expect(label([10, 20, 50, 100], 40)).toStrictEqual([10, null, null, 100]);
});
it('labels both ends of a range spanning under a decade', () => {
// 100 sits at 200px; 20 (130px) and 500 (270px) clear it, 50 and 200 do not.
expect(label([20, 50, 100, 200, 500], 60)).toStrictEqual([
20,
null,
100,
null,
500,
]);
});
});

View File

@@ -0,0 +1,62 @@
import { Quadtree, QuadtreeRect } from '../quadtree';
interface Item extends QuadtreeRect {
id: number;
}
function collect(
tree: Quadtree<Item>,
x: number,
y: number,
w: number,
h: number,
): Set<number> {
const ids = new Set<number>();
tree.get(x, y, w, h, (item) => ids.add(item.id));
return ids;
}
describe('Quadtree', () => {
it('returns items in the queried region and not those far from it', () => {
const tree = new Quadtree<Item>(0, 0, 100, 100);
tree.add({ id: 1, x: 10, y: 10, w: 5, h: 5 });
tree.add({ id: 2, x: 80, y: 80, w: 5, h: 5 });
// Below the split threshold every item is visited; callers refine the hit.
expect(collect(tree, 9, 9, 8, 8)).toStrictEqual(new Set([1, 2]));
});
it('splits past the object limit and still finds every item', () => {
const tree = new Quadtree<Item>(0, 0, 100, 100);
const total = 50;
for (let id = 0; id < total; id++) {
tree.add({ id, x: (id % 10) * 10, y: Math.floor(id / 10) * 10, w: 4, h: 4 });
}
expect(collect(tree, 0, 0, 100, 100).size).toBe(total);
});
it('after a split, a query in one quadrant skips items confined to another', () => {
const tree = new Quadtree<Item>(0, 0, 100, 100);
for (let id = 0; id < 20; id++) {
// All in the north-west quadrant.
tree.add({ id, x: 1 + id, y: 1, w: 2, h: 2 });
}
tree.add({ id: 99, x: 90, y: 90, w: 2, h: 2 });
const northWest = collect(tree, 0, 0, 10, 10);
expect(northWest.has(99)).toBe(false);
expect(collect(tree, 85, 85, 10, 10).has(99)).toBe(true);
});
it('reports an item straddling the midline from either side', () => {
const tree = new Quadtree<Item>(0, 0, 100, 100);
for (let id = 0; id < 20; id++) {
tree.add({ id, x: 1, y: 1 + id, w: 2, h: 2 });
}
tree.add({ id: 99, x: 48, y: 48, w: 4, h: 4 });
expect(collect(tree, 40, 40, 5, 5).has(99)).toBe(true);
expect(collect(tree, 55, 55, 5, 5).has(99)).toBe(true);
});
});

View File

@@ -195,3 +195,22 @@ describe('scale utils', () => {
});
});
});
describe('symmetric log scale', () => {
it('maps to uPlot arcsinh with the given linear threshold', () => {
expect(
scaleUtils.getDistributionConfig({
time: false,
distr: DistributionType.SymmetricLog,
asinhThreshold: 0.01,
}),
).toStrictEqual({ distr: 4, log: 10, asinh: 0.01 });
expect(
scaleUtils.getDistributionConfig({
time: false,
distr: DistributionType.SymmetricLog,
}).asinh,
).toBe(scaleUtils.DEFAULT_ASINH_THRESHOLD);
});
});

View File

@@ -0,0 +1,61 @@
import type uPlot from 'uplot';
const DECADE_TOLERANCE = 1e-9;
function isDecade(value: number): boolean {
if (value === 0) {
return true;
}
const exponent = Math.log10(Math.abs(value));
return Math.abs(exponent - Math.round(exponent)) < DECADE_TOLERANCE;
}
const LOG_SPLIT_MULTIPLES = [1, 2, 5];
/** 1, 2, 5 × 10ⁿ in range; uPlot's log splits step from an unsnapped min (10.79, 21.58…). */
export const logScaleSplits: uPlot.Axis.Splits = (_u, _axisIdx, min, max) => {
if (!(min > 0) || !(max > min)) {
return [];
}
const splits: number[] = [];
for (
let exponent = Math.floor(Math.log10(min));
exponent <= Math.ceil(Math.log10(max));
exponent++
) {
LOG_SPLIT_MULTIPLES.forEach((multiple) => {
// Rounded through toPrecision so 0.1 × 3 noise can't fall outside [min, max].
const split = Number((multiple * 10 ** exponent).toPrecision(12));
if (split >= min && split <= max) {
splits.push(split);
}
});
}
return splits;
};
/** Labels every decade, plus any 2 or 5 at least `space` px clear. */
export const spacedLogLabels: uPlot.Axis.Filter = (
u,
splits,
axisIdx,
space,
) => {
const scaleKey = u.axes[axisIdx]?.scale ?? 'x';
const positions = splits.map((split) =>
split == null ? null : u.valToPos(split, scaleKey),
);
const kept = splits.map((split) => split != null && isDecade(split));
const isClear = (index: number): boolean =>
kept.every(
(isKept, other) =>
!isKept ||
Math.abs((positions[other] ?? 0) - (positions[index] ?? 0)) >= space,
);
splits.forEach((split, index) => {
if (split != null && !kept[index] && isClear(index)) {
kept[index] = true;
}
});
return splits.map((split, index) => (kept[index] ? split : null));
};

View File

@@ -0,0 +1,100 @@
export interface QuadtreeRect {
x: number;
y: number;
w: number;
h: number;
}
const MAX_OBJECTS = 10;
const MAX_LEVELS = 4;
/** Rect index for cursor hit-testing; items straddling quadrants are stored in each, so `get` may repeat them. */
export class Quadtree<T extends QuadtreeRect = QuadtreeRect> {
private items: T[] = [];
private quadrants: Quadtree<T>[] | null = null;
constructor(
private readonly x: number,
private readonly y: number,
private readonly w: number,
private readonly h: number,
private readonly level = 0,
) {}
add(item: T): void {
if (this.quadrants) {
this.forEachQuadrant(item, (quadrant) => quadrant.add(item));
return;
}
this.items.push(item);
if (this.items.length > MAX_OBJECTS && this.level < MAX_LEVELS) {
this.split();
const items = this.items;
this.items = [];
for (const existing of items) {
this.forEachQuadrant(existing, (quadrant) => quadrant.add(existing));
}
}
}
/** Visits every item whose quadrant overlaps the rectangle; callers refine the test. */
get(
x: number,
y: number,
w: number,
h: number,
visit: (item: T) => void,
): void {
for (const item of this.items) {
visit(item);
}
if (this.quadrants) {
this.forEachQuadrant({ x, y, w, h }, (quadrant) =>
quadrant.get(x, y, w, h, visit),
);
}
}
private split(): void {
const w = this.w / 2;
const h = this.h / 2;
const level = this.level + 1;
// North-east, north-west, south-west, south-east.
this.quadrants = [
new Quadtree<T>(this.x + w, this.y, w, h, level),
new Quadtree<T>(this.x, this.y, w, h, level),
new Quadtree<T>(this.x, this.y + h, w, h, level),
new Quadtree<T>(this.x + w, this.y + h, w, h, level),
];
}
private forEachQuadrant(
rect: QuadtreeRect,
visit: (quadrant: Quadtree<T>) => void,
): void {
if (!this.quadrants) {
return;
}
const midX = this.x + this.w / 2;
const midY = this.y + this.h / 2;
const startsNorth = rect.y < midY;
const startsWest = rect.x < midX;
const endsEast = rect.x + rect.w > midX;
const endsSouth = rect.y + rect.h > midY;
if (startsNorth && endsEast) {
visit(this.quadrants[0]);
}
if (startsWest && startsNorth) {
visit(this.quadrants[1]);
}
if (startsWest && endsSouth) {
visit(this.quadrants[2]);
}
if (endsEast && endsSouth) {
visit(this.quadrants[3]);
}
}
}

View File

@@ -58,18 +58,23 @@ function normalizeLogLimit(
return logBase ** exp;
}
export const DEFAULT_ASINH_THRESHOLD = 1;
/**
* Returns uPlot scale distribution options for the Y axis.
* Time (X) scale gets no distr/log; Y scale gets distr 1 (linear) or 3 (log) and log base 2 or 10.
* Returns uPlot scale distribution options for a value axis.
* Time scales get no distr/log; value scales get distr 1 (linear), 3 (log) or
* 4 (arcsinh, uPlot's symmetric log) and log base 2 or 10.
*/
export function getDistributionConfig({
time,
distr,
logBase,
asinhThreshold,
}: {
time: ScaleProps['time'];
distr?: DistributionType;
logBase?: number;
asinhThreshold?: number;
}): Partial<Scale> {
if (time) {
return {};
@@ -77,6 +82,14 @@ export function getDistributionConfig({
const resolvedLogBase = (logBase ?? 10) === 2 ? 2 : 10;
if (distr === DistributionType.SymmetricLog) {
return {
distr: 4,
log: resolvedLogBase,
asinh: asinhThreshold ?? DEFAULT_ASINH_THRESHOLD,
};
}
return {
distr: distr === DistributionType.Logarithmic ? 3 : 1,
log: resolvedLogBase,

View File

@@ -37,6 +37,7 @@ export default function ChartWrapper({
syncKey,
syncFilterMode,
onDestroy = noop,
plotRef,
children,
layoutChildren,
yAxisUnit,
@@ -123,6 +124,7 @@ export default function ChartWrapper({
height={chartHeight}
plotRef={(plot): void => {
plotInstanceRef.current = plot;
plotRef?.(plot);
}}
onDestroy={(plot: uPlot): void => {
plotInstanceRef.current = null;

View File

@@ -0,0 +1,45 @@
import { render } from 'tests/test-utils';
import type uPlot from 'uplot';
import { LegendPosition } from 'lib/uPlotV2/components/types';
import { UPlotConfigBuilder } from 'lib/uPlotV2/config/UPlotConfigBuilder';
import ChartWrapper from '../ChartWrapper';
const FAKE_PLOT = { id: 'plot' } as unknown as uPlot;
jest.mock('lib/uPlotV2/components/UPlotChart/UPlotChart', () => ({
__esModule: true,
default: ({ plotRef }: { plotRef?: (plot: uPlot | null) => void }): null => {
plotRef?.(FAKE_PLOT);
return null;
},
}));
window.ResizeObserver =
window.ResizeObserver ||
jest.fn().mockImplementation(() => ({
disconnect: jest.fn(),
observe: jest.fn(),
unobserve: jest.fn(),
}));
describe('ChartWrapper', () => {
it('hands the plot instance to the caller', () => {
const plotRef = jest.fn();
render(
<ChartWrapper
config={new UPlotConfigBuilder({ id: 'chart' })}
data={[[1], [2]]}
width={400}
height={300}
legendConfig={{ position: LegendPosition.BOTTOM }}
showLegend={false}
showTooltip={false}
plotRef={plotRef}
/>,
);
expect(plotRef).toHaveBeenCalledWith(FAKE_PLOT);
});
});

View File

@@ -0,0 +1,244 @@
import { useMemo } from 'react';
import type { Meta, StoryObj } from '@storybook/react-vite';
import { useIsDarkMode } from 'hooks/useDarkMode';
import { LegendPosition } from 'lib/uPlotV2/components/types';
import type { Threshold } from 'lib/uPlotV2/hooks/types';
import type { ScatterPointLabel } from 'lib/uPlotV2/plugins/ScatterPlugin/types';
import Scatter from './Scatter';
import {
buildScatterConfig,
prepareScatterChartData,
ScatterAxisScale,
ScatterSeries,
} from './utils';
const SERVICES = [
'frontend',
'cart',
'checkout',
'payment',
'shipping',
'currency',
'email',
'recommendation',
'ads',
'product-catalog',
];
type Shape = 'spread' | 'single' | 'sameX';
interface ScatterStoryProps {
groups: number;
pointsPerGroup: number;
/** Adds an error-count size column. */
sized: boolean;
xScale: ScatterAxisScale;
yScale: ScatterAxisScale;
/** Zeroes a share of y values, which forces the symmetric log. */
withZeros: boolean;
shape: Shape;
thresholds: boolean;
pointSize: number;
/** 0–1. */
fillOpacity: number;
width: number;
height: number;
}
/** Deterministic, so a story renders the same points on every run. */
function createRng(seed: number): () => number {
let state = seed >>> 0;
return (): number => {
state = (state * 1664525 + 1013904223) >>> 0;
return state / 2 ** 32;
};
}
function buildSeries({
groups,
pointsPerGroup,
sized,
withZeros,
shape,
}: ScatterStoryProps): ScatterSeries[] {
const rng = createRng(42);
return Array.from({ length: groups }, (_, groupIndex) => {
const label = SERVICES[groupIndex % SERVICES.length];
// Each service sits in its own throughput/latency band, so groups are telling
// apart rather than one cloud.
const baseThroughput = 20 * 2 ** (groupIndex % 5);
const baseLatency = 40 + 60 * (groupIndex % 4);
const count = shape === 'single' ? 1 : pointsPerGroup;
const xs: number[] = [];
const ys: number[] = [];
const sizes: Array<number | null> = [];
for (let i = 0; i < count; i++) {
const throughput =
shape === 'sameX' ? baseThroughput : baseThroughput * (0.5 + rng() * 1.5);
// Latency grows with load, plus noise; the odd outlier keeps the axis honest.
const outlier = rng() < 0.03 ? 4 + rng() * 6 : 1;
let latency =
baseLatency *
(0.8 + (throughput / baseThroughput) * 0.4 + rng() * 0.3) *
outlier;
if (withZeros && rng() < 0.2) {
latency = 0;
}
xs.push(Number(throughput.toFixed(2)));
ys.push(Number(latency.toFixed(2)));
sizes.push(rng() < 0.1 ? null : Math.round(rng() * rng() * 500));
}
return sized ? { label, xs, ys, sizes } : { label, xs, ys };
});
}
const THRESHOLDS: Threshold[] = [
{
thresholdValue: 300,
thresholdUnit: 'ms',
thresholdColor: '#E5484D',
thresholdLabel: 'p99 SLO',
},
];
function ScatterStory(props: ScatterStoryProps): JSX.Element {
const {
xScale,
yScale,
sized,
thresholds,
pointSize,
fillOpacity,
width,
height,
} = props;
const isDarkMode = useIsDarkMode();
const series = useMemo(() => buildSeries(props), [props]);
const config = useMemo(
() =>
buildScatterConfig({
id: 'scatter-story',
series,
isDarkMode,
x: { unit: 'reqps', scale: xScale },
y: { unit: 'ms', scale: yScale },
pointSize: { fixed: pointSize, min: 4, max: pointSize * 4 },
fillOpacity,
thresholds: thresholds ? THRESHOLDS : undefined,
}),
[series, isDarkMode, xScale, yScale, pointSize, fillOpacity, thresholds],
);
const data = useMemo(() => prepareScatterChartData(series), [series]);
const resolvePointLabels = (
seriesIndex: number,
dataIndex: number,
): ScatterPointLabel[] => [
{ key: 'service.name', value: series[seriesIndex - 1]?.label ?? '' },
{
key: 'k8s.pod.name',
value: `pod-${dataIndex.toString().padStart(3, '0')}`,
},
];
return (
<div style={{ width, padding: 16 }}>
<Scatter
config={config}
data={data}
width={width}
height={height}
legendConfig={{ position: LegendPosition.BOTTOM }}
channels={{
x: { label: 'Throughput', unit: 'reqps' },
y: { label: 'p99 latency', unit: 'ms' },
...(sized && { size: { label: 'Errors', unit: 'short' } }),
}}
resolvePointLabels={resolvePointLabels}
canPinTooltip
/>
</div>
);
}
const meta = {
title: 'Charts/Scatter',
component: ScatterStory,
parameters: { layout: 'padded' },
args: {
groups: 1,
pointsPerGroup: 10,
sized: false,
xScale: ScatterAxisScale.Linear,
yScale: ScatterAxisScale.Linear,
withZeros: false,
shape: 'spread',
thresholds: false,
pointSize: 6,
fillOpacity: 0.7,
width: 800,
height: 420,
},
argTypes: {
shape: { control: 'radio', options: ['spread', 'single', 'sameX'] },
xScale: { control: 'radio', options: Object.values(ScatterAxisScale) },
yScale: { control: 'radio', options: Object.values(ScatterAxisScale) },
fillOpacity: { control: { type: 'range', min: 0, max: 1, step: 0.05 } },
pointSize: { control: { type: 'range', min: 2, max: 16, step: 1 } },
},
} satisfies Meta<ScatterStoryProps>;
export default meta;
type Story = StoryObj<ScatterStoryProps>;
/** One service, ten points: axes formatted with units, hover picks the right point. */
export const Basic: Story = {};
/** Five services, one legend entry each; toggling a row hides its points. */
export const Grouped: Story = {
args: { groups: 5, pointsPerGroup: 40 },
};
/** Error count as disc area, between the configured min and max diameters. */
export const Sized: Story = {
args: { groups: 5, pointsPerGroup: 40, sized: true, pointSize: 5 },
};
/** Log x; a fifth of the latencies are 0, so y falls back to the symmetric log. */
export const LogAxes: Story = {
args: {
groups: 5,
pointsPerGroup: 60,
xScale: ScatterAxisScale.Log,
yScale: ScatterAxisScale.Log,
withZeros: true,
},
};
/** A single point still gets a padded range rather than an empty plot. */
export const SinglePoint: Story = {
args: { shape: 'single' },
};
/** Fifty points sharing one x collide on nothing: no shared x array to align. */
export const SameX: Story = {
args: { groups: 3, pointsPerGroup: 50, shape: 'sameX' },
};
/** Horizontal line with label on the y axis; the scale stretches to include it. */
export const Thresholds: Story = {
args: { groups: 3, pointsPerGroup: 40, thresholds: true },
};
/** Perf harness: raise `pointsPerGroup` and read the draw time under the chart. */
export const Dense: Story = {
args: { groups: 5, pointsPerGroup: 1000, pointSize: 4, fillOpacity: 0.5 },
};

View File

@@ -0,0 +1,98 @@
import { useCallback, useRef } from 'react';
import ChartWrapper from 'lib/visualization/charts/ChartWrapper/ChartWrapper';
import ScatterTooltip from 'lib/uPlotV2/components/Tooltip/ScatterTooltip';
import {
ScatterTooltipProps,
TooltipRenderArgs,
} from 'lib/uPlotV2/components/types';
import { getSeriesStroke } from 'lib/uPlotV2/plugins/ScatterPlugin/scatterPlugin';
import type { ChartClickData } from 'lib/uPlotV2/plugins/TooltipPlugin/types';
import { ScatterChartProps } from 'lib/visualization/charts/types';
import uPlot from 'uplot';
import { getCursorHit } from './utils';
// Faceted uPlot can't mount without series; empty aligned data shows no-data instead.
const EMPTY_ALIGNED_DATA: uPlot.AlignedData = [[]];
export default function Scatter(props: ScatterChartProps): JSX.Element {
const {
children,
customTooltip,
channels,
resolvePointLabels,
pinnedTooltipElement,
onPointClick,
plotRef,
...rest
} = props;
const plotInstanceRef = useRef<uPlot | null>(null);
const handlePlotRef = useCallback(
(plot: uPlot | null): void => {
plotInstanceRef.current = plot;
plotRef?.(plot);
},
[plotRef],
);
// Faceted plots have no shared x; take the hit from the cursor.
const handleClick = useCallback(
(click: ChartClickData): void => {
const plot = plotInstanceRef.current;
const hit = plot ? getCursorHit(plot) : null;
if (!plot || !hit || !onPointClick) {
return;
}
onPointClick({
...hit,
color: getSeriesStroke(plot, hit.seriesIndex),
coordinates: { x: click.absoluteMouseX, y: click.absoluteMouseY },
});
},
[onPointClick],
);
const renderTooltip = useCallback(
(args: TooltipRenderArgs): React.ReactNode => {
if (customTooltip) {
return customTooltip(args);
}
const tooltipProps: ScatterTooltipProps = {
...args,
id: rest.config.getId(),
channels,
resolvePointLabels,
decimalPrecision: rest.decimalPrecision,
canPinTooltip: rest.canPinTooltip,
renderTooltipFooter: rest.renderTooltipFooter,
};
return <ScatterTooltip {...tooltipProps} />;
},
[
customTooltip,
channels,
resolvePointLabels,
rest.config,
rest.decimalPrecision,
rest.canPinTooltip,
rest.renderTooltipFooter,
],
);
const hasSeries = rest.data.length > 1;
return (
<ChartWrapper
{...rest}
data={hasSeries ? rest.data : EMPTY_ALIGNED_DATA}
plotRef={handlePlotRef}
onClick={onPointClick ? handleClick : rest.onClick}
customTooltip={renderTooltip}
pinnedTooltipElement={pinnedTooltipElement}
>
{children}
</ChartWrapper>
);
}

View File

@@ -0,0 +1,97 @@
import { DistributionType } from 'lib/uPlotV2/config/types';
import uPlot from 'uplot';
import { createScatterRange, type ScatterRangeOptions } from '../scatterRange';
/** A 220px-tall, 420px-wide plot at a device pixel ratio of 1. */
const plot = { bbox: { width: 420, height: 220 } } as unknown as uPlot;
function rangeOf(
options: Partial<ScatterRangeOptions>,
dataMin: number | null,
dataMax: number | null,
): [number, number] {
const range = createScatterRange({
dimension: 'y',
marginPx: 10,
distribution: DistributionType.Linear,
...options,
}) as (u: uPlot, min: number | null, max: number | null) => [number, number];
return range(plot, dataMin, dataMax);
}
describe('createScatterRange', () => {
beforeAll(() => {
Object.defineProperty(uPlot, 'pxRatio', { value: 1, configurable: true });
});
it('widens a linear range so the data sits marginPx inside each edge', () => {
// 200px of data inside 220px: 10px at each end is 1/20 of the span.
expect(rangeOf({}, 0, 100)).toStrictEqual([-5, 105]);
});
it('measures the margin along its own axis', () => {
// 400px of data inside 420px.
expect(rangeOf({ dimension: 'x' }, 0, 100)).toStrictEqual([-2.5, 102.5]);
});
it('pads a log range in decades, without snapping to the next power of ten', () => {
const [min, max] = rangeOf(
{ distribution: DistributionType.Logarithmic },
1,
10,
);
expect(Math.log10(min)).toBeCloseTo(-0.05);
expect(Math.log10(max)).toBeCloseTo(1.05);
});
it('pads a symmetric log range in its asinh space', () => {
const [min, max] = rangeOf(
{ distribution: DistributionType.SymmetricLog, asinhThreshold: 2 },
0,
20,
);
const span = Math.asinh(10);
expect(Math.asinh(min / 2)).toBeCloseTo(-span / 20);
expect(Math.asinh(max / 2)).toBeCloseTo(span + span / 20);
});
it('stretches to the soft limits before padding', () => {
expect(rangeOf({ softMin: 0, softMax: 200 }, 50, 100)).toStrictEqual([
-10, 210,
]);
});
it('ignores a soft min at or below zero on a log scale', () => {
const [min] = rangeOf(
{ distribution: DistributionType.Logarithmic, softMin: 0 },
10,
100,
);
expect(min).toBeGreaterThan(0);
});
it('spreads a single value around itself', () => {
const [min, max] = rangeOf({}, 5, 5);
expect(min).toBeLessThan(5);
expect(max).toBeGreaterThan(5);
expect(5 - min).toBeCloseTo(max - 5);
});
it('falls back to a unit range with no data', () => {
expect(rangeOf({}, null, null)).toStrictEqual([0, 1]);
expect(
rangeOf({ distribution: DistributionType.Logarithmic }, null, null),
).toStrictEqual([1, 10]);
});
it('skips the margin on a plot too small to hold it', () => {
const tiny = { bbox: { width: 15, height: 15 } } as unknown as uPlot;
const range = createScatterRange({
dimension: 'y',
marginPx: 10,
distribution: DistributionType.Linear,
}) as (u: uPlot, min: number, max: number) => [number, number];
expect(range(tiny, 0, 100)).toStrictEqual([0, 100]);
});
});

View File

@@ -0,0 +1,243 @@
import type uPlot from 'uplot';
import { DistributionType } from 'lib/uPlotV2/config/types';
import {
buildScatterConfig,
getCursorHit,
prepareScatterChartData,
resolveAxisDistribution,
ScatterAxisScale,
ScatterSeries,
} from '../utils';
jest.mock('lib/visualization/panels/utils/legendVisibilityUtils', () => ({
getStoredSeriesVisibility: jest.fn(),
}));
const SERIES: ScatterSeries[] = [
{ label: 'cart', xs: [10, 20], ys: [100, 200], sizes: [1, null] },
{ label: 'checkout', xs: [30], ys: [0] },
];
describe('prepareScatterChartData', () => {
it('lays series out as facets behind an empty x slot', () => {
expect(prepareScatterChartData(SERIES)).toStrictEqual([
null,
[
[10, 20],
[100, 200],
[1, null],
],
[[30], [0]],
]);
});
});
describe('getCursorHit', () => {
const plotWith = (idxs: Array<number | null>): uPlot =>
({ cursor: { idxs } }) as unknown as uPlot;
it('reads the series the scatter plugin answered for', () => {
expect(getCursorHit(plotWith([null, null, 4, null]))).toStrictEqual({
seriesIndex: 2,
dataIndex: 4,
});
});
it('takes a data index of 0 as a hit', () => {
expect(getCursorHit(plotWith([0, 0]))).toStrictEqual({
seriesIndex: 1,
dataIndex: 0,
});
});
it('is null off every dot', () => {
expect(getCursorHit(plotWith([null, null]))).toBeNull();
expect(getCursorHit({ cursor: {} } as unknown as uPlot)).toBeNull();
});
});
describe('resolveAxisDistribution', () => {
it('is linear when asked, whatever the values', () => {
expect(
resolveAxisDistribution([1, 1e6], ScatterAxisScale.Linear),
).toStrictEqual({ distribution: DistributionType.Linear });
});
it('is a plain log when every value is positive', () => {
expect(resolveAxisDistribution([1, 100], ScatterAxisScale.Log)).toStrictEqual(
{ distribution: DistributionType.Logarithmic },
);
});
it('falls back from log to a symmetric log around the smallest magnitude when zero is present', () => {
expect(
resolveAxisDistribution([0, 0.05, 300], ScatterAxisScale.Log),
).toStrictEqual({
distribution: DistributionType.SymmetricLog,
asinhThreshold: 0.01,
});
});
it('keeps an explicit log axis for a soft min it cannot place', () => {
expect(
resolveAxisDistribution([1, 100], ScatterAxisScale.Log, [-10]),
).toStrictEqual({ distribution: DistributionType.Logarithmic });
});
it('is a symmetric log when asked, even with only positive values', () => {
expect(
resolveAxisDistribution([2, 300], ScatterAxisScale.SymLog),
).toStrictEqual({
distribution: DistributionType.SymmetricLog,
asinhThreshold: 1,
});
});
it('uses a unit threshold when nothing is positive', () => {
expect(
resolveAxisDistribution([0, -5], ScatterAxisScale.SymLog),
).toStrictEqual({
distribution: DistributionType.SymmetricLog,
asinhThreshold: 1,
});
});
describe('auto', () => {
const SKEWED = [1, 2, 3, 5, 8, 10, 20, 50, 1000];
it.each([
['skewed positive values', SKEWED, DistributionType.Logarithmic],
[
'values spread evenly across decades',
[1, 250, 500, 750, 1000],
DistributionType.Linear,
],
[
'a single outlier among clustered values',
[0.001, 40, 45, 50, 55, 60],
DistributionType.Linear,
],
[
'skewed values spanning under two decades',
[10, 10, 10, 11, 50],
DistributionType.Linear,
],
['no finite values', [Number.NaN], DistributionType.Linear],
])('picks a scale for %s', (_, values, distribution) => {
expect(resolveAxisDistribution(values, ScatterAxisScale.Auto)).toStrictEqual(
{ distribution },
);
});
it('picks symlog for skewed values that include zero', () => {
expect(
resolveAxisDistribution(
[0, 0, 1, 2, 5, 10, 50, 100, 10_000],
ScatterAxisScale.Auto,
),
).toStrictEqual({
distribution: DistributionType.SymmetricLog,
asinhThreshold: 1,
});
});
it('picks symlog when a soft limit reaches zero or below', () => {
expect(
resolveAxisDistribution(SKEWED, ScatterAxisScale.Auto, [0]),
).toStrictEqual({
distribution: DistributionType.SymmetricLog,
asinhThreshold: 1,
});
});
it('counts a soft limit towards the decades it spans', () => {
expect(
resolveAxisDistribution([5, 50], ScatterAxisScale.Auto, [null, 5000]),
).toStrictEqual({ distribution: DistributionType.Logarithmic });
});
it('is the default', () => {
expect(resolveAxisDistribution(SKEWED)).toStrictEqual({
distribution: DistributionType.Logarithmic,
});
});
});
});
describe('buildScatterConfig', () => {
const build = (
overrides: Partial<Parameters<typeof buildScatterConfig>[0]> = {},
): ReturnType<typeof buildScatterConfig> =>
buildScatterConfig({
id: 'scatter',
series: SERIES,
isDarkMode: true,
x: { unit: 'reqps' },
y: { unit: 'ms', scale: ScatterAxisScale.Log },
...overrides,
});
it('draws a line along both plot edges', () => {
const [xAxis, yAxis] = build().getConfig().axes ?? [];
expect(xAxis?.border?.show).toBe(true);
expect(yAxis?.border?.show).toBe(true);
});
it('titles each axis with its label, and leaves an unlabelled one bare', () => {
const config = build({
x: { unit: 'reqps', label: 'Throughput' },
y: { unit: 'ms', label: '' },
}).getConfig();
const [xAxis, yAxis] = config.axes ?? [];
expect(xAxis?.label).toBe('Throughput');
expect(yAxis?.label).toBeUndefined();
});
it('emits a faceted plot with two value scales', () => {
const config = build().getConfig();
expect(config.mode).toBe(2);
expect(config.scales?.x).toMatchObject({ time: false, distr: 1 });
// The y column has a 0, so log becomes the symmetric variant.
expect(config.scales?.y).toMatchObject({ time: false, distr: 4 });
});
it('draws one faceted series per group with the plugin path builder', () => {
const config = build().getConfig();
const [, cart, checkout] = config.series ?? [];
expect(config.series).toHaveLength(3);
expect(cart).toMatchObject({
label: 'cart',
facets: [
{ scale: 'x', auto: true },
{ scale: 'y', auto: true },
],
});
expect(typeof cart?.paths).toBe('function');
expect(cart?.paths).toBe(checkout?.paths);
expect(cart?.points?.show).toBe(false);
});
it('formats both axes with their units', () => {
const config = build().getConfig();
const [xAxis, yAxis] = config.axes ?? [];
expect(xAxis).toMatchObject({ scale: 'x', side: 2, space: 90 });
expect(yAxis).toMatchObject({ scale: 'y', side: 3 });
expect(typeof xAxis?.values).toBe('function');
expect(typeof yAxis?.values).toBe('function');
});
it('registers a y threshold draw hook when thresholds are given', () => {
const config = build({
thresholds: [{ thresholdValue: 300, thresholdUnit: 'ms' }],
}).getConfig();
expect(config.hooks?.draw).toHaveLength(1);
expect(build().getConfig().hooks?.draw).toBeUndefined();
});
});

View File

@@ -0,0 +1,101 @@
import { DistributionType } from 'lib/uPlotV2/config/types';
import { DEFAULT_ASINH_THRESHOLD } from 'lib/uPlotV2/utils/scale';
import uPlot from 'uplot';
import type { AxisDistribution } from './utils';
interface ScaleTransform {
forward: (value: number) => number;
inverse: (position: number) => number;
}
export function getScaleTransform({
distribution,
asinhThreshold = DEFAULT_ASINH_THRESHOLD,
}: AxisDistribution): ScaleTransform {
switch (distribution) {
case DistributionType.Logarithmic:
return {
forward: (value): number => Math.log10(value),
inverse: (position): number => 10 ** position,
};
case DistributionType.SymmetricLog:
return {
forward: (value): number => Math.asinh(value / asinhThreshold),
inverse: (position): number => Math.sinh(position) * asinhThreshold,
};
default:
return {
forward: (value): number => value,
inverse: (position): number => position,
};
}
}
/** In transformed units. */
const SINGLE_VALUE_HALF_SPAN = 0.5;
/** The data's extent stretched to the soft limits; null with neither. */
function resolveExtent(
dataMin: number | null,
dataMax: number | null,
softMin: number | null | undefined,
softMax: number | null | undefined,
): [number, number] | null {
const lows = [dataMin, softMin].filter(
(value): value is number => value != null && Number.isFinite(value),
);
const highs = [dataMax, softMax].filter(
(value): value is number => value != null && Number.isFinite(value),
);
return lows.length > 0 && highs.length > 0
? [Math.min(...lows), Math.max(...highs)]
: null;
}
export interface ScatterRangeOptions extends AxisDistribution {
dimension: 'x' | 'y';
/** CSS pixels kept clear at each end so the largest dot draws whole. */
marginPx: number;
softMin?: number | null;
softMax?: number | null;
}
/** Data extent padded by `marginPx` so edge dots draw whole; log stays unsnapped (snapping after the margin can add an empty decade). */
export function createScatterRange({
dimension,
marginPx,
softMin,
softMax,
...distribution
}: ScatterRangeOptions): uPlot.Scale.Range {
const transform = getScaleTransform(distribution);
const isLog = distribution.distribution === DistributionType.Logarithmic;
// A log scale cannot place a soft limit at or below zero.
const usableSoftMin = isLog && (softMin ?? 0) <= 0 ? null : softMin;
return (u, dataMin, dataMax): uPlot.Range.MinMax => {
const extent = resolveExtent(dataMin, dataMax, usableSoftMin, softMax);
if (!extent) {
return isLog ? [1, 10] : [0, 1];
}
let start = transform.forward(extent[0]);
let end = transform.forward(extent[1]);
if (start === end) {
start -= SINGLE_VALUE_HALF_SPAN;
end += SINGLE_VALUE_HALF_SPAN;
}
const plotPx =
(dimension === 'x' ? u.bbox.width : u.bbox.height) / uPlot.pxRatio;
if (plotPx > 2 * marginPx) {
const pad = ((end - start) * marginPx) / (plotPx - 2 * marginPx);
start -= pad;
end += pad;
}
return [transform.inverse(start), transform.inverse(end)];
};
}

View File

@@ -0,0 +1,366 @@
import { PrecisionOption } from 'components/Graph/types';
import {
type AxisProps,
DistributionType,
DrawStyle,
SelectionPreferencesSource,
} from 'lib/uPlotV2/config/types';
import { UPlotConfigBuilder } from 'lib/uPlotV2/config/UPlotConfigBuilder';
import { Threshold } from 'lib/uPlotV2/hooks/types';
import {
applyScatterPlugin,
createScatterPlugin,
SCATTER_FACETS,
} from 'lib/uPlotV2/plugins/ScatterPlugin/scatterPlugin';
import {
DEFAULT_SCATTER_POINT_SIZE,
ScatterChartData,
ScatterPointSize,
ScatterSeriesData,
} from 'lib/uPlotV2/plugins/ScatterPlugin/types';
import {
logScaleSplits,
spacedLogLabels,
} from 'lib/uPlotV2/utils/logGridSplits';
import { adjustSoftLimitsWithThresholds } from 'lib/uPlotV2/utils/scale';
import uPlot from 'uplot';
import { createScatterRange, getScaleTransform } from './scatterRange';
const POINT_STROKE_WIDTH = 1;
/** Unit-suffixed x labels are wider than uPlot's 50px default assumes. */
const X_AXIS_TICK_SPACE_PX = 90;
const X_AXIS_END_LABEL_PADDING_PX = 40;
export interface ScatterSeries {
label: string;
xs: number[];
ys: number[];
/** Optional third channel, in the caller's units. */
sizes?: Array<number | null>;
}
export enum ScatterAxisScale {
/** Log or symlog when it spreads the points out more than linear does. */
Auto = 'auto',
Linear = 'linear',
Log = 'log',
/** Log-like, but places zero and negatives. */
SymLog = 'symlog',
}
export interface ScatterAxisOptions {
label?: string;
unit?: string;
softMin?: number | null;
softMax?: number | null;
scale?: ScatterAxisScale;
}
export interface BuildScatterConfigArgs {
id: string;
series: ScatterSeries[];
isDarkMode: boolean;
x: ScatterAxisOptions;
y: ScatterAxisOptions;
pointSize?: ScatterPointSize;
/** 0–1. */
fillOpacity?: number;
colorMapping?: Record<string, string>;
/** Drawn on the y axis. */
thresholds?: Threshold[];
decimalPrecision?: PrecisionOption;
selectionPreferencesSource?: SelectionPreferencesSource;
shouldSaveSelectionPreference?: boolean;
}
/** The plugin's `dataIdx` answers only for the hit series. */
export function getCursorHit(
plot: uPlot,
): { seriesIndex: number; dataIndex: number } | null {
const idxs = plot.cursor.idxs ?? [];
for (let seriesIndex = 1; seriesIndex < idxs.length; seriesIndex++) {
const dataIndex = idxs[seriesIndex];
if (dataIndex != null) {
return { seriesIndex, dataIndex };
}
}
return null;
}
/** `[null, [xs, ys, sizes?], …]`: uPlot's faceted layout, series 0 empty. */
export function prepareScatterChartData(
series: ScatterSeries[],
): uPlot.AlignedData {
const data: ScatterChartData = [
null,
...series.map(
(entry): ScatterSeriesData =>
entry.sizes ? [entry.xs, entry.ys, entry.sizes] : [entry.xs, entry.ys],
),
];
return data as unknown as uPlot.AlignedData;
}
export interface AxisDistribution {
distribution: DistributionType;
asinhThreshold?: number;
}
const LINEAR_DISTRIBUTION: AxisDistribution = {
distribution: DistributionType.Linear,
};
/** Orders of magnitude `auto` needs before it considers leaving linear. */
export const AUTO_MIN_DECADES = 2;
/** How many times more axis the middle half of the points must cover for `auto` to leave linear. */
export const AUTO_SPREAD_GAIN = 2;
function getMagnitudeRange(values: number[]): {
minMagnitude: number;
maxMagnitude: number;
hasNonPositive: boolean;
} {
let minMagnitude = Infinity;
let maxMagnitude = -Infinity;
let hasNonPositive = false;
for (const value of values) {
if (!Number.isFinite(value)) {
continue;
}
if (value <= 0) {
hasNonPositive = true;
}
if (value !== 0) {
minMagnitude = Math.min(minMagnitude, Math.abs(value));
maxMagnitude = Math.max(maxMagnitude, Math.abs(value));
}
}
return { minMagnitude, maxMagnitude, hasNonPositive };
}
/** The linear band ends at the smallest non-zero magnitude so small values still spread. */
function symmetricLogDistribution(minMagnitude: number): AxisDistribution {
const asinhThreshold = Number.isFinite(minMagnitude)
? 10 ** Math.floor(Math.log10(minMagnitude))
: 1;
return { distribution: DistributionType.SymmetricLog, asinhThreshold };
}
function quantile(sorted: number[], q: number): number {
const position = (sorted.length - 1) * q;
const lower = Math.floor(position);
const upper = Math.ceil(position);
return sorted[lower] + (sorted[upper] - sorted[lower]) * (position - lower);
}
/** Share of the axis the middle half of the points covers. */
function middleHalfSpread(
sorted: number[],
[min, max]: [number, number],
distribution: AxisDistribution,
): number {
const { forward } = getScaleTransform(distribution);
const span = forward(max) - forward(min);
return span > 0
? (forward(quantile(sorted, 0.75)) - forward(quantile(sorted, 0.25))) / span
: 0;
}
/**
* Log (symlog with zero or negatives) when it spreads the points out clearly
* more than linear does; min/max alone would let one outlier decide.
*/
function resolveAutoDistribution(
values: number[],
softLimits: number[],
): AxisDistribution {
const sorted = values
.filter((value) => Number.isFinite(value))
.sort((a, b) => a - b);
if (sorted.length < 2) {
return LINEAR_DISTRIBUTION;
}
const { minMagnitude, maxMagnitude, hasNonPositive } = getMagnitudeRange([
...sorted,
...softLimits,
]);
if (!(Math.log10(maxMagnitude / minMagnitude) >= AUTO_MIN_DECADES)) {
return LINEAR_DISTRIBUTION;
}
const candidate = hasNonPositive
? symmetricLogDistribution(minMagnitude)
: { distribution: DistributionType.Logarithmic };
const extent: [number, number] = [
Math.min(sorted[0], ...softLimits),
Math.max(sorted[sorted.length - 1], ...softLimits),
];
const candidateSpread = middleHalfSpread(sorted, extent, candidate);
const linearSpread = middleHalfSpread(sorted, extent, LINEAR_DISTRIBUTION);
return candidateSpread > 0 &&
candidateSpread >= AUTO_SPREAD_GAIN * linearSpread
? candidate
: LINEAR_DISTRIBUTION;
}
/** `log` falls back to symlog rather than drop zero or negatives; callers filter first to drop them. */
export function resolveAxisDistribution(
values: number[],
scale: ScatterAxisScale = ScatterAxisScale.Auto,
/** Soft limits and thresholds the axis must reach; only `auto` weighs them. */
softLimits: Array<number | null | undefined> = [],
): AxisDistribution {
switch (scale) {
case ScatterAxisScale.Linear:
return LINEAR_DISTRIBUTION;
case ScatterAxisScale.SymLog:
return symmetricLogDistribution(getMagnitudeRange(values).minMagnitude);
case ScatterAxisScale.Log: {
const { minMagnitude, hasNonPositive } = getMagnitudeRange(values);
return hasNonPositive
? symmetricLogDistribution(minMagnitude)
: { distribution: DistributionType.Logarithmic };
}
case ScatterAxisScale.Auto:
default:
return resolveAutoDistribution(
values,
softLimits.filter(
(limit): limit is number => limit != null && Number.isFinite(limit),
),
);
}
}
/** The scatter range is unsnapped, so a log axis places and labels its own ticks. */
function getLogAxisTicks({
distribution,
}: AxisDistribution): Pick<AxisProps, 'splits' | 'filter'> {
return distribution === DistributionType.Logarithmic
? { splits: logScaleSplits, filter: spacedLogLabels }
: {};
}
export function buildScatterConfig({
id,
series,
isDarkMode,
x,
y,
pointSize = DEFAULT_SCATTER_POINT_SIZE,
fillOpacity,
colorMapping = {},
thresholds,
decimalPrecision,
selectionPreferencesSource,
shouldSaveSelectionPreference,
}: BuildScatterConfigArgs): UPlotConfigBuilder {
const builder = new UPlotConfigBuilder({
id,
selectionPreferencesSource,
shouldSaveSelectionPreference,
});
const plugin = createScatterPlugin({ pointSize });
applyScatterPlugin(builder, plugin);
// The last x label is centred on the plot's right edge; room for its unit.
builder.setPadding([16, X_AXIS_END_LABEL_PADDING_PX, 8, 8]);
const ySoftLimits = adjustSoftLimitsWithThresholds(
y.softMin ?? null,
y.softMax ?? null,
thresholds,
y.unit,
);
const xDistribution = resolveAxisDistribution(
series.flatMap((entry) => entry.xs),
x.scale,
[x.softMin, x.softMax],
);
const yDistribution = resolveAxisDistribution(
series.flatMap((entry) => entry.ys),
y.scale,
[ySoftLimits.softMin, ySoftLimits.softMax],
);
const yThresholds =
thresholds && thresholds.length > 0
? { scaleKey: 'y', thresholds, yAxisUnit: y.unit }
: undefined;
// The largest disc drawn, plus its outline, kept clear of each plot edge.
const largestDiameter = series.some((entry) => entry.sizes)
? pointSize.max
: pointSize.fixed;
const marginPx = largestDiameter / 2 + POINT_STROKE_WIDTH;
builder.addScale({
scaleKey: 'x',
time: false,
range: createScatterRange({
dimension: 'x',
marginPx,
softMin: x.softMin,
softMax: x.softMax,
...xDistribution,
}),
...xDistribution,
});
builder.addScale({
scaleKey: 'y',
time: false,
range: createScatterRange({
dimension: 'y',
marginPx,
...ySoftLimits,
...yDistribution,
}),
...yDistribution,
});
builder.addAxis({
scaleKey: 'x',
side: 2,
label: x.label || undefined,
isDarkMode,
isTimeAxis: false,
yAxisUnit: x.unit ?? '',
decimalPrecision,
isLogScale: xDistribution.distribution !== DistributionType.Linear,
...getLogAxisTicks(xDistribution),
space: X_AXIS_TICK_SPACE_PX,
});
builder.addAxis({
scaleKey: 'y',
side: 3,
label: y.label || undefined,
isDarkMode,
yAxisUnit: y.unit ?? '',
decimalPrecision,
isLogScale: yDistribution.distribution !== DistributionType.Linear,
...getLogAxisTicks(yDistribution),
});
series.forEach((entry) => {
builder.addSeries({
scaleKey: 'y',
label: entry.label,
colorMapping,
drawStyle: DrawStyle.Scatter,
pathBuilder: plugin.pathBuilder,
facets: SCATTER_FACETS,
lineWidth: POINT_STROKE_WIDTH,
pointSize: pointSize.fixed,
fillOpacity,
isDarkMode,
});
});
if (yThresholds) {
builder.addThresholds(yThresholds);
}
return builder;
}

View File

@@ -16,6 +16,10 @@ import type {
HeatmapColorOptions,
HeatmapSeries,
} from 'lib/uPlotV2/plugins/HeatmapPlugin/types';
import type {
ScatterChannels,
ScatterPointLabel,
} from 'lib/uPlotV2/plugins/ScatterPlugin/types';
import {
DashboardCursorSync,
SyncTooltipFilterMode,
@@ -88,6 +92,26 @@ export interface HistogramChartProps extends ChartWrapperProps {
isQueriesMerged?: boolean;
}
/** The dot a click landed on; `seriesIndex` is uPlot's, so the first series is 1. */
export interface ScatterPointClick {
seriesIndex: number;
dataIndex: number;
color: string;
/** Viewport coordinates of the click. */
coordinates: { x: number; y: number };
}
/** `data` is mode-2 (`prepareScatterChartData`); `config` comes from `buildScatterConfig`. */
export interface ScatterChartProps extends ChartWrapperProps {
channels: ScatterChannels;
resolvePointLabels?: (
seriesIndex: number,
dataIndex: number,
) => ScatterPointLabel[];
/** Fires for a click on a dot; a click on empty plot space fires nothing. */
onPointClick?: (click: ScatterPointClick) => void;
}
/**
* Data arrives as the query response carries it — bucket bounds plus one series per
* group — and the chart pivots and sums it, so no caller has to get the transpose

View File

@@ -66,7 +66,6 @@ import {
import { sanitizeOrderByForExplorer } from 'utils/sanitizeOrderBy';
import { v4 as uuid } from 'uuid';
import { getUnstableCurrentSearchParams } from 'utils/getUnstableCurrentSearchParams';
import { readPanelTypeFromUrl } from 'utils/readPanelTypeFromUrl';
export const QueryBuilderContext = createContext<QueryBuilderContextType>({
currentQuery: initialQueriesMap.metrics,
@@ -183,11 +182,6 @@ export function QueryBuilderProvider({
})) ?? [],
};
const urlPanelType = readPanelTypeFromUrl(PANEL_TYPES.LIST);
const isRawPanel =
urlPanelType === PANEL_TYPES.LIST || urlPanelType === PANEL_TYPES.TRACE;
const setupedQueryData = builder.queryData.map((item) => {
const currentElement: IBuilderQuery = {
...item,
@@ -206,10 +200,8 @@ export function QueryBuilderProvider({
// Explorer pages: sanitize stale orderBy before first query
const isExplorer =
(location.pathname === ROUTES.LOGS_EXPLORER ||
location.pathname === ROUTES.TRACES_EXPLORER ||
location.pathname === ROUTES.AI_OBSERVABILITY_EXPLORER) &&
!isRawPanel;
location.pathname === ROUTES.LOGS_EXPLORER ||
location.pathname === ROUTES.TRACES_EXPLORER;
if (isExplorer) {
const sanitizedOrderBy = sanitizeOrderByForExplorer(currentElement);
return calledFromHandleRunQuery
@@ -270,8 +262,6 @@ export function QueryBuilderProvider({
setStagedQuery(nextQuery);
setCurrentQuery(newQueryState);
setQueryType(type);
setCalledFromHandleRunQuery(false);
},
[prepareQueryBuilderData],
);
@@ -1048,8 +1038,7 @@ export function QueryBuilderProvider({
const handleRunQuery = useCallback(() => {
const isExplorer =
location.pathname === ROUTES.LOGS_EXPLORER ||
location.pathname === ROUTES.TRACES_EXPLORER ||
location.pathname === ROUTES.AI_OBSERVABILITY_EXPLORER;
location.pathname === ROUTES.TRACES_EXPLORER;
if (isExplorer) {
setCalledFromHandleRunQuery(true);
}

View File

@@ -1,176 +0,0 @@
import { initialQueriesMap, PANEL_TYPES } from 'constants/queryBuilder';
import ROUTES from 'constants/routes';
import { useQueryBuilder } from 'hooks/queryBuilder/useQueryBuilder';
import {
act,
AllTheProviders,
renderHook,
RenderHookResult,
} from 'tests/test-utils';
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import { QueryBuilderContextType } from 'types/common/queryBuilder';
import {
__resetSearchParamsGetter,
__setSearchParamsGetterForTest,
} from 'utils/getUnstableCurrentSearchParams';
const timestampOrderBy = [{ columnName: 'timestamp', order: 'desc' }];
const queryWithTimestampOrder = (base: Query): Query => ({
...base,
builder: {
...base.builder,
queryData: base.builder.queryData.map((item) => ({
...item,
orderBy: timestampOrderBy,
})),
},
});
const renderOn = (
route: string,
panelType: PANEL_TYPES | null,
): RenderHookResult<QueryBuilderContextType, unknown> => {
const search = new URLSearchParams();
if (panelType) {
search.set('panelTypes', JSON.stringify(panelType));
}
__setSearchParamsGetterForTest(() => search);
return renderHook(() => useQueryBuilder(), {
wrapper: ({ children }): JSX.Element => (
<AllTheProviders initialRoute={route}>{children}</AllTheProviders>
),
});
};
const initOn = (
route: string,
panelType: PANEL_TYPES | null,
query: Query,
): Query | null => {
const { result } = renderOn(route, panelType);
act(() => {
result.current.initQueryBuilderData(query);
});
return result.current.stagedQuery;
};
describe('explorer orderBy sanitizing by panel type', () => {
afterEach(() => {
__resetSearchParamsGetter();
});
it('keeps the order on the logs list panel', () => {
const staged = initOn(
ROUTES.LOGS_EXPLORER,
PANEL_TYPES.LIST,
queryWithTimestampOrder(initialQueriesMap.logs),
);
expect(staged?.builder.queryData[0].orderBy).toStrictEqual(timestampOrderBy);
});
it('keeps the order when panelTypes is absent, since the explorers default to list', () => {
const staged = initOn(
ROUTES.LOGS_EXPLORER,
null,
queryWithTimestampOrder(initialQueriesMap.logs),
);
expect(staged?.builder.queryData[0].orderBy).toStrictEqual(timestampOrderBy);
});
it('keeps the order on the traces trace panel', () => {
const staged = initOn(
ROUTES.TRACES_EXPLORER,
PANEL_TYPES.TRACE,
queryWithTimestampOrder(initialQueriesMap.traces),
);
expect(staged?.builder.queryData[0].orderBy).toStrictEqual(timestampOrderBy);
});
// engineering-pod#2813: a timestamp order carried from the list view into the
// time series tab is not a group-by key or an aggregation, and errors the query.
it('drops the order on the time series panel', () => {
const staged = initOn(
ROUTES.LOGS_EXPLORER,
PANEL_TYPES.TIME_SERIES,
queryWithTimestampOrder(initialQueriesMap.logs),
);
expect(staged?.builder.queryData[0].orderBy).toStrictEqual([]);
});
it('keeps the order on the LLM observability list panel', () => {
const staged = initOn(
ROUTES.AI_OBSERVABILITY_EXPLORER,
PANEL_TYPES.LIST,
queryWithTimestampOrder(initialQueriesMap.traces),
);
expect(staged?.builder.queryData[0].orderBy).toStrictEqual(timestampOrderBy);
});
it('drops the order on the LLM observability time series panel', () => {
const staged = initOn(
ROUTES.AI_OBSERVABILITY_EXPLORER,
PANEL_TYPES.TIME_SERIES,
queryWithTimestampOrder(initialQueriesMap.traces),
);
expect(staged?.builder.queryData[0].orderBy).toStrictEqual([]);
});
it('drops the order on the table panel', () => {
const staged = initOn(
ROUTES.TRACES_EXPLORER,
PANEL_TYPES.TABLE,
queryWithTimestampOrder(initialQueriesMap.traces),
);
expect(staged?.builder.queryData[0].orderBy).toStrictEqual([]);
});
});
// handleRunQuery means "respect the query I just built" for the init it causes.
// It used to stay set for the rest of the visit, so every later init skipped
// sanitizing as well.
describe('the run bypass is one-shot', () => {
afterEach(() => {
__resetSearchParamsGetter();
});
it('skips sanitizing for the init after a run, and sanitizes the next one', () => {
const { result } = renderOn(ROUTES.LOGS_EXPLORER, PANEL_TYPES.TIME_SERIES);
act(() => {
result.current.handleRunQuery();
});
const afterRun = {
...queryWithTimestampOrder(initialQueriesMap.logs),
id: 'run-1',
};
act(() => {
result.current.initQueryBuilderData(afterRun);
});
expect(
result.current.stagedQuery?.builder.queryData[0].orderBy,
).toStrictEqual(timestampOrderBy);
const next = {
...queryWithTimestampOrder(initialQueriesMap.logs),
id: 'run-2',
};
act(() => {
result.current.initQueryBuilderData(next);
});
expect(
result.current.stagedQuery?.builder.queryData[0].orderBy,
).toStrictEqual([]);
});
});

View File

@@ -1,137 +0,0 @@
import { initialQueriesMap } from 'constants/queryBuilder';
import { Query } from 'types/api/queryBuilder/queryBuilderData';
import {
DEFAULT_LIST_ORDER_BY,
getListOrderBy,
getRawPanelOrderBy,
parseListOrderBy,
setListOrderBy,
} from 'utils/explorerUtils';
const withOrderBy = (
orderBy: { columnName: string; order: string }[],
): Query => ({
...initialQueriesMap.logs,
builder: {
...initialQueriesMap.logs.builder,
queryData: initialQueriesMap.logs.builder.queryData.map((item) => ({
...item,
orderBy,
})),
},
});
describe('getRawPanelOrderBy', () => {
it('keeps row fields', () => {
expect(
getRawPanelOrderBy(
withOrderBy([{ columnName: 'duration_nano', order: 'desc' }]),
),
).toStrictEqual([{ columnName: 'duration_nano', order: 'desc' }]);
});
it('keeps a group-by key, which is a row field too', () => {
expect(
getRawPanelOrderBy(
withOrderBy([{ columnName: 'service.name', order: 'asc' }]),
),
).toStrictEqual([{ columnName: 'service.name', order: 'asc' }]);
});
it('drops an aggregation carried in from an aggregated panel', () => {
expect(
getRawPanelOrderBy(withOrderBy([{ columnName: 'count()', order: 'desc' }])),
).toStrictEqual([]);
});
it('drops the aggregated value sentinel', () => {
expect(
getRawPanelOrderBy(
withOrderBy([{ columnName: '#SIGNOZ_VALUE', order: 'desc' }]),
),
).toStrictEqual([]);
});
it('keeps the usable entry and drops the rest', () => {
expect(
getRawPanelOrderBy(
withOrderBy([
{ columnName: 'count()', order: 'desc' },
{ columnName: 'timestamp', order: 'desc' },
]),
),
).toStrictEqual([{ columnName: 'timestamp', order: 'desc' }]);
});
});
describe('getListOrderBy', () => {
it('falls back to the default when the query holds only an aggregation', () => {
expect(
getListOrderBy(withOrderBy([{ columnName: 'count()', order: 'desc' }])),
).toBe(DEFAULT_LIST_ORDER_BY);
});
it('reads the first order entry as a select value', () => {
expect(
getListOrderBy(withOrderBy([{ columnName: 'timestamp', order: 'asc' }])),
).toBe('timestamp:asc');
});
it('lower-cases a hand-written direction from a deep link', () => {
expect(
getListOrderBy(withOrderBy([{ columnName: 'timestamp', order: 'ASC' }])),
).toBe('timestamp:asc');
});
it('shows the primary entry when the query also carries the id tiebreaker', () => {
const query = withOrderBy([
{ columnName: 'timestamp', order: 'desc' },
{ columnName: 'id', order: 'desc' },
]);
expect(getListOrderBy(query)).toBe('timestamp:desc');
});
it('falls back to the default when the query carries no order', () => {
expect(getListOrderBy(withOrderBy([]))).toBe(DEFAULT_LIST_ORDER_BY);
expect(getListOrderBy(null)).toBe(DEFAULT_LIST_ORDER_BY);
});
it('falls back to the given default', () => {
expect(getListOrderBy(withOrderBy([]), 'last_activity_time:desc')).toBe(
'last_activity_time:desc',
);
});
});
describe('parseListOrderBy', () => {
it('splits a select value into an order payload', () => {
expect(parseListOrderBy('durationNano:asc')).toStrictEqual({
columnName: 'durationNano',
order: 'asc',
});
});
});
describe('setListOrderBy', () => {
it('replaces the order on every queryData entry', () => {
const base = withOrderBy([{ columnName: 'timestamp', order: 'desc' }]);
const next = setListOrderBy(base, 'severity_text:asc');
next.builder.queryData.forEach((item) => {
expect(item.orderBy).toStrictEqual([
{ columnName: 'severity_text', order: 'asc' },
]);
});
});
it('does not mutate the query it is given', () => {
const base = withOrderBy([{ columnName: 'timestamp', order: 'desc' }]);
setListOrderBy(base, 'timestamp:asc');
expect(base.builder.queryData[0].orderBy).toStrictEqual([
{ columnName: 'timestamp', order: 'desc' },
]);
});
});

View File

@@ -1,45 +0,0 @@
import { PANEL_TYPES } from 'constants/queryBuilder';
import {
__resetSearchParamsGetter,
__setSearchParamsGetterForTest,
} from 'utils/getUnstableCurrentSearchParams';
import { readPanelTypeFromUrl } from 'utils/readPanelTypeFromUrl';
const withParam = (value?: string): void => {
const search = new URLSearchParams();
if (value !== undefined) {
search.set('panelTypes', value);
}
__setSearchParamsGetterForTest(() => search);
};
describe('readPanelTypeFromUrl', () => {
afterEach(() => {
__resetSearchParamsGetter();
});
it('decodes the JSON-encoded value the app writes', () => {
withParam(JSON.stringify(PANEL_TYPES.TIME_SERIES));
expect(readPanelTypeFromUrl()).toBe(PANEL_TYPES.TIME_SERIES);
});
it('accepts an unencoded value from a hand-written link', () => {
withParam('list');
expect(readPanelTypeFromUrl()).toBe(PANEL_TYPES.LIST);
});
it('returns the raw value instead of throwing on a malformed one', () => {
withParam('{list');
expect(readPanelTypeFromUrl()).toBe('{list');
});
it('returns the fallback when the param is absent', () => {
withParam();
expect(readPanelTypeFromUrl(PANEL_TYPES.LIST)).toBe(PANEL_TYPES.LIST);
expect(readPanelTypeFromUrl()).toBeNull();
});
});

View File

@@ -1,69 +1,8 @@
import { QueryParams } from 'constants/query';
import { PANEL_TYPES } from 'constants/queryBuilder';
import ROUTES from 'constants/routes';
import { SIGNOZ_VALUE } from 'container/QueryBuilder/filters/OrderByFilter/constants';
import { ExplorerViews } from 'pages/LogsExplorer/utils';
import { OrderByPayload, Query } from 'types/api/queryBuilder/queryBuilderData';
import { getParsedAggregationOptionsForOrderBy } from './aggregationConverter';
export const DEFAULT_LIST_ORDER_BY = 'timestamp:desc';
export const getRawPanelOrderBy = (query: Query | null): OrderByPayload[] => {
const queryData = query?.builder?.queryData?.[0];
if (!queryData) {
return [];
}
const disallowed = new Set<string>([SIGNOZ_VALUE]);
getParsedAggregationOptionsForOrderBy(queryData).forEach((aggregation) => {
if (aggregation?.key) {
disallowed.add(aggregation.key);
}
});
return (queryData.orderBy ?? []).filter(
(item) => !disallowed.has(item.columnName),
);
};
export const getListOrderBy = (
query: Query | null,
fallback: string = DEFAULT_LIST_ORDER_BY,
): string => {
const [orderBy] = getRawPanelOrderBy(query);
if (!orderBy?.columnName || !orderBy?.order) {
return fallback;
}
return `${orderBy.columnName}:${orderBy.order.toLowerCase()}`;
};
export const parseListOrderBy = (orderBy: string): OrderByPayload => {
const [columnName, order] = orderBy.split(':');
return {
columnName: columnName || 'timestamp',
order: order || 'desc',
};
};
export const setListOrderBy = (query: Query, orderBy: string): Query => {
const nextOrderBy = [parseListOrderBy(orderBy)];
return {
...query,
builder: {
...query.builder,
queryData: query.builder.queryData.map((item) => ({
...item,
orderBy: nextOrderBy,
})),
},
};
};
import { Query } from 'types/api/queryBuilder/queryBuilderData';
// Mapping between panel types and explorer views
export const panelTypeToExplorerView: Record<PANEL_TYPES, ExplorerViews> = {

View File

@@ -1,21 +0,0 @@
import { QueryParams } from 'constants/query';
import { PANEL_TYPES } from 'constants/queryBuilder';
import { getUnstableCurrentSearchParams } from './getUnstableCurrentSearchParams';
// The param is JSON-encoded, but a hand-written link can carry it unencoded.
export const readPanelTypeFromUrl = (
fallback: PANEL_TYPES | null = null,
): PANEL_TYPES | null => {
const raw = getUnstableCurrentSearchParams().get(QueryParams.panelTypes);
if (!raw) {
return fallback;
}
try {
return JSON.parse(raw) as PANEL_TYPES;
} catch {
return raw as PANEL_TYPES;
}
};

View File

@@ -9,6 +9,7 @@ var (
FeaturePutMetersInZeus = featuretypes.MustNewName("put_meters_in_zeus")
FeatureUseMeterReporter = featuretypes.MustNewName("use_meter_reporter")
FeatureUseJSONBody = featuretypes.MustNewName("use_json_body")
FeatureJSONBodyDualIngestion = featuretypes.MustNewName("json_body_dual_ingestion")
FeatureEnableMetricsReduction = featuretypes.MustNewName("enable_metrics_reduction")
FeatureResolveSemconvFamilies = featuretypes.MustNewName("resolve_semconv_families")
FeatureUseTraceAttributesJSON = featuretypes.MustNewName("use_trace_attributes_json")
@@ -64,6 +65,14 @@ func MustNewRegistry() featuretypes.Registry {
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureJSONBodyDualIngestion,
Kind: featuretypes.KindBoolean,
Stage: featuretypes.StageExperimental,
Description: "Controls whether the collector's normalize operator keeps the original log body so it is ingested into both the legacy body and the JSON body columns",
DefaultVariant: featuretypes.MustNewName("disabled"),
Variants: featuretypes.NewBooleanVariants(),
},
&featuretypes.Feature{
Name: FeatureEnableMetricsReduction,
Kind: featuretypes.KindBoolean,

View File

@@ -2,7 +2,6 @@ package logparsingpipeline
import (
"encoding/json"
"slices"
"strings"
"sync"
@@ -25,13 +24,10 @@ var (
)
const (
memoryLimiterProcessor = "memory_limiter"
memoryLimiterProcessorPrefix = "memory_limiter/"
batchProcessor = "batch"
batchProcessorPrefix = "batch/"
normalizerProcessor = "signozlogsnormalizer"
normalizerProcessorPrefix = "signozlogsnormalizer/"
userPipelinesBeforeNormalizer = "before"
memoryLimiterProcessor = "memory_limiter"
memoryLimiterProcessorPrefix = "memory_limiter/"
batchProcessor = "batch"
batchProcessorPrefix = "batch/"
)
// check if the processors already exist
@@ -94,14 +90,9 @@ func getOtelPipelineFromConfig(config map[string]interface{}) (*otelPipeline, er
// 2. signoz user-pipeline processors (in the order given by signozPipelineProcessorNames)
// 3. custom processors (non-signoz, non-memory_limiter, non-batch processors from the current config)
// 4. batch processors (any processor named "batch" or "batch/<id>") and anything after them
//
// If a signozlogsnormalizer processor is present, the order above is skipped: user-pipeline
// processors go right after the first one, or right before it when its config sets
// `opamp.position: before`, and the other processors keep their places.
func buildCollectorPipelineProcessorsList(
currentCollectorProcessors []string,
signozPipelineProcessorNames []string,
processorConfigs map[string]interface{},
) ([]string, error) {
lockLogsPipelineSpec.Lock()
defer lockLogsPipelineSpec.Unlock()
@@ -112,23 +103,6 @@ func buildCollectorPipelineProcessorsList(
userPipelineSet[p] = struct{}{}
}
if normalizerIdx := slices.IndexFunc(currentCollectorProcessors, isNormalizerProcessor); normalizerIdx >= 0 {
before := opampPosition(processorConfigs, currentCollectorProcessors[normalizerIdx]) == userPipelinesBeforeNormalizer
result := make([]string, 0, len(currentCollectorProcessors)+len(signozPipelineProcessorNames))
for idx, p := range currentCollectorProcessors {
if idx == normalizerIdx && before {
result = append(result, signozPipelineProcessorNames...)
}
if _, inUserPipelines := userPipelineSet[p]; !inUserPipelines && !hasSignozPipelineProcessorPrefix(p) {
result = append(result, p)
}
if idx == normalizerIdx && !before {
result = append(result, signozPipelineProcessorNames...)
}
}
return result, nil
}
var memoryLimiters []string
var customProcessors []string
batchIdx := -1
@@ -161,51 +135,13 @@ func buildCollectorPipelineProcessorsList(
return result, nil
}
func isNormalizerProcessor(name string) bool {
return name == normalizerProcessor || strings.HasPrefix(name, normalizerProcessorPrefix)
}
// collectorNormalizesFirst reports whether the logs pipeline of the collector config runs
// signozlogsnormalizer ahead of user pipelines, judged by the first one as in buildCollectorPipelineProcessorsList.
// Malformed configs report false; generateCollectorConfigWithPipelines surfaces their errors.
func collectorNormalizesFirst(collectorConf map[string]interface{}) bool {
p, err := getOtelPipelineFromConfig(collectorConf)
if err != nil || p.Pipelines.Logs == nil {
return false
}
normalizerIdx := slices.IndexFunc(p.Pipelines.Logs.Processors, isNormalizerProcessor)
if normalizerIdx < 0 {
return false
}
processorConfigs, _ := collectorConf["processors"].(map[string]interface{})
return opampPosition(processorConfigs, p.Pipelines.Logs.Processors[normalizerIdx]) != userPipelinesBeforeNormalizer
}
func opampPosition(processorConfigs map[string]interface{}, name string) string {
config, _ := processorConfigs[name].(map[string]interface{})
opamp, _ := config["opamp"].(map[string]interface{})
position, _ := opamp["position"].(string)
return position
}
func unmarshalCollectorConfig(config []byte) (map[string]interface{}, error) {
func GenerateCollectorConfigWithPipelines(config []byte, pipelines []pipelinetypes.GettablePipeline) ([]byte, error) {
var collectorConf map[string]interface{}
if err := yaml.Unmarshal(config, &collectorConf); err != nil {
err := yaml.Unmarshal([]byte(config), &collectorConf)
if err != nil {
return nil, errors.WrapInvalidInputf(err, CodeCollectorConfigUnmarshalFailed, "could not unmarshal collector config")
}
return collectorConf, nil
}
func GenerateCollectorConfigWithPipelines(config []byte, pipelines []pipelinetypes.GettablePipeline) ([]byte, error) {
collectorConf, err := unmarshalCollectorConfig(config)
if err != nil {
return nil, err
}
return generateCollectorConfigWithPipelines(collectorConf, pipelines)
}
// generateCollectorConfigWithPipelines modifies collectorConf in place.
func generateCollectorConfigWithPipelines(collectorConf map[string]any, pipelines []pipelinetypes.GettablePipeline) ([]byte, error) {
signozPipelineProcessors, signozPipelineProcNames, err := PreparePipelineProcessor(pipelines)
if err != nil {
return nil, err
@@ -244,9 +180,7 @@ func generateCollectorConfigWithPipelines(collectorConf map[string]any, pipeline
return nil, errors.NewInternalf(CodeCollectorConfigLogsPipelineNotFound, "logs pipeline doesn't exist")
}
updatedProcessorList, _ := buildCollectorPipelineProcessorsList(
p.Pipelines.Logs.Processors, signozPipelineProcNames, collectorConf["processors"].(map[string]interface{}),
)
updatedProcessorList, _ := buildCollectorPipelineProcessorsList(p.Pipelines.Logs.Processors, signozPipelineProcNames)
p.Pipelines.Logs.Processors = updatedProcessorList
// add the new processor to the data ( no checks required as the keys will exists)

View File

@@ -5,14 +5,11 @@ import (
"fmt"
"testing"
"github.com/SigNoz/signoz/pkg/flagger/flaggertest"
"github.com/SigNoz/signoz/pkg/query-service/constants"
"github.com/SigNoz/signoz/pkg/query-service/model"
v3 "github.com/SigNoz/signoz/pkg/query-service/model/v3"
"github.com/SigNoz/signoz/pkg/types/pipelinetypes"
"github.com/SigNoz/signoz/pkg/valuer"
. "github.com/smartystreets/goconvey/convey"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gopkg.in/yaml.v3"
)
@@ -209,74 +206,13 @@ var BuildLogsPipelineTestData = []struct {
func TestBuildLogsPipeline(t *testing.T) {
for _, test := range BuildLogsPipelineTestData {
Convey(test.Name, t, func() {
v, err := buildCollectorPipelineProcessorsList(test.fromCollector, test.userPipelines, nil)
v, err := buildCollectorPipelineProcessorsList(test.fromCollector, test.userPipelines)
So(err, ShouldBeNil)
So(v, ShouldResemble, test.finalOutput)
})
}
}
func TestBuildLogsPipelineWithNormalizer(t *testing.T) {
a := constants.LogsPPLPfx + "a"
b := constants.LogsPPLPfx + "b"
before := map[string]any{"opamp": map[string]any{"position": "before"}}
after := map[string]any{"opamp": map[string]any{"position": "after"}}
testCases := []struct {
name string
fromCollector []string
userPipelines []string
configs map[string]any
expected []string
}{
{
name: "ExistingAndStaleUserProcessorsReplaced",
fromCollector: []string{"memory_limiter", a, constants.LogsPPLPfx + "stale", constants.OldLogsPPLPfx + "old", "signozlogsnormalizer", "custom", "batch"},
userPipelines: []string{a, b},
expected: []string{"memory_limiter", "signozlogsnormalizer", a, b, "custom", "batch"},
},
{
name: "NamedNormalizerBefore_CustomProcessorsKeepPlace",
fromCollector: []string{"memory_limiter", "custom_a", "signozlogsnormalizer/x", a, "custom_b", "batch"},
userPipelines: []string{a},
configs: map[string]any{"signozlogsnormalizer/x": before},
expected: []string{"memory_limiter", "custom_a", a, "signozlogsnormalizer/x", "custom_b", "batch"},
},
{
name: "FirstNormalizerBefore_SecondIgnored",
fromCollector: []string{"memory_limiter", "signozlogsnormalizer/first", "signozlogsnormalizer/second", "batch"},
userPipelines: []string{a},
configs: map[string]any{"signozlogsnormalizer/first": before, "signozlogsnormalizer/second": after},
expected: []string{"memory_limiter", a, "signozlogsnormalizer/first", "signozlogsnormalizer/second", "batch"},
},
{
name: "FirstNormalizerAfter_SecondIgnored",
fromCollector: []string{"memory_limiter", "signozlogsnormalizer/first", "signozlogsnormalizer/second", "batch"},
userPipelines: []string{a},
configs: map[string]any{"signozlogsnormalizer/first": after, "signozlogsnormalizer/second": before},
expected: []string{"memory_limiter", "signozlogsnormalizer/first", a, "signozlogsnormalizer/second", "batch"},
},
{
name: "NormalizerAfterBatch",
fromCollector: []string{"memory_limiter", "batch", "signozlogsnormalizer"},
userPipelines: []string{a},
expected: []string{"memory_limiter", "batch", "signozlogsnormalizer", a},
},
{
name: "NoUserPipelines_StaleRemoved",
fromCollector: []string{"memory_limiter", a, "signozlogsnormalizer", "batch"},
expected: []string{"memory_limiter", "signozlogsnormalizer", "batch"},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
processors, err := buildCollectorPipelineProcessorsList(testCase.fromCollector, testCase.userPipelines, testCase.configs)
require.NoError(t, err)
assert.Equal(t, testCase.expected, processors)
})
}
}
func TestPipelineAliasCollisionsDontResultInDuplicateCollectorProcessors(t *testing.T) {
require := require.New(t)
@@ -501,61 +437,3 @@ func TestPipeCharInAliasDoesntBreakCollectorConfig(t *testing.T) {
}, result[0].Attributes_string,
)
}
func TestRecommendAgentConfigWithNormalizerProcessorAndJSONBody(t *testing.T) {
userPipeline := makeTestAddAttributePipeline()
controller := &LogParsingPipelineController{
fl: flaggertest.WithUseJSONBody(t, true),
GetIntegrationPipelines: func(context.Context, string) ([]pipelinetypes.GettablePipeline, error) {
return []pipelinetypes.GettablePipeline{userPipeline}, nil
},
}
normalizeOperator := CollectorConfProcessorName(getNormalizePipeline())
user := CollectorConfProcessorName(userPipeline)
testCases := []struct {
name string
processors string
logs string
expected []any
}{
{
name: "NoNormalizer_OperatorPrepended",
logs: "[memory_limiter, batch]",
expected: []any{"memory_limiter", normalizeOperator, user, "batch"},
},
{
name: "NormalizerDefault_OperatorDropped",
logs: "[memory_limiter, signozlogsnormalizer, batch]",
expected: []any{"memory_limiter", "signozlogsnormalizer", user, "batch"},
},
{
name: "NormalizerAfter_OperatorDropped",
processors: "signozlogsnormalizer:\n opamp:\n position: after\n",
logs: "[memory_limiter, signozlogsnormalizer, batch]",
expected: []any{"memory_limiter", "signozlogsnormalizer", user, "batch"},
},
{
name: "NormalizerBefore_OperatorKept",
processors: "signozlogsnormalizer:\n opamp:\n position: before\n",
logs: "[memory_limiter, signozlogsnormalizer, batch]",
expected: []any{"memory_limiter", normalizeOperator, user, "signozlogsnormalizer", "batch"},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
conf := fmt.Sprintf(
"processors:\n memory_limiter: {}\n batch: {}\n %s\nservice:\n pipelines:\n logs:\n receivers: [otlp]\n processors: %s\n exporters: [clickhouselogsexporter]\n",
testCase.processors, testCase.logs,
)
recommended, _, err := controller.RecommendAgentConfig(valuer.GenerateUUID(), []byte(conf), nil)
require.NoError(t, err)
var recommendedConf map[string]any
require.NoError(t, yaml.Unmarshal(recommended, &recommendedConf))
logs := recommendedConf["service"].(map[string]any)["pipelines"].(map[string]any)["logs"].(map[string]any)
assert.Equal(t, testCase.expected, logs["processors"])
})
}
}

View File

@@ -220,17 +220,26 @@ func (ic *LogParsingPipelineController) ValidatePipelines(ctx context.Context,
return err
}
// withNormalizePipeline places normalize ahead of user pipelines when queries run on body_v2
// (use_json_body), so operators see the body the explorer shows. Absent when the flag is off
// or collectorNormalizesFirst.
func (ic *LogParsingPipelineController) withNormalizePipeline(ctx context.Context, orgID valuer.UUID, pipelines []pipelinetypes.GettablePipeline, collectorNormalizesFirst bool) []pipelinetypes.GettablePipeline {
if collectorNormalizesFirst || !ic.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID)) {
// withNormalizePipeline places normalize where the read path dictates. Ahead of user pipelines
// when queries run on body_v2 (use_json_body), so operators see the body the explorer shows.
// After them when dual ingestion alone writes body_v2, so operators keep seeing the raw body
// users still query. Absent when neither flag is on.
func (ic *LogParsingPipelineController) withNormalizePipeline(ctx context.Context, orgID valuer.UUID, pipelines []pipelinetypes.GettablePipeline) []pipelinetypes.GettablePipeline {
evalCtx := featuretypes.NewFlaggerEvaluationContext(orgID)
dualIngestion := ic.fl.BooleanOrEmpty(ctx, flagger.FeatureJSONBodyDualIngestion, evalCtx)
switch {
case ic.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, evalCtx):
return append([]pipelinetypes.GettablePipeline{getNormalizePipeline(dualIngestion)}, pipelines...)
case dualIngestion:
return append(slices.Clone(pipelines), getNormalizePipeline(true))
default:
return pipelines
}
return append([]pipelinetypes.GettablePipeline{getNormalizePipeline()}, pipelines...)
}
func getNormalizePipeline() pipelinetypes.GettablePipeline {
// stashOriginalBody makes normalize carry the pre-normalization body in an internal attribute
// for the exporter to restore into the legacy body column.
func getNormalizePipeline(stashOriginalBody bool) pipelinetypes.GettablePipeline {
return pipelinetypes.GettablePipeline{
StoreablePipeline: pipelinetypes.StoreablePipeline{
Name: "Default Pipeline - PreProcessing Body",
@@ -249,10 +258,11 @@ func getNormalizePipeline() pipelinetypes.GettablePipeline {
},
Config: []pipelinetypes.PipelineOperator{
{
ID: uuid.NewString(),
Type: "normalize",
Enabled: true,
If: "body != nil",
ID: uuid.NewString(),
Type: "normalize",
Enabled: true,
If: "body != nil",
JSONBodyDualIngestion: stashOriginalBody,
},
},
}
@@ -361,8 +371,11 @@ func (ic *LogParsingPipelineController) PreviewLogsPipelines(
}
// The collector gets the same pipeline prepended over opamp; see RecommendAgentConfig.
// Under dual ingestion alone it runs after user operators and only feeds body_v2, which
// the explorer does not show yet, so the preview leaves it out. The original-body stash
// is left off: the preview has no exporter to restore and strip it.
if ic.fl.BooleanOrEmpty(ctx, flagger.FeatureUseJSONBody, featuretypes.NewFlaggerEvaluationContext(orgID)) {
pipelines = append([]pipelinetypes.GettablePipeline{getNormalizePipeline()}, pipelines...)
pipelines = append([]pipelinetypes.GettablePipeline{getNormalizePipeline(false)}, pipelines...)
}
result, collectorLogs, err := SimulatePipelinesProcessing(ctx, pipelines, request.Logs)
@@ -383,14 +396,14 @@ func (pc *LogParsingPipelineController) AgentFeatureType() agentConf.AgentFeatur
// Implements agentConf.AgentFeature interface.
// RecommendAgentConfig generates the collector config to be sent to agents.
// The normalize pipeline (when use_json_body is on) is placed
// The normalize pipeline (when use_json_body or json_body_dual_ingestion is on) is placed
// here, after rawPipelineData is serialized. So it is only present in the config sent to
// the collector and never persisted to the database as part of the user's pipeline list.
//
// NOTE: The configId sent to agents is derived from the pipeline version number
// (e.g. "LogPipelines:5"), not the YAML content. If server-side logic changes
// the generated YAML without bumping the version (e.g. toggling the use_json_body
// flag or updating operator IfExpressions), agents that already
// the generated YAML without bumping the version (e.g. toggling the use_json_body or
// json_body_dual_ingestion flags or updating operator IfExpressions), agents that already
// applied that version will not re-apply the new config. In such cases, users must save a
// new pipeline version via the API to force agents to pick up the change.
func (pc *LogParsingPipelineController) RecommendAgentConfig(
@@ -418,15 +431,10 @@ func (pc *LogParsingPipelineController) RecommendAgentConfig(
return nil, "", err
}
collectorConf, err := unmarshalCollectorConfig(currentConfYaml)
if err != nil {
return nil, "", err
}
// normalize is only for sending to the collector, never persisted
enrichedPipelines = pc.withNormalizePipeline(ctx, orgId, enrichedPipelines, collectorNormalizesFirst(collectorConf))
enrichedPipelines = pc.withNormalizePipeline(ctx, orgId, enrichedPipelines)
updatedConf, err := generateCollectorConfigWithPipelines(collectorConf, enrichedPipelines)
updatedConf, err := GenerateCollectorConfigWithPipelines(currentConfYaml, enrichedPipelines)
if err != nil {
return nil, "", err
}

View File

@@ -123,10 +123,22 @@ func (d *DashboardSpec) validatePanels() error {
return err
}
}
// validatePanelQueryCount left exactly one query for a spec that reads it.
if spec, ok := panel.Spec.Plugin.Spec.(queryBoundPanelSpec); ok {
if err := spec.validateAgainstQuery(panel.Spec.Queries[0], path+".spec.plugin.spec"); err != nil {
return err
}
}
}
return nil
}
// queryBoundPanelSpec is a panel spec with fields that depend on the panel's
// query. It is checked after the query itself validates.
type queryBoundPanelSpec interface {
validateAgainstQuery(query Query, path string) error
}
func validatePanelQueryCount(queries []Query, panelKind PanelPluginKind, path string) error {
if queries == nil {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s.spec.queries: is required and must not be null; use [] for a panel that renders without a query", path)

View File

@@ -1869,6 +1869,319 @@ func TestInvalidateAreaChartPanelSpecValues(t *testing.T) {
}
}
func TestScatterPlotPanelDefaults(t *testing.T) {
data := []byte(`{
"variables": [],
"panels": {
"p1": {
"kind": "Panel",
"spec": {
"links": [],
"plugin": {
"kind": "signoz/ScatterPlotPanel",
"spec": {}
},
"queries": [{"kind": "scalar", "spec": {"plugin": {"kind": "signoz/ClickHouseSQL", "spec": {"name": "A", "query": "SELECT 1"}}}}]
}
}
},
"links": [],
"layouts": []
}`)
d, err := unmarshalDashboard(data)
require.NoError(t, err, "unmarshal and validate failed")
require.IsType(t, &ScatterPlotPanelSpec{}, d.Panels["p1"].Spec.Plugin.Spec)
spec := d.Panels["p1"].Spec.Plugin.Spec.(*ScatterPlotPanelSpec)
assert.Empty(t, spec.Dimensions.X, "an omitted x stays empty so the renderer picks the first value column")
assert.Empty(t, spec.Dimensions.Y, "an omitted y stays empty so the renderer picks the second value column")
assert.Empty(t, spec.Dimensions.ColorBy, "an omitted colorBy stays empty so the renderer colours by every group-by label")
assert.Equal(t, "auto", spec.Axes.X.Scale.ValueOrDefault(), "expected x scale default auto")
assert.Equal(t, "auto", spec.Axes.Y.Scale.ValueOrDefault(), "expected y scale default auto")
assert.Nil(t, spec.Axes.X.SoftMin, "expected x softMin unset")
assert.Equal(t, "2", spec.Formatting.DecimalPrecision.ValueOrDefault(), "expected DecimalPrecision default 2")
assert.Equal(t, "global_time", spec.Visualization.TimePreference.ValueOrDefault(), "expected TimePreference default global_time")
assert.Equal(t, "bottom", spec.Legend.Position.ValueOrDefault(), "expected LegendPosition default bottom")
output, err := json.Marshal(d)
require.NoError(t, err, "marshal dashboard failed")
assert.Contains(t, string(output), `"axes":{"x":{"softMin":null,"softMax":null,"scale":"auto","label":""},"y":{"softMin":null,"softMax":null,"scale":"auto","label":""}}`, "expected default axes in stored/response JSON")
assert.Contains(t, string(output), `"dimensions":{"x":"","y":"","sizeBy":"","colorBy":null}`, "expected empty dimensions in stored/response JSON")
assert.Contains(t, string(output), `"chartAppearance":{"points":{"size":null,"minSize":null,"maxSize":null,"opacity":null}}`, "omitted point settings stay null so the renderer applies its defaults")
}
func TestValidateScatterPlotDashboard(t *testing.T) {
data, err := os.ReadFile("testdata/perses_scatter_plot_panel.json")
require.NoError(t, err, "reading example file")
spec, err := unmarshalDashboard(data)
require.NoError(t, err, "unmarshal and validate failed")
require.IsType(t, &ScatterPlotPanelSpec{}, spec.Panels["p1"].Spec.Plugin.Spec)
panelSpec := spec.Panels["p1"].Spec.Plugin.Spec.(*ScatterPlotPanelSpec)
softMin, softMax := 0.0, 1000.0
size, minSize, maxSize, opacity := PointDiameter(8), PointDiameter(4), PointDiameter(24), PointOpacity(0.1)
assert.Equal(t, &ScatterPlotPanelSpec{
Visualization: BasicVisualization{TimePreference: TimePreferenceLast1Hr},
Dimensions: ScatterPlotDimensions{X: "A.count()", Y: "A.p99(duration_nano)", SizeBy: "A.countIf(has_error = true)", ColorBy: []string{"k8s.namespace.name", "k8s.pod.name"}},
Formatting: TableFormatting{ColumnUnits: map[string]string{"A.p99(duration_nano)": "ns"}, DecimalPrecision: PrecisionOption1},
Axes: ScatterPlotAxes{
X: ScatterPlotAxis{SoftMin: &softMin, Scale: AxisScaleLog, Label: "Throughput"},
Y: ScatterPlotAxis{SoftMax: &softMax, Scale: AxisScaleSymlog, Label: "p99 latency"},
},
ChartAppearance: ScatterPlotChartAppearance{Points: ScatterPlotPoints{Size: &size, MinSize: &minSize, MaxSize: &maxSize, Opacity: &opacity}},
Legend: Legend{Position: LegendPositionRight, Mode: LegendModeList, CustomColors: map[string]string{"checkout": "#1f77b4"}},
Thresholds: []ScatterPlotThreshold{
{ThresholdWithLabel: ThresholdWithLabel{Value: 300, Unit: "ms", Color: "#f00", Label: "p99 SLO"}, Axis: ThresholdAxisY},
{ThresholdWithLabel: ThresholdWithLabel{Value: 50, Color: "#0f0"}, Axis: ThresholdAxisX},
},
}, panelSpec)
dashboard := &DashboardV2{Spec: *spec}
request, err := dashboard.GetPanelQuery(1, 2, "p1")
require.NoError(t, err, "building the panel's query failed")
assert.Equal(t, qb.RequestTypeScalar, request.RequestType)
require.NotNil(t, request.FormatOptions)
assert.True(t, request.FormatOptions.FormatTableResultForUI, "scatter rows are joined the way a table's are")
require.Len(t, request.CompositeQuery.Queries, 1)
query, ok := request.CompositeQuery.Queries[0].Spec.(qb.QueryBuilderQuery[qb.TraceAggregation])
require.True(t, ok, "expected a traces builder query")
assert.Len(t, query.Aggregations, 3)
assert.Len(t, query.GroupBy, 2)
require.NoError(t, request.Validate(), "the request built from the panel is not a valid scalar request")
// the panel read back out of storage draws the same scatter plot
stored, err := json.Marshal(spec)
require.NoError(t, err, "marshal dashboard failed")
reread, err := unmarshalDashboard(stored)
require.NoError(t, err, "the stored dashboard does not validate")
assert.Equal(t, panelSpec, reread.Panels["p1"].Spec.Plugin.Spec)
rereadRequest, err := (&DashboardV2{Spec: *reread}).GetPanelQuery(1, 2, "p1")
require.NoError(t, err, "building the stored panel's query failed")
assert.Equal(t, request, rereadRequest)
}
func TestInvalidateScatterPlotPanelSpecValues(t *testing.T) {
tests := []struct {
scenario string
panelKind string
panelSpec string
expectedErrorSubstring string
}{
{
scenario: "unknown axis scale",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"axes": {"x": {"scale": "sqrt"}}}`,
expectedErrorSubstring: "invalid axis scale \"sqrt\": must be `auto`, `linear`, `log`, or `symlog`",
},
{
scenario: "non-string axis scale",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"axes": {"y": {"scale": true}}}`,
expectedErrorSubstring: "cannot unmarshal bool",
},
{
scenario: "isLogScale on a scatter plot axis",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"axes": {"x": {"isLogScale": true}}}`,
expectedErrorSubstring: "unknown field",
},
{
scenario: "single-axis axes shape on a scatter plot panel",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"axes": {"softMin": 0}}`,
expectedErrorSubstring: "unknown field",
},
{
scenario: "unknown threshold axis",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"thresholds": [{"value": 1, "color": "#f00", "axis": "z"}]}`,
expectedErrorSubstring: "invalid threshold axis \"z\": must be `x` or `y`",
},
{
scenario: "unknown dimension",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"dimensions": {"z": "A"}}`,
expectedErrorSubstring: "unknown field",
},
{
scenario: "point size below the minimum",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"chartAppearance": {"points": {"size": 1}}}`,
expectedErrorSubstring: "invalid point diameter 1: must be a whole number between 2 and 40",
},
{
scenario: "max point size above the maximum",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"chartAppearance": {"points": {"maxSize": 41}}}`,
expectedErrorSubstring: "invalid point diameter 41",
},
{
scenario: "fractional point size",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"chartAppearance": {"points": {"minSize": 4.5}}}`,
expectedErrorSubstring: "cannot unmarshal number 4.5",
},
{
scenario: "min point size above max",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"chartAppearance": {"points": {"minSize": 20, "maxSize": 10}}}`,
expectedErrorSubstring: "minSize 20 must not exceed maxSize 10",
},
{
scenario: "point opacity below the floor",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"chartAppearance": {"points": {"opacity": 0.05}}}`,
expectedErrorSubstring: "invalid point opacity 0.05: must be between 0.1 and 1",
},
{
scenario: "point opacity as a percentage",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"chartAppearance": {"points": {"opacity": 70}}}`,
expectedErrorSubstring: "invalid point opacity 70",
},
{
scenario: "area fill fields on a scatter plot",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"chartAppearance": {"fillOpacity": 0.5}}`,
expectedErrorSubstring: "unknown field",
},
{
scenario: "a single colour key as a string",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"dimensions": {"colorBy": "service.name"}}`,
expectedErrorSubstring: "cannot unmarshal string",
},
{
scenario: "panel-wide unit on a scatter plot panel",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"formatting": {"unit": "ms"}}`,
expectedErrorSubstring: "unknown field",
},
{
scenario: "threshold without a color",
panelKind: "signoz/ScatterPlotPanel",
panelSpec: `{"thresholds": [{"value": 300}]}`,
expectedErrorSubstring: "Color' Error:Field validation for 'Color' failed on the 'required' tag",
},
{
scenario: "scale on a time series panel",
panelKind: "signoz/TimeSeriesPanel",
panelSpec: `{"axes": {"scale": "log"}}`,
expectedErrorSubstring: "unknown field",
},
}
for _, test := range tests {
t.Run(test.scenario, func(t *testing.T) {
data := []byte(`{
"variables": [],
"panels": {
"p1": {
"kind": "Panel",
"spec": {
"links": [],
"plugin": {"kind": "` + test.panelKind + `", "spec": ` + test.panelSpec + `},
"queries": [{"kind": "scalar", "spec": {"plugin": {"kind": "signoz/ClickHouseSQL", "spec": {"name": "A", "query": "SELECT 1"}}}}]
}
}
},
"links": [],
"layouts": []
}`)
_, err := unmarshalDashboard(data)
require.Error(t, err, "expected the spec to be rejected")
assert.Contains(t, err.Error(), test.expectedErrorSubstring, "unexpected error message: %s", err.Error())
})
}
}
func TestScatterPlotDimensionsBindToQuery(t *testing.T) {
tests := []struct {
scenario string
queryPlugin string
dimensions string
expectedErrorSubstring string
}{
{
scenario: "a single-aggregation query is one value column keyed by its name",
queryPlugin: `{"kind": "signoz/BuilderQuery", "spec": {"name": "A", "signal": "traces", "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "service.name"}]}}`,
dimensions: `{"x": "A", "colorBy": ["service.name"]}`,
},
{
scenario: "a multi-aggregation query keys each value column by name and expression",
queryPlugin: `{"kind": "signoz/BuilderQuery", "spec": {"name": "A", "signal": "traces", "aggregations": [{"expression": "count()"}, {"expression": "p99(duration_nano)"}]}}`,
dimensions: `{"x": "A"}`,
expectedErrorSubstring: "dimensions.x: \"A\" is not a value column of the panel's query; value columns: `A.count()`, `A.p99(duration_nano)`",
},
{
scenario: "a value column of a query the panel does not have",
queryPlugin: `{"kind": "signoz/BuilderQuery", "spec": {"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}]}}`,
dimensions: `{"y": "B"}`,
expectedErrorSubstring: "dimensions.y: \"B\" is not a value column of the panel's query; value columns: `A`",
},
{
scenario: "a colour key the query does not group by",
queryPlugin: `{"kind": "signoz/BuilderQuery", "spec": {"name": "A", "signal": "traces", "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "service.name"}]}}`,
dimensions: `{"colorBy": ["service.name", "k8s.pod.name"]}`,
expectedErrorSubstring: "dimensions.colorBy: \"k8s.pod.name\" is not a group-by key of the panel's query; group-by keys: `service.name`",
},
{
scenario: "a formula is a value column and carries the labels of the disabled queries it reads",
queryPlugin: `{"kind": "signoz/CompositeQuery", "spec": {"queries": [
{"type": "builder_query", "spec": {"name": "A", "signal": "traces", "disabled": true, "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "service.name"}]}},
{"type": "builder_query", "spec": {"name": "B", "signal": "traces", "disabled": true, "aggregations": [{"expression": "countIf(has_error = true)"}], "groupBy": [{"name": "service.name"}]}},
{"type": "builder_formula", "spec": {"name": "F1", "expression": "B / A"}}
]}}`,
dimensions: `{"x": "F1", "colorBy": ["service.name"]}`,
},
{
scenario: "a disabled query yields no value column",
queryPlugin: `{"kind": "signoz/CompositeQuery", "spec": {"queries": [
{"type": "builder_query", "spec": {"name": "A", "signal": "logs", "disabled": true, "aggregations": [{"expression": "count()"}]}},
{"type": "builder_formula", "spec": {"name": "F1", "expression": "A * 2"}}
]}}`,
dimensions: `{"sizeBy": "A"}`,
expectedErrorSubstring: "dimensions.sizeBy: \"A\" is not a value column of the panel's query; value columns: `F1`",
},
{
scenario: "clickhouse columns are only known at run time, so the bindings go unchecked",
queryPlugin: `{"kind": "signoz/ClickHouseSQL", "spec": {"name": "A", "query": "SELECT 1"}}`,
dimensions: `{"x": "throughput", "colorBy": ["pod"]}`,
},
}
for _, test := range tests {
t.Run(test.scenario, func(t *testing.T) {
data := []byte(`{
"variables": [],
"panels": {
"p1": {
"kind": "Panel",
"spec": {
"links": [],
"plugin": {"kind": "signoz/ScatterPlotPanel", "spec": {"dimensions": ` + test.dimensions + `}},
"queries": [{"kind": "scalar", "spec": {"plugin": ` + test.queryPlugin + `}}]
}
}
},
"links": [],
"layouts": []
}`)
_, err := unmarshalDashboard(data)
if test.expectedErrorSubstring == "" {
require.NoError(t, err)
return
}
require.Error(t, err, "expected the binding to be rejected")
assert.Contains(t, err.Error(), test.expectedErrorSubstring, "unexpected error message: %s", err.Error())
})
}
}
func TestNumberPanelDefaults(t *testing.T) {
data := []byte(`{
"variables": [],
@@ -2190,7 +2503,7 @@ func TestPanelTypeQueryTypeCompatibility(t *testing.T) {
switch panelKind {
case "signoz/ListPanel":
return "raw"
case "signoz/TablePanel", "signoz/NumberPanel", "signoz/PieChartPanel", "signoz/HistogramPanel":
case "signoz/TablePanel", "signoz/NumberPanel", "signoz/PieChartPanel", "signoz/HistogramPanel", "signoz/ScatterPlotPanel":
return "scalar"
default:
return "time_series"
@@ -2232,14 +2545,17 @@ func TestPanelTypeQueryTypeCompatibility(t *testing.T) {
{"TimeSeries+PromQL", mkQuery("signoz/TimeSeriesPanel", "signoz/PromQLQuery", `{"name":"A","query":"up"}`), false},
{"Table+ClickHouse", mkQuery("signoz/TablePanel", "signoz/ClickHouseSQL", `{"name":"A","query":"SELECT 1"}`), false},
{"List+Builder", mkQuery("signoz/ListPanel", "signoz/BuilderQuery", `{"name":"A","signal":"logs"}`), false},
{"ScatterPlot+ClickHouse", mkQuery("signoz/ScatterPlotPanel", "signoz/ClickHouseSQL", `{"name":"A","query":"SELECT 1"}`), false},
// Top-level: rejected
{"Table+PromQL", mkQuery("signoz/TablePanel", "signoz/PromQLQuery", `{"name":"A","query":"up"}`), true},
{"ScatterPlot+PromQL", mkQuery("signoz/ScatterPlotPanel", "signoz/PromQLQuery", `{"name":"A","query":"up"}`), true},
{"List+ClickHouse", mkQuery("signoz/ListPanel", "signoz/ClickHouseSQL", `{"name":"A","query":"SELECT 1"}`), true},
{"List+PromQL", mkQuery("signoz/ListPanel", "signoz/PromQLQuery", `{"name":"A","query":"up"}`), true},
{"List+Composite", mkQuery("signoz/ListPanel", "signoz/CompositeQuery", `{"queries":[]}`), true},
{"List+Formula", mkQuery("signoz/ListPanel", "signoz/Formula", `{"name":"F1","expression":"A+B"}`), true},
// Composite sub-queries
{"Table+Composite(promql)", mkComposite("signoz/TablePanel", "promql", `{"name":"A","query":"up"}`), true},
{"ScatterPlot+Composite(promql)", mkComposite("signoz/ScatterPlotPanel", "promql", `{"name":"A","query":"up"}`), true},
{"Table+Composite(clickhouse)", mkComposite("signoz/TablePanel", "clickhouse_sql", `{"name":"A","query":"SELECT 1"}`), false},
}

View File

@@ -28,16 +28,17 @@ type PanelPlugin struct {
// (and strips the duplicate parent properties) after reflection.
func (PanelPlugin) PrepareJSONSchema(s *jsonschema.Schema) error {
return markDiscriminator(s, "kind", map[string]string{
string(PanelKindTimeSeries): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTimeSeriesPanelSpec"),
string(PanelKindBarChart): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBarChartPanelSpec"),
string(PanelKindAreaChart): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAreaChartPanelSpec"),
string(PanelKindNumber): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesNumberPanelSpec"),
string(PanelKindPieChart): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesPieChartPanelSpec"),
string(PanelKindTable): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTablePanelSpec"),
string(PanelKindHistogram): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHistogramPanelSpec"),
string(PanelKindList): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec"),
string(PanelKindText): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTextPanelSpec"),
string(PanelKindHeatmap): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec"),
string(PanelKindTimeSeries): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTimeSeriesPanelSpec"),
string(PanelKindBarChart): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBarChartPanelSpec"),
string(PanelKindAreaChart): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAreaChartPanelSpec"),
string(PanelKindNumber): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesNumberPanelSpec"),
string(PanelKindPieChart): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesPieChartPanelSpec"),
string(PanelKindTable): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTablePanelSpec"),
string(PanelKindHistogram): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHistogramPanelSpec"),
string(PanelKindList): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec"),
string(PanelKindText): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTextPanelSpec"),
string(PanelKindHeatmap): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec"),
string(PanelKindScatterPlot): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesScatterPlotPanelSpec"),
})
}
@@ -71,6 +72,7 @@ func (PanelPlugin) JSONSchemaOneOf() []any {
PanelPluginVariant[ListPanelSpec]{Kind: string(PanelKindList)},
PanelPluginVariant[TextPanelSpec]{Kind: string(PanelKindText)},
PanelPluginVariant[HeatmapPanelSpec]{Kind: string(PanelKindHeatmap)},
PanelPluginVariant[ScatterPlotPanelSpec]{Kind: string(PanelKindScatterPlot)},
}
}
@@ -227,16 +229,17 @@ func (v VariablePluginVariant[S]) PrepareJSONSchema(s *jsonschema.Schema) error
var (
panelPluginSpecs = map[PanelPluginKind]func() any{
PanelKindTimeSeries: func() any { return new(TimeSeriesPanelSpec) },
PanelKindBarChart: func() any { return new(BarChartPanelSpec) },
PanelKindAreaChart: func() any { return new(AreaChartPanelSpec) },
PanelKindNumber: func() any { return new(NumberPanelSpec) },
PanelKindPieChart: func() any { return new(PieChartPanelSpec) },
PanelKindTable: func() any { return new(TablePanelSpec) },
PanelKindHistogram: func() any { return new(HistogramPanelSpec) },
PanelKindList: func() any { return new(ListPanelSpec) },
PanelKindText: func() any { return new(TextPanelSpec) },
PanelKindHeatmap: func() any { return new(HeatmapPanelSpec) },
PanelKindTimeSeries: func() any { return new(TimeSeriesPanelSpec) },
PanelKindBarChart: func() any { return new(BarChartPanelSpec) },
PanelKindAreaChart: func() any { return new(AreaChartPanelSpec) },
PanelKindNumber: func() any { return new(NumberPanelSpec) },
PanelKindPieChart: func() any { return new(PieChartPanelSpec) },
PanelKindTable: func() any { return new(TablePanelSpec) },
PanelKindHistogram: func() any { return new(HistogramPanelSpec) },
PanelKindList: func() any { return new(ListPanelSpec) },
PanelKindText: func() any { return new(TextPanelSpec) },
PanelKindHeatmap: func() any { return new(HeatmapPanelSpec) },
PanelKindScatterPlot: func() any { return new(ScatterPlotPanelSpec) },
}
queryPluginSpecs = map[QueryPluginKind]func() any{
QueryKindBuilder: func() any { return new(BuilderQuerySpec) },
@@ -252,16 +255,17 @@ var (
VariableKindCustom: func() any { return new(CustomVariableSpec) },
}
allowedQueryKinds = map[PanelPluginKind][]QueryPluginKind{
PanelKindTimeSeries: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindBarChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindAreaChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindNumber: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindHistogram: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindPieChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindTable: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindList: {QueryKindBuilder},
PanelKindText: {},
PanelKindHeatmap: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindTimeSeries: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindBarChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindAreaChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindNumber: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindHistogram: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindPieChart: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindTable: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
PanelKindList: {QueryKindBuilder},
PanelKindText: {},
PanelKindHeatmap: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindScatterPlot: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindTraceOperator, QueryKindClickHouseSQL},
}
)

View File

@@ -214,7 +214,7 @@ func (d *DashboardV2) GetPanelQuery(startTime, endTime uint64, panelKey string)
CompositeQuery: composite,
FormatOptions: &qb.FormatOptions{
FillGaps: fillGaps,
FormatTableResultForUI: panel.Spec.Plugin.Kind == PanelKindTable,
FormatTableResultForUI: panel.Spec.Plugin.Kind == PanelKindTable || panel.Spec.Plugin.Kind == PanelKindScatterPlot,
},
}, nil
}

View File

@@ -209,34 +209,38 @@ func TestDashboardV2GetPanelQuery(t *testing.T) {
}
})
t.Run("sets FormatTableResultForUI only for table panels", func(t *testing.T) {
dashboard := &DashboardV2{
Spec: DashboardSpec{
Panels: map[string]*Panel{
"panel-1": {
Spec: PanelSpec{
Plugin: PanelPlugin{Kind: PanelKindTable},
Queries: []Query{
{
Kind: qb.RequestTypeScalar,
Spec: QuerySpec{
Plugin: QueryPlugin{
Kind: QueryKindBuilder,
Spec: &BuilderQuerySpec{Spec: qb.QueryBuilderQuery[qb.MetricAggregation]{Name: "A"}},
t.Run("sets FormatTableResultForUI for panels that join scalar rows", func(t *testing.T) {
for _, kind := range []PanelPluginKind{PanelKindTable, PanelKindScatterPlot} {
t.Run(string(kind), func(t *testing.T) {
dashboard := &DashboardV2{
Spec: DashboardSpec{
Panels: map[string]*Panel{
"panel-1": {
Spec: PanelSpec{
Plugin: PanelPlugin{Kind: kind},
Queries: []Query{
{
Kind: qb.RequestTypeScalar,
Spec: QuerySpec{
Plugin: QueryPlugin{
Kind: QueryKindBuilder,
Spec: &BuilderQuerySpec{Spec: qb.QueryBuilderQuery[qb.MetricAggregation]{Name: "A"}},
},
},
},
},
},
},
},
},
},
},
}
}
req, err := dashboard.GetPanelQuery(1, 2, "panel-1")
require.NoError(t, err)
require.NotNil(t, req.FormatOptions)
assert.True(t, req.FormatOptions.FormatTableResultForUI)
req, err := dashboard.GetPanelQuery(1, 2, "panel-1")
require.NoError(t, err)
require.NotNil(t, req.FormatOptions)
assert.True(t, req.FormatOptions.FormatTableResultForUI)
})
}
})
t.Run("sets FillGaps from the panel visualization", func(t *testing.T) {

View File

@@ -2,6 +2,8 @@ package dashboardtypes
import (
"encoding/json"
"maps"
"slices"
"strconv"
"github.com/SigNoz/signoz/pkg/errors"
@@ -166,20 +168,21 @@ func (BuilderQuerySpec) JSONSchemaOneOf() []any {
type PanelPluginKind string
const (
PanelKindTimeSeries PanelPluginKind = "signoz/TimeSeriesPanel"
PanelKindBarChart PanelPluginKind = "signoz/BarChartPanel"
PanelKindAreaChart PanelPluginKind = "signoz/AreaChartPanel"
PanelKindNumber PanelPluginKind = "signoz/NumberPanel"
PanelKindPieChart PanelPluginKind = "signoz/PieChartPanel"
PanelKindTable PanelPluginKind = "signoz/TablePanel"
PanelKindHistogram PanelPluginKind = "signoz/HistogramPanel"
PanelKindList PanelPluginKind = "signoz/ListPanel"
PanelKindText PanelPluginKind = "signoz/TextPanel"
PanelKindHeatmap PanelPluginKind = "signoz/HeatmapPanel"
PanelKindTimeSeries PanelPluginKind = "signoz/TimeSeriesPanel"
PanelKindBarChart PanelPluginKind = "signoz/BarChartPanel"
PanelKindAreaChart PanelPluginKind = "signoz/AreaChartPanel"
PanelKindNumber PanelPluginKind = "signoz/NumberPanel"
PanelKindPieChart PanelPluginKind = "signoz/PieChartPanel"
PanelKindTable PanelPluginKind = "signoz/TablePanel"
PanelKindHistogram PanelPluginKind = "signoz/HistogramPanel"
PanelKindList PanelPluginKind = "signoz/ListPanel"
PanelKindText PanelPluginKind = "signoz/TextPanel"
PanelKindHeatmap PanelPluginKind = "signoz/HeatmapPanel"
PanelKindScatterPlot PanelPluginKind = "signoz/ScatterPlotPanel"
)
func (PanelPluginKind) Enum() []any {
return []any{PanelKindTimeSeries, PanelKindBarChart, PanelKindAreaChart, PanelKindNumber, PanelKindPieChart, PanelKindTable, PanelKindHistogram, PanelKindList, PanelKindText, PanelKindHeatmap}
return []any{PanelKindTimeSeries, PanelKindBarChart, PanelKindAreaChart, PanelKindNumber, PanelKindPieChart, PanelKindTable, PanelKindHistogram, PanelKindList, PanelKindText, PanelKindHeatmap, PanelKindScatterPlot}
}
func (k PanelPluginKind) rendersWithoutQuery() bool {
@@ -329,6 +332,95 @@ type TextPanelSpec struct {
HeaderOptions HeaderOptions `json:"headerOptions"`
}
type ScatterPlotPanelSpec struct {
Visualization BasicVisualization `json:"visualization"`
Dimensions ScatterPlotDimensions `json:"dimensions"`
Formatting TableFormatting `json:"formatting"`
Axes ScatterPlotAxes `json:"axes"`
ChartAppearance ScatterPlotChartAppearance `json:"chartAppearance"`
Legend Legend `json:"legend"`
Thresholds []ScatterPlotThreshold `json:"thresholds" validate:"dive"`
}
func (s *ScatterPlotPanelSpec) validateAgainstQuery(query Query, path string) error {
composite, err := query.Spec.Plugin.buildV5CompositeQueryFromPlugin()
if err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "%s.dimensions: %s", path, err.Error())
}
keys, known := composite.CollectResultColumnKeys()
if !known {
return nil
}
for _, binding := range []struct{ field, key string }{{"x", s.Dimensions.X}, {"y", s.Dimensions.Y}, {"sizeBy", s.Dimensions.SizeBy}} {
// Empty is unset: the renderer picks a column or a fixed size.
if binding.key == "" {
continue
}
if _, ok := keys.Aggregations[binding.key]; !ok {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s.dimensions.%s: %q is not a value column of the panel's query; value columns: %s", path, binding.field, binding.key, allowedValuesForKind(slices.Sorted(maps.Keys(keys.Aggregations))))
}
}
for _, label := range s.Dimensions.ColorBy {
if _, ok := keys.Groups[label]; !ok {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s.dimensions.colorBy: %q is not a group-by key of the panel's query; group-by keys: %s", path, label, allowedValuesForKind(slices.Sorted(maps.Keys(keys.Groups))))
}
}
return nil
}
// ScatterPlotDimensions binds result columns to what a dot encodes. Value columns
// use the Table column key, so formatting.columnUnits carries over between the
// two kinds.
type ScatterPlotDimensions struct {
X string `json:"x" description:"Value column key (queryName, or queryName.expression for a multi-aggregation query) plotted on the x axis. Empty uses the first value column."`
Y string `json:"y" description:"Value column key plotted on the y axis. Empty uses the second value column."`
SizeBy string `json:"sizeBy" description:"Value column key that scales dot size. Empty draws every dot at the default size."`
ColorBy []string `json:"colorBy" description:"Group-by label names (e.g. k8s.namespace.name) whose combined values colour dots and drive the legend. Empty colours by every group-by label."`
}
type ScatterPlotAxes struct {
X ScatterPlotAxis `json:"x"`
Y ScatterPlotAxis `json:"y"`
}
type ScatterPlotAxis struct {
SoftMin *float64 `json:"softMin"`
SoftMax *float64 `json:"softMax"`
Scale AxisScale `json:"scale"`
Label string `json:"label" description:"Axis title. Empty draws none."`
}
type ScatterPlotChartAppearance struct {
Points ScatterPlotPoints `json:"points"`
}
// ScatterPlotPoints keeps the fixed size and the size range side by side so
// binding or unbinding dimensions.sizeBy restores the other's last setting. Nil
// fields resolve to the renderer default.
type ScatterPlotPoints struct {
Size *PointDiameter `json:"size" description:"Diameter of every dot when dimensions.sizeBy is unset."`
MinSize *PointDiameter `json:"minSize" description:"Diameter of the smallest dot when dimensions.sizeBy is set."`
MaxSize *PointDiameter `json:"maxSize" description:"Diameter of the largest dot when dimensions.sizeBy is set."`
Opacity *PointOpacity `json:"opacity"`
}
func (p *ScatterPlotPoints) UnmarshalJSON(data []byte) error {
type alias ScatterPlotPoints
var tmp alias
if err := json.Unmarshal(data, &tmp); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid points")
}
*p = ScatterPlotPoints(tmp)
return p.validate()
}
func (p ScatterPlotPoints) validate() error {
if p.MinSize != nil && p.MaxSize != nil && *p.MinSize > *p.MaxSize {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid points: minSize %d must not exceed maxSize %d", *p.MinSize, *p.MaxSize)
}
return nil
}
type TextPresentation struct {
TextAlign TextAlign `json:"textAlign"`
VerticalAlign VerticalAlign `json:"verticalAlign"`
@@ -410,6 +502,11 @@ type ComparisonThreshold struct {
Format ThresholdFormat `json:"format"`
}
type ScatterPlotThreshold struct {
ThresholdWithLabel
Axis ThresholdAxis `json:"axis"`
}
type TableThreshold struct {
ComparisonThreshold
ColumnName string `json:"columnName" validate:"required" required:"true"`
@@ -695,6 +792,86 @@ func (ls *LineStyle) UnmarshalJSON(data []byte) error {
}
}
// AxisScale `auto` leaves the choice to the renderer; `symlog` is log-like but
// defined at and below zero.
type AxisScale struct{ valuer.String }
var (
AxisScaleAuto = AxisScale{valuer.NewString("auto")} // default
AxisScaleLinear = AxisScale{valuer.NewString("linear")}
AxisScaleLog = AxisScale{valuer.NewString("log")}
AxisScaleSymlog = AxisScale{valuer.NewString("symlog")}
)
func (AxisScale) Enum() []any {
return []any{AxisScaleAuto, AxisScaleLinear, AxisScaleLog, AxisScaleSymlog}
}
func (as AxisScale) ValueOrDefault() string {
if as.IsZero() {
return AxisScaleAuto.StringValue()
}
return as.StringValue()
}
func (as AxisScale) MarshalJSON() ([]byte, error) {
return json.Marshal(as.ValueOrDefault())
}
func (as *AxisScale) UnmarshalJSON(data []byte) error {
var v string
if err := json.Unmarshal(data, &v); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid axis scale: must be a string, one of `auto`, `linear`, `log`, or `symlog`")
}
val := AxisScale{valuer.NewString(v)}
switch val {
case AxisScaleAuto, AxisScaleLinear, AxisScaleLog, AxisScaleSymlog:
*as = val
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid axis scale %q: must be `auto`, `linear`, `log`, or `symlog`", v)
}
}
// ThresholdAxis is the axis a scatter plot threshold line marks: `y` draws it
// horizontally, `x` vertically.
type ThresholdAxis struct{ valuer.String }
var (
ThresholdAxisX = ThresholdAxis{valuer.NewString("x")}
ThresholdAxisY = ThresholdAxis{valuer.NewString("y")} // default
)
func (ThresholdAxis) Enum() []any {
return []any{ThresholdAxisX, ThresholdAxisY}
}
func (ta ThresholdAxis) ValueOrDefault() string {
if ta.IsZero() {
return ThresholdAxisY.StringValue()
}
return ta.StringValue()
}
func (ta ThresholdAxis) MarshalJSON() ([]byte, error) {
return json.Marshal(ta.ValueOrDefault())
}
func (ta *ThresholdAxis) UnmarshalJSON(data []byte) error {
var v string
if err := json.Unmarshal(data, &v); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid threshold axis: must be a string, one of `x` or `y`")
}
val := ThresholdAxis{valuer.NewString(v)}
switch val {
case ThresholdAxisX, ThresholdAxisY:
*ta = val
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid threshold axis %q: must be `x` or `y`", v)
}
}
type FillMode struct{ valuer.String }
var (
@@ -833,6 +1010,53 @@ func (o *FillOpacity) UnmarshalJSON(data []byte) error {
return nil
}
const (
minPointDiameter = 2
maxPointDiameter = 40
)
// PointDiameter is a scatter dot's diameter in whole CSS pixels.
type PointDiameter int
func (PointDiameter) PrepareJSONSchema(s *jsonschema.Schema) error {
s.WithMinimum(minPointDiameter).WithMaximum(maxPointDiameter)
return nil
}
func (d *PointDiameter) UnmarshalJSON(data []byte) error {
var v int
if err := json.Unmarshal(data, &v); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid point diameter: must be a whole number between %d and %d", minPointDiameter, maxPointDiameter)
}
if v < minPointDiameter || v > maxPointDiameter {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid point diameter %d: must be a whole number between %d and %d", v, minPointDiameter, maxPointDiameter)
}
*d = PointDiameter(v)
return nil
}
const minPointOpacity = 0.1
// PointOpacity is a scatter dot's fill alpha. The floor keeps a dot visible.
type PointOpacity float64
func (PointOpacity) PrepareJSONSchema(s *jsonschema.Schema) error {
s.WithMinimum(minPointOpacity).WithMaximum(1)
return nil
}
func (o *PointOpacity) UnmarshalJSON(data []byte) error {
var v float64
if err := json.Unmarshal(data, &v); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid point opacity: must be a number between %v and 1", minPointOpacity)
}
if v < minPointOpacity || v > 1 {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid point opacity %v: must be between %v and 1", v, minPointOpacity)
}
*o = PointOpacity(v)
return nil
}
type SpanGaps struct {
FillOnlyBelow bool `json:"fillOnlyBelow" description:"Controls whether lines connect across null values. When false (default), all gaps are connected. When true, only gaps smaller than fillLessThan are connected."`
FillLessThan string `json:"fillLessThan" description:"The maximum gap size to connect when fillOnlyBelow is true. Gaps larger than this duration are left disconnected."`

View File

@@ -0,0 +1,134 @@
{
"display": {
"name": "service health",
"description": "throughput against tail latency per pod"
},
"variables": [],
"panels": {
"p1": {
"kind": "Panel",
"spec": {
"display": {
"name": "throughput vs p99",
"description": ""
},
"plugin": {
"kind": "signoz/ScatterPlotPanel",
"spec": {
"visualization": {
"timePreference": "last_1_hr"
},
"dimensions": {
"x": "A.count()",
"y": "A.p99(duration_nano)",
"sizeBy": "A.countIf(has_error = true)",
"colorBy": ["k8s.namespace.name", "k8s.pod.name"]
},
"formatting": {
"columnUnits": {
"A.p99(duration_nano)": "ns"
},
"decimalPrecision": "1"
},
"axes": {
"x": {
"softMin": 0,
"softMax": null,
"scale": "log",
"label": "Throughput"
},
"y": {
"softMin": null,
"softMax": 1000,
"scale": "symlog",
"label": "p99 latency"
}
},
"chartAppearance": {
"points": {
"size": 8,
"minSize": 4,
"maxSize": 24,
"opacity": 0.1
}
},
"legend": {
"position": "right",
"mode": "list",
"customColors": {
"checkout": "#1f77b4"
}
},
"thresholds": [
{
"value": 300,
"unit": "ms",
"color": "#f00",
"label": "p99 SLO",
"axis": "y"
},
{
"value": 50,
"color": "#0f0",
"axis": "x"
}
]
}
},
"links": [],
"queries": [
{
"kind": "scalar",
"spec": {
"plugin": {
"kind": "signoz/BuilderQuery",
"spec": {
"name": "A",
"signal": "traces",
"aggregations": [
{
"expression": "count()"
},
{
"expression": "p99(duration_nano)"
},
{
"expression": "countIf(has_error = true)"
}
],
"groupBy": [
{
"name": "k8s.namespace.name"
},
{
"name": "k8s.pod.name"
}
]
}
}
}
}
]
}
}
},
"links": [],
"layouts": [
{
"kind": "Grid",
"spec": {
"items": [
{
"x": 0,
"y": 0,
"width": 12,
"height": 8,
"content": {
"$ref": "#/spec/panels/p1"
}
}
]
}
}
]
}

View File

@@ -122,6 +122,9 @@ type PipelineOperator struct {
EnablePaths bool `json:"enable_paths,omitempty" yaml:"enable_paths,omitempty"`
PathPrefix string `json:"path_prefix,omitempty" yaml:"path_prefix,omitempty"`
// normalize fields, set by the server only
JSONBodyDualIngestion bool `json:"-" yaml:"json_body_dual_ingestion,omitempty"`
// Used in Severity Parsing and JSON Flattening mapping
Mapping map[string][]string `json:"mapping,omitempty" yaml:"mapping,omitempty"`
// severity parser fields

View File

@@ -266,6 +266,43 @@ type CompositeQuery struct {
Queries []QueryEnvelope `json:"queries"`
}
// ResultColumnKeys is what a scalar query yields per row, keyed the way the
// UI keys columns: an aggregation column is the query name, or
// queryName.expression when the query has several aggregations; a group
// column is a group-by key name.
type ResultColumnKeys struct {
Aggregations map[string]struct{}
Groups map[string]struct{}
}
// CollectResultColumnKeys reports false when a query's columns are only known
// at run time, as for ClickHouse SQL and PromQL. A disabled query yields no
// aggregation column but keeps its group columns, since a formula over it
// still carries them.
func (c *CompositeQuery) CollectResultColumnKeys() (ResultColumnKeys, bool) {
keys := ResultColumnKeys{Aggregations: map[string]struct{}{}, Groups: map[string]struct{}{}}
for _, envelope := range c.Queries {
if envelope.GetType() == QueryTypeClickHouseSQL || envelope.GetType() == QueryTypePromQL {
return ResultColumnKeys{}, false
}
for _, key := range envelope.GetGroupBy() {
keys.Groups[key.Name] = struct{}{}
}
if envelope.IsDisabled() {
continue
}
expressions := envelope.GetAggregationExpressions()
if len(expressions) <= 1 {
keys.Aggregations[envelope.GetQueryName()] = struct{}{}
continue
}
for _, expression := range expressions {
keys.Aggregations[envelope.GetQueryName()+"."+expression] = struct{}{}
}
}
return keys, true
}
// PrepareJSONSchema adds description to the CompositeQuery schema.
func (c *CompositeQuery) PrepareJSONSchema(schema *jsonschema.Schema) error {
schema.WithDescription("Composite query containing one or more query envelopes. Each query envelope specifies its type and corresponding spec.")

View File

@@ -225,6 +225,34 @@ func (q *QueryEnvelope) GetGroupBy() []GroupByKey {
return nil
}
// GetAggregationExpressions returns one entry per aggregation. A metric
// aggregation has no expression, so it contributes an empty entry.
func (q *QueryEnvelope) GetAggregationExpressions() []string {
switch spec := q.Spec.(type) {
case QueryBuilderTraceOperator:
expressions := make([]string, len(spec.Aggregations))
for i, aggregation := range spec.Aggregations {
expressions[i] = aggregation.Expression
}
return expressions
case QueryBuilderQuery[TraceAggregation]:
expressions := make([]string, len(spec.Aggregations))
for i, aggregation := range spec.Aggregations {
expressions[i] = aggregation.Expression
}
return expressions
case QueryBuilderQuery[LogAggregation]:
expressions := make([]string, len(spec.Aggregations))
for i, aggregation := range spec.Aggregations {
expressions[i] = aggregation.Expression
}
return expressions
case QueryBuilderQuery[MetricAggregation]:
return make([]string, len(spec.Aggregations))
}
return nil
}
// GetFilter returns the filter.
func (q *QueryEnvelope) GetFilter() *Filter {
switch spec := q.Spec.(type) {

View File

@@ -1903,3 +1903,63 @@ func TestQueryRangeRequest_StepIntervalForQuery(t *testing.T) {
})
}
}
func TestCompositeQueryCollectResultColumnKeys(t *testing.T) {
testCases := []struct {
scenario string
composite CompositeQuery
expectedKeys ResultColumnKeys
expectedKnown bool
}{
{
scenario: "a single aggregation is keyed by the query name",
composite: CompositeQuery{Queries: []QueryEnvelope{{Type: QueryTypeBuilder, Spec: QueryBuilderQuery[TraceAggregation]{
Name: "A",
Aggregations: []TraceAggregation{{Expression: "count()"}},
GroupBy: []GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "service.name"}}},
}}}},
expectedKeys: ResultColumnKeys{Aggregations: map[string]struct{}{"A": {}}, Groups: map[string]struct{}{"service.name": {}}},
expectedKnown: true,
},
{
scenario: "several aggregations are keyed by query name and expression",
composite: CompositeQuery{Queries: []QueryEnvelope{{Type: QueryTypeBuilder, Spec: QueryBuilderQuery[LogAggregation]{
Name: "A",
Aggregations: []LogAggregation{{Expression: "count()"}, {Expression: "p99(duration_nano)"}},
}}}},
expectedKeys: ResultColumnKeys{Aggregations: map[string]struct{}{"A.count()": {}, "A.p99(duration_nano)": {}}, Groups: map[string]struct{}{}},
expectedKnown: true,
},
{
scenario: "a disabled query yields no aggregation but keeps its groups for the formula over it",
composite: CompositeQuery{Queries: []QueryEnvelope{
{Type: QueryTypeBuilder, Spec: QueryBuilderQuery[MetricAggregation]{
Name: "A",
Disabled: true,
Aggregations: []MetricAggregation{{MetricName: "http.server.request.count"}},
GroupBy: []GroupByKey{{TelemetryFieldKey: telemetrytypes.TelemetryFieldKey{Name: "service.name"}}},
}},
{Type: QueryTypeFormula, Spec: QueryBuilderFormula{Name: "F1", Expression: "A * 2"}},
}},
expectedKeys: ResultColumnKeys{Aggregations: map[string]struct{}{"F1": {}}, Groups: map[string]struct{}{"service.name": {}}},
expectedKnown: true,
},
{
scenario: "clickhouse columns are only known at run time",
composite: CompositeQuery{Queries: []QueryEnvelope{{Type: QueryTypeClickHouseSQL, Spec: ClickHouseQuery{
Name: "A",
Query: "SELECT 1",
}}}},
expectedKeys: ResultColumnKeys{},
expectedKnown: false,
},
}
for _, testCase := range testCases {
t.Run(testCase.scenario, func(t *testing.T) {
keys, known := testCase.composite.CollectResultColumnKeys()
assert.Equal(t, testCase.expectedKnown, known)
assert.Equal(t, testCase.expectedKeys, keys)
})
}
}

View File

@@ -311,15 +311,14 @@ def create_json_index(
) -> Generator[Callable[[str, list[dict[str, Any]]], None]]:
"""
Create ClickHouse data-skipping indexes on body_v2 JSON sub-columns via
POST /api/v1/promoted_paths.
POST /api/v1/logs/promote_paths.
**Must be called BEFORE insert_logs** so that newly inserted data parts are
covered by the index and the QB uses the indexed condition path.
Each entry in `paths` follows the PromotePath API shape (signal/context are
filled in as logs/body):
Each entry in `paths` follows the PromotePath API shape:
{
"path": "user.name", # bare path, no "body." prefix
"path": "body.user.name", # must start with "body."
"indexes": [
{
"fieldDataType": "string", # string | int64 | float64
@@ -338,9 +337,9 @@ def create_json_index(
token = get_token(...)
export_json_types(logs_list)
create_json_body_index(token, [
{"path": "user.name",
{"path": "body.user.name",
"indexes": [{"fieldDataType": "string", "type": "ngrambf_v1(3, 256, 2, 0)", "granularity": 1}]},
{"path": "user.age",
{"path": "body.user.age",
"indexes": [{"fieldDataType": "int64", "type": "minmax", "granularity": 1}]},
])
insert_logs(logs_list) # data inserted after index exists — index is built automatically
@@ -349,15 +348,18 @@ def create_json_index(
def _create_json_body_index(token: str, paths: list[dict[str, Any]]) -> None:
response = requests.post(
signoz.self.host_configs["8080"].get("/api/v1/promoted_paths"),
signoz.self.host_configs["8080"].get("/api/v1/logs/promote_paths"),
headers={"authorization": f"Bearer {token}"},
json=[{"signal": "logs", "context": "body", **path} for path in paths],
json=paths,
timeout=30,
)
assert response.status_code == HTTPStatus.CREATED, f"Failed to create JSON body indexes: {response.status_code} {response.text}"
for path in paths:
if path["path"] not in created_paths:
created_paths.append(path["path"])
# The API strips the "body." prefix before storing — mirror that here
# so our cleanup query uses the bare path (e.g. "user.name").
raw = path["path"].removeprefix("body.")
if raw not in created_paths:
created_paths.append(raw)
yield _create_json_body_index

View File

@@ -404,7 +404,7 @@ def test_indexed_paths(
token,
[
{
"path": "user.raw-data.name",
"path": "body.user.raw-data.name",
"indexes": [
{
"fieldDataType": "string",
@@ -414,7 +414,7 @@ def test_indexed_paths(
],
},
{
"path": "user.raw-data.age",
"path": "body.user.raw-data.age",
"indexes": [
{
"fieldDataType": "int64",