Compare commits

..

24 Commits

Author SHA1 Message Date
Abhi Kumar
7ce8285091 Merge remote-tracking branch 'origin/main' into feat/geo-map-panel-spec
# Conflicts:
#	docs/api/openapi.yml
#	pkg/types/dashboardtypes/perses_dashboard_test.go
#	pkg/types/dashboardtypes/perses_plugin_wrappers.go
#	pkg/types/dashboardtypes/perses_public_dashboard.go
#	pkg/types/dashboardtypes/perses_public_dashboard_panel_query_test.go
#	pkg/types/dashboardtypes/perses_signoz_plugins.go
2026-11-03 10:14:51 +05:30
Abhi Kumar
e06b07788f chore(openapi): regenerate specs 2026-11-02 16:10:54 +05:30
Abhi Kumar
8e63624de8 feat(dashboards): join geo map results on public dashboards 2026-11-02 16:10:54 +05:30
Abhi Kumar
4c815e0224 feat(dashboards): add plugin schema for geo map panel 2026-11-02 16:10:54 +05:30
Abhi Kumar
223b64e5af feat(dashboard): let scatter plot values take a legend
Legend is pinned so it opens empty instead of seeding a group-by template, which
would name the value literally. Value labels take the query-name prefix only when
two value columns share a name.
2026-11-02 16:10:36 +05:30
Abhi kumar
10f4364abc fix(dashboards): scatter plot save validation fixes (#13329)
#### Description

Stacked on #13030. Addresses its two open Greptile findings.

- Editing a query no longer blocks saving: scatter
`x`/`y`/`sizeBy`/`colorBy` bindings the query no longer produces are
cleared on save, so the saved panel matches the preview's fallback
columns. Done through a new optional `prepareSaveSpec` hook on query
panel definitions.
- `points.minSize` / `maxSize` are compared at their renderer defaults
(4 / 24) when one is null, so `{"maxSize": 2}` or `{"minSize": 40}` can
no longer invert the size range.

#### Issues closed by this PR

Closes SigNoz/pulse-pod#660
2026-11-02 12:02:00 +05:30
Naman Verma
3aa53d67e3 test: follow test guidelines 2026-10-30 20:00:22 +05:30
Abhi Kumar
8f87ed3ed4 chore(api): regenerate frontend api client 2026-10-30 17:15:43 +05:30
Abhi kumar
1161c249f2 Merge branch 'main' into feat/scatter-plot-panel-spec 2026-10-30 17:01:48 +05:30
Abhi kumar
3778d23a41 feat(charts): add the scatter chart layer (#13050)
#### Description

- Adds the scatter chart layer: a uPlot faceted-mode (`mode: 2`)
`ScatterPlugin` with quadtree hover/click hit-testing and a
hovered-point outline, plus a `Scatter` chart with its own tooltip.
- Per-axis scales `auto` | `linear` | `log` | `symlog` and titles.
`auto` picks log (symlog with zero or negatives) when it spreads the
middle half of the points at least 2× more than linear, over ≥ 2
decades.
- Log axes tick at 1/2/5 × 10ⁿ since the scatter range is left
unsnapped; dots stay whole at plot edges.
- Click handlers report the scatter point hit; `buildClickData` skips x
lookups on faceted plots and `ChartWrapper` forwards `plotRef`.

#### Additional Information

- Shared uPlotV2 changes affecting every chart: axes now draw an axis
line in the grid's style (off when the grid is hidden), and value-axis
tick labels outside the scale's range are blanked.
- New shared config: `PlotMode`, `DrawStyle.Scatter`,
`DistributionType.SymmetricLog`.
- Chart library and Storybook stories only; the dashboard panel lands in
#13051.
2026-10-30 17:01:17 +05:30
Naman Verma
b9104596ca fix: add dot in comment 2026-10-30 16:03:46 +05:30
Naman Verma
e9ab776ced chore: add decsriptions and todo 2026-10-30 13:25:09 +05:30
Naman Verma
eaa922cabe fix: dont allow trace operator 2026-10-30 12:27:10 +05:30
Abhi Kumar
f64bc495cc fix(dashboards): key metric aggregations by query name and reject unknown point fields 2026-10-30 12:27:10 +05:30
Naman Verma
6a42b9af90 feat: add validation to ensure queries have same group by 2026-10-30 12:27:10 +05:30
Abhi Kumar
517cba0bee feat(dashboards): add an axis to scatter plot thresholds 2026-10-30 12:27:10 +05:30
Naman Verma
118ccb8f12 chore: add more validations for scatter plot 2026-10-30 12:27:10 +05:30
Abhi Kumar
3bd710deb7 feat(dashboards): rename scatter plot dimensions to colorBy and sizeBy 2026-10-30 12:27:10 +05:30
Abhi Kumar
44e31d7dd8 chore(openapi): regenerate specs for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-30 12:27:10 +05:30
Abhi Kumar
5d411a8221 feat(dashboards): add point size and opacity to scatter plot spec
The fixed size and the min/max range are both stored so binding or unbinding
the size dimension restores the other's last setting.

Assisted-by: Claude Opus 5.5
2026-10-30 12:27:10 +05:30
Abhi Kumar
effb7df6fa chore(openapi): regenerate specs for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-30 12:27:10 +05:30
Abhi Kumar
12a0c3fd64 feat(dashboards): add axis labels and multi-key colour to scatter plot spec
An empty colour list colours by every group-by label.

Assisted-by: Claude Opus 5.5
2026-10-30 12:27:10 +05:30
Abhi Kumar
d34b7ee637 chore(openapi): regenerate specs for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-30 12:27:10 +05:30
Abhi Kumar
a802e1422e feat(dashboards): add plugin schema for scatter plot panel
Assisted-by: Claude Opus 5.5
2026-10-30 12:27:10 +05:30
17 changed files with 929 additions and 675 deletions

View File

@@ -4283,6 +4283,11 @@ components:
last_6_hr, last_1_day, last_3_days, last_1_week or last_1_month pin a
fixed window.'
type: object
DashboardtypesBubbleRadius:
maximum: 64
minimum: 1
nullable: true
type: integer
DashboardtypesBuilderQuerySpec:
discriminator:
mapping:
@@ -4515,6 +4520,231 @@ components:
minimum: 0
nullable: true
type: number
DashboardtypesGeoMapBubbles:
properties:
maxRadius:
allOf:
- $ref: '#/components/schemas/DashboardtypesBubbleRadius'
description: Radius of the largest circle, in CSS pixels from 1 to 64. Null
means 8 on the points layer, 28 otherwise.
minRadius:
allOf:
- $ref: '#/components/schemas/DashboardtypesBubbleRadius'
description: Radius of the smallest circle, in CSS pixels from 1 to 64.
Null means 2 on the points layer, 3 otherwise.
type: object
DashboardtypesGeoMapChartAppearance:
properties:
bubbles:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapBubbles'
description: The circle radius range of the bubbles and points layers.
colors:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapColors'
description: The color scale applied to the colorBy values.
labels:
description: Shows country and state names once each fits inside its shape.
Null shows them.
nullable: true
type: boolean
type: object
DashboardtypesGeoMapColors:
properties:
fill:
description: The hex color varied in opacity mode. Empty uses the default
fill.
type: string
max:
description: The value mapped to the highest color. Null uses the highest
value in the data.
nullable: true
type: number
min:
description: The value mapped to the lowest color. Null uses the lowest
value in the data. Must not exceed max.
nullable: true
type: number
mode:
allOf:
- $ref: '#/components/schemas/DashboardtypesHeatmapColorMode'
description: 'How values are colored: palette (default) picks a hue from
palette, opacity varies the opacity of fill.'
palette:
allOf:
- $ref: '#/components/schemas/DashboardtypesHeatmapPalette'
description: The named color ramp used in palette mode, such as lava (default),
ice or ember.
scale:
allOf:
- $ref: '#/components/schemas/DashboardtypesHeatmapColorScale'
description: 'How values map onto the color range: log (default), sqrt or
linear.'
steps:
description: The number of distinct color steps, 2 to 128. 0 uses the chart
default.
type: integer
type: object
DashboardtypesGeoMapCountryFormat:
enum:
- auto
- alpha2
- alpha3
- name
type: string
DashboardtypesGeoMapLayer:
enum:
- regions
- bubbles
- points
type: string
DashboardtypesGeoMapLocation:
properties:
country:
description: Group-by key holding the country, for the country and subdivision
modes. Empty detects it by name, such as geo.country.iso_code.
type: string
countryFormat:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapCountryFormat'
description: 'How country values are written: auto (default) detects it,
alpha2, alpha3 or name.'
lat:
description: Group-by key holding the latitude, for the coordinates mode.
Empty detects it by name.
type: string
lon:
description: Group-by key holding the longitude, for the coordinates mode.
Empty detects it by name.
type: string
mode:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapLocationMode'
description: 'How rows are placed: auto (default) detects it from the group-by
keys, country, subdivision or coordinates.'
region:
description: Group-by key holding the ISO 3166-2 region code (e.g. geo.region.iso_code),
for the subdivision mode. Empty detects it by name.
type: string
required:
- mode
- countryFormat
type: object
DashboardtypesGeoMapLocationMode:
enum:
- auto
- country
- subdivision
- coordinates
type: string
DashboardtypesGeoMapPanelSpec:
properties:
chartAppearance:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapChartAppearance'
description: Color scale, bubble size and labels.
formatting:
allOf:
- $ref: '#/components/schemas/DashboardtypesTableFormatting'
description: Per-column units and decimal precision.
legend:
allOf:
- $ref: '#/components/schemas/DashboardtypesLegend'
description: Legend placement.
location:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapLocation'
description: The group-by columns that place each row on the map.
map:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapSettings'
description: Layer, view, projection and the value columns drawn.
thresholds:
description: Rules that recolor a shape, bubble or point when its value
compares to a threshold as specified.
items:
$ref: '#/components/schemas/DashboardtypesGeoMapThreshold'
nullable: true
type: array
visualization:
allOf:
- $ref: '#/components/schemas/DashboardtypesBasicVisualization'
description: The time range the values are computed over.
type: object
DashboardtypesGeoMapProjection:
enum:
- mercator
- equal_earth
- natural_earth
type: string
DashboardtypesGeoMapSettings:
properties:
colorBy:
description: Value column key, as in a Table panel, that sets the color.
Empty uses the first value column.
type: string
layer:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapLayer'
description: 'What is drawn: regions (default) fills each shape, bubbles
draws a circle per region, points a circle per coordinate.'
projection:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapProjection'
description: 'The map projection: mercator (default), equal_earth or natural_earth.'
sizeBy:
description: Value column key, as in a Table panel, that sets the bubble
or point size. Empty uses colorBy.
type: string
view:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapView'
description: 'The area shown: world (default), fit to the data, a continent,
or country with viewCountry.'
viewCountry:
description: Uppercase ISO 3166-1 alpha-2 code of the country the country
view shows.
type: string
required:
- layer
- view
- projection
type: object
DashboardtypesGeoMapThreshold:
properties:
color:
description: 'The CSS color of the shape, bubble or point while the threshold
is active, for example #F1575F; CSS named colors such as red also work.'
type: string
operator:
allOf:
- $ref: '#/components/schemas/DashboardtypesComparisonOperator'
description: 'The comparison that activates the threshold: above (default),
below, above_or_equal, below_or_equal, equal or not_equal.'
unit:
description: The unit id of value, such as ms or percent. Empty means the
colorBy column's unit.
type: string
value:
description: The value compared against, in unit.
format: double
type: number
required:
- value
- color
type: object
DashboardtypesGeoMapView:
enum:
- world
- fit
- africa
- asia
- europe
- north_america
- south_america
- oceania
- country
type: string
DashboardtypesGettableDashboardV2:
properties:
createdAt:
@@ -5252,6 +5482,7 @@ components:
mapping:
signoz/AreaChartPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAreaChartPanelSpec'
signoz/BarChartPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesBarChartPanelSpec'
signoz/GeoMapPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesGeoMapPanelSpec'
signoz/HeatmapPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec'
signoz/HistogramPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHistogramPanelSpec'
signoz/ListPanel: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesListPanelSpec'
@@ -5274,6 +5505,7 @@ components:
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTextPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesScatterPlotPanelSpec'
- $ref: '#/components/schemas/DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesGeoMapPanelSpec'
type: object
DashboardtypesPanelPluginKind:
enum:
@@ -5288,6 +5520,7 @@ components:
- signoz/TextPanel
- signoz/HeatmapPanel
- signoz/ScatterPlotPanel
- signoz/GeoMapPanel
type: string
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesAreaChartPanelSpec:
properties:
@@ -5319,6 +5552,21 @@ components:
- kind
- spec
type: object
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesGeoMapPanelSpec:
properties:
kind:
description: The panel kind.
enum:
- signoz/GeoMapPanel
type: string
spec:
allOf:
- $ref: '#/components/schemas/DashboardtypesGeoMapPanelSpec'
description: The options for this panel kind. {} accepts every default.
required:
- kind
- spec
type: object
DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec:
properties:
kind:

5
pkg/cache/cache.go vendored
View File

@@ -16,11 +16,6 @@ type Cache interface {
// Get gets the cacheble entity in the dest entity passed.
Get(ctx context.Context, orgID valuer.UUID, cacheKey string, dest cachetypes.Cacheable) error
// GetMany gets the cacheable entities of cacheKeys into dests, one per key in
// the same order. A key that is not found or could not be read sets its dest
// to nil. The returned error joins the read failures; a miss alone is not one.
GetMany(ctx context.Context, orgID valuer.UUID, cacheKeys []string, dests []cachetypes.Cacheable) error
// Delete deletes the cacheable entity from cache
Delete(ctx context.Context, orgID valuer.UUID, cacheKey string)

View File

@@ -196,28 +196,6 @@ func (provider *provider) Get(ctx context.Context, orgID valuer.UUID, cacheKey s
return errors.NewInternalf(errors.CodeInternal, "unrecognized: (value: \"%s\")", reflect.TypeOf(cachedData).String())
}
func (provider *provider) GetMany(ctx context.Context, orgID valuer.UUID, cacheKeys []string, dests []cachetypes.Cacheable) error {
if len(dests) != len(cacheKeys) {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "got %d destinations for %d keys", len(dests), len(cacheKeys))
}
var errs []error
for i, cacheKey := range cacheKeys {
err := provider.Get(ctx, orgID, cacheKey, dests[i])
if err == nil {
continue
}
dests[i] = nil
if !errors.Ast(err, errors.TypeNotFound) {
errs = append(errs, err)
}
}
if len(errs) == 0 {
return nil
}
return errors.Join(errs...)
}
func (provider *provider) Delete(ctx context.Context, orgID valuer.UUID, cacheKey string) {
_, span := provider.settings.Tracer().Start(ctx, "memory.delete", trace.WithAttributes(
attribute.String(semconv.AttributeDBSystem, "memory"),

View File

@@ -260,27 +260,3 @@ func TestCloneableConcurrentSetGet(t *testing.T) {
assert.NotSame(t, cachedCloneable, cloneables[i])
}
}
func TestGetMany(t *testing.T) {
cache, err := New(context.Background(), factorytest.NewSettings(), cache.Config{Provider: "memory", Memory: cache.Memory{
NumCounters: 10 * 1000,
MaxCost: 1 << 26,
}})
require.NoError(t, err)
orgID := valuer.GenerateUUID()
require.NoError(t, cache.Set(context.Background(), orgID, "first", &CloneableA{Key: "first", Value: 1}, 10*time.Second))
require.NoError(t, cache.Set(context.Background(), orgID, "third", &CloneableA{Key: "third", Value: 3}, 10*time.Second))
// bytes that are not JSON cannot be read into a CloneableA
memory := cache.(*provider)
memory.cc.SetWithTTL(strings.Join([]string{orgID.StringValue(), "fourth"}, "::"), []byte("not json"), 1, 10*time.Second)
memory.cc.Wait()
dests := []cachetypes.Cacheable{new(CloneableA), new(CloneableA), new(CloneableA), new(CloneableA)}
err = cache.GetMany(context.Background(), orgID, []string{"first", "second", "third", "fourth"}, dests)
// the unreadable key is reported and nilled, the readable ones are still served
assert.Error(t, err)
assert.Equal(t, []cachetypes.Cacheable{&CloneableA{Key: "first", Value: 1}, nil, &CloneableA{Key: "third", Value: 3}, nil}, dests)
}

View File

@@ -66,49 +66,6 @@ func (c *provider) Get(ctx context.Context, orgID valuer.UUID, cacheKey string,
return nil
}
func (c *provider) GetMany(ctx context.Context, orgID valuer.UUID, cacheKeys []string, dests []cachetypes.Cacheable) error {
if len(dests) != len(cacheKeys) {
return errors.NewInvalidInputf(errors.CodeInvalidInput, "got %d destinations for %d keys", len(dests), len(cacheKeys))
}
if len(cacheKeys) == 0 {
return nil
}
updatedCacheKeys := make([]string, len(cacheKeys))
for i, cacheKey := range cacheKeys {
updatedCacheKeys[i] = strings.Join([]string{orgID.StringValue(), cacheKey}, "::")
}
values, err := c.client.MGet(ctx, updatedCacheKeys...).Result()
if err != nil {
for i := range dests {
dests[i] = nil
}
return err
}
var errs []error
for i, value := range values {
// MGET answers a missing key with nil and a present one with its string
switch stored := value.(type) {
case nil:
dests[i] = nil
case string:
if err := dests[i].UnmarshalBinary([]byte(stored)); err != nil {
dests[i] = nil
errs = append(errs, err)
}
default:
dests[i] = nil
errs = append(errs, errors.NewInternalf(errors.CodeInternal, "unexpected value of type %T for key %s", value, cacheKeys[i]))
}
}
if len(errs) == 0 {
return nil
}
return errors.Join(errs...)
}
func (c *provider) Delete(ctx context.Context, orgID valuer.UUID, cacheKey string) {
c.DeleteMany(ctx, orgID, []string{cacheKey})
}

View File

@@ -13,7 +13,6 @@ import (
"github.com/SigNoz/signoz/pkg/valuer"
"github.com/go-redis/redismock/v9"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type CacheableA struct {
@@ -59,28 +58,3 @@ func TestSet(t *testing.T) {
assert.NoError(t, cache.Set(context.Background(), orgID, "key", cacheable, 10*time.Second))
assert.NoError(t, mock.ExpectationsWereMet())
}
func TestGetMany(t *testing.T) {
db, mock := redismock.NewClientMock()
providerSettings := instrumentationtest.New().ToProviderSettings()
cache := &provider{client: db, settings: factory.NewScopedProviderSettings(providerSettings, "github.com/SigNoz/signoz/pkg/cache/rediscache")}
orgID := valuer.GenerateUUID()
stored, err := json.Marshal(&CacheableA{Key: "first", Value: 1, Expiry: time.Microsecond})
require.NoError(t, err)
// one round trip for all keys, with the missing one answered as nil and
// the third holding something that is not a CacheableA
mock.ExpectMGet(
strings.Join([]string{orgID.StringValue(), "first"}, "::"),
strings.Join([]string{orgID.StringValue(), "second"}, "::"),
strings.Join([]string{orgID.StringValue(), "third"}, "::"),
).SetVal([]any{string(stored), nil, "not json"})
dests := []cachetypes.Cacheable{new(CacheableA), new(CacheableA), new(CacheableA)}
err = cache.GetMany(context.Background(), orgID, []string{"first", "second", "third"}, dests)
// the unreadable key is reported and nilled, the readable one is still served
assert.Error(t, err)
assert.Equal(t, []cachetypes.Cacheable{&CacheableA{Key: "first", Value: 1, Expiry: time.Microsecond}, nil, nil}, dests)
assert.NoError(t, mock.ExpectationsWereMet())
}

View File

@@ -1,7 +1,6 @@
package querier
import (
"cmp"
"context"
"encoding/json"
"fmt"
@@ -12,16 +11,11 @@ import (
"github.com/SigNoz/signoz/pkg/cache"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/types/cachetypes"
"github.com/SigNoz/signoz/pkg/types/metrictypes"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/valuer"
)
// bucketCacheCellSteps is the grid cell width in steps; no cached bucket
// crosses a cell boundary.
const bucketCacheCellSteps = 60
// BucketCache is the interface for bucket-based caching.
type BucketCache interface {
// cached portion + list of gaps to fetch
@@ -64,59 +58,44 @@ func (bc *bucketCache) GetMissRanges(
bc.logger.DebugContext(ctx, "getting miss ranges", slog.String("fingerprint", q.Fingerprint()), slog.Uint64("start", startMs), slog.Uint64("end", endMs))
// Generate cache key
cacheKey := bc.generateCacheKey(q)
bc.logger.DebugContext(ctx, "cache key", slog.String("cache_key", cacheKey))
// Try to get cached data
var data qbtypes.CachedData
err := bc.cache.Get(ctx, orgID, cacheKey, &data)
if err != nil {
if !errors.Ast(err, errors.TypeNotFound) {
bc.logger.ErrorContext(ctx, "error getting cached data", errors.Attr(err))
}
// No cached data, need to fetch entire range
missing = []*qbtypes.TimeRange{{From: startMs, To: endMs}}
return nil, missing
}
// Extract step interval if this is a builder query
stepMs := uint64(step.Milliseconds())
// No cached data, need to fetch entire range
missing = []*qbtypes.TimeRange{{From: startMs, To: endMs}}
if endMs <= startMs {
return nil, missing
}
cellWidthMs := bucketCacheCellSteps * stepMs
firstCellIdx, lastCellIdx := deriveCellIdx(startMs, cellWidthMs), deriveCellIdx(endMs-1, cellWidthMs)
// Gather the buckets of every cell the window touches
cellKeys := make([]string, 0, lastCellIdx-firstCellIdx+1)
cellEntries := make([]cachetypes.Cacheable, 0, lastCellIdx-firstCellIdx+1)
for cellIdx := firstCellIdx; cellIdx <= lastCellIdx; cellIdx++ {
cellKeys = append(cellKeys, bc.generateCacheKey(q, cellIdx))
cellEntries = append(cellEntries, &qbtypes.CachedData{})
}
// A cell that could not be read comes back nil like a miss, so the rest
// are still worth serving
if err := bc.cache.GetMany(ctx, orgID, cellKeys, cellEntries); err != nil {
bc.logger.ErrorContext(ctx, "error getting cached data", errors.Attr(err))
}
cachedBuckets := []*qbtypes.CachedBucket{}
cachedWarnings := []string{}
for _, cellEntry := range cellEntries {
if cellEntry == nil {
continue
}
cellData := cellEntry.(*qbtypes.CachedData)
cachedBuckets = append(cachedBuckets, cellData.Buckets...)
cachedWarnings = append(cachedWarnings, cellData.Warnings...)
}
if len(cachedBuckets) == 0 {
return nil, missing
}
// Find missing ranges with step alignment
missing = bc.findMissingRangesWithStep(cachedBuckets, startMs, endMs, stepMs)
missing = bc.findMissingRangesWithStep(data.Buckets, startMs, endMs, stepMs)
bc.logger.DebugContext(ctx, "missing ranges", slog.Any("missing", missing), slog.Uint64("step", stepMs))
// If no cached data overlaps with requested range, return empty result
if len(data.Buckets) == 0 {
return nil, missing
}
// Extract relevant buckets and merge them
relevantBuckets := bc.filterRelevantBuckets(cachedBuckets, startMs, endMs)
relevantBuckets := bc.filterRelevantBuckets(data.Buckets, startMs, endMs)
if len(relevantBuckets) == 0 {
// nothing cached overlaps the window, so all of it has to be fetched.
return nil, []*qbtypes.TimeRange{{From: startMs, To: endMs}}
}
// Merge buckets into a single result
mergedResult := bc.mergeBuckets(ctx, relevantBuckets, cachedWarnings)
mergedResult := bc.mergeBuckets(ctx, relevantBuckets, data.Warnings)
// Filter the merged result to only include values within the requested time range
mergedResult = bc.filterResultToTimeRange(mergedResult, q, startMs, endMs, stepMs)
@@ -151,6 +130,15 @@ func (bc *bucketCache) Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Que
slog.Uint64("cachable_end", cachableEndMs))
}
// Generate cache key
cacheKey := bc.generateCacheKey(q)
// Get existing cached data
var existingData qbtypes.CachedData
if err := bc.cache.Get(ctx, orgID, cacheKey, &existingData); err != nil {
existingData = qbtypes.CachedData{}
}
// Trim the result to exclude data within flux interval
trimmedResult := bc.trimResultToFluxBoundary(fresh, cachableEndMs)
if trimmedResult == nil {
@@ -186,54 +174,38 @@ func (bc *bucketCache) Put(ctx context.Context, orgID valuer.UUID, q qbtypes.Que
}
}
// Convert trimmed result to buckets, one per grid cell it spans
cellWidthMs := bucketCacheCellSteps * stepMs
freshBuckets := bc.resultToBuckets(ctx, trimmedResult, cachableStartMs, cachableEndMs, cellWidthMs)
// Convert trimmed result to buckets with adjusted boundaries
freshBuckets := bc.resultToBuckets(ctx, trimmedResult, cachableStartMs, cachableEndMs)
cellKeys := make([]string, len(freshBuckets))
cellEntries := make([]cachetypes.Cacheable, len(freshBuckets))
for i, freshBucket := range freshBuckets {
cellKeys[i] = bc.generateCacheKey(q, deriveCellIdx(freshBucket.StartMs, cellWidthMs))
cellEntries[i] = &qbtypes.CachedData{}
}
// A cell that could not be read comes back nil like a miss, so its entry is
// rebuilt from the fresh bucket alone
if err := bc.cache.GetMany(ctx, orgID, cellKeys, cellEntries); err != nil {
bc.logger.ErrorContext(ctx, "error getting cached data", errors.Attr(err))
// If no fresh buckets and no existing data, don't cache
if len(freshBuckets) == 0 && len(existingData.Buckets) == 0 {
return
}
// Each cell is its own cache entry, rewritten only when the fresh bucket
// changes what it holds, so an entry's TTL runs from its last new data
for i, freshBucket := range freshBuckets {
cellEntry := &qbtypes.CachedData{}
if cellEntries[i] != nil {
cellEntry = cellEntries[i].(*qbtypes.CachedData)
}
// Merge with existing buckets
mergedBuckets := bc.mergeAndDeduplicateBuckets(existingData.Buckets, freshBuckets)
cellBuckets, changed := bc.mergeFreshBucketIntoCell(ctx, cellEntry.Buckets, freshBucket)
if !changed {
continue
}
cellEntry.Buckets = cellBuckets
cellEntry.Warnings = bc.deduplicateWarnings(append(cellEntry.Warnings, trimmedResult.Warnings...))
if err := bc.cache.Set(ctx, orgID, cellKeys[i], cellEntry, bc.cacheTTL); err != nil {
bc.logger.ErrorContext(ctx, "error setting cached data", errors.Attr(err))
}
// Update warnings
allWarnings := append(existingData.Warnings, trimmedResult.Warnings...)
uniqueWarnings := bc.deduplicateWarnings(allWarnings)
// Create updated cached data
updatedData := qbtypes.CachedData{
Buckets: mergedBuckets,
Warnings: uniqueWarnings,
}
// Marshal and store in cache
if err := bc.cache.Set(ctx, orgID, cacheKey, &updatedData, bc.cacheTTL); err != nil {
bc.logger.ErrorContext(ctx, "error setting cached data", errors.Attr(err))
}
}
// generateCacheKey names the cache entry holding one grid cell of a query.
func (bc *bucketCache) generateCacheKey(q qbtypes.Query, cellIdx uint64) string {
return fmt.Sprintf("v6:query:%s:cellidx:%d", q.Fingerprint(), cellIdx)
}
// generateCacheKey creates a unique cache key based on query fingerprint.
func (bc *bucketCache) generateCacheKey(q qbtypes.Query) string {
fingerprint := q.Fingerprint()
// deriveCellIdx returns the index of the grid cell holding startMs; with no grid
// every bucket shares cell 0.
func deriveCellIdx(startMs, cellWidthMs uint64) uint64 {
if cellWidthMs == 0 {
return 0
}
return startMs / cellWidthMs
return fmt.Sprintf("v5:query:%s", fingerprint)
}
// findMissingRangesWithStep identifies time ranges not covered by cached buckets with step alignment.
@@ -674,7 +646,7 @@ func (bc *bucketCache) isEmptyResult(result *qbtypes.Result) (isEmpty bool, isFi
}
// resultToBuckets converts a query result into time-based buckets.
func (bc *bucketCache) resultToBuckets(ctx context.Context, result *qbtypes.Result, startMs, endMs, cellWidthMs uint64) []*qbtypes.CachedBucket {
func (bc *bucketCache) resultToBuckets(ctx context.Context, result *qbtypes.Result, startMs, endMs uint64) []*qbtypes.CachedBucket {
// Check if result is empty
isEmpty, isFiltered := bc.isEmptyResult(result)
@@ -685,74 +657,62 @@ func (bc *bucketCache) resultToBuckets(ctx context.Context, result *qbtypes.Resu
return nil
}
// Empty filtered results still become buckets, so the range is not re-queried
perCellBuckets, err := result.SplitIntoCacheCellBuckets(startMs, endMs, cellWidthMs)
// For now, create a single bucket for the entire range
// In the future, we could split large ranges into smaller buckets
valueBytes, err := json.Marshal(result.Value)
if err != nil {
bc.logger.ErrorContext(ctx, "failed to convert result to buckets", errors.Attr(err))
bc.logger.ErrorContext(ctx, "failed to marshal result value", errors.Attr(err))
return nil
}
return perCellBuckets
// Always create a bucket, even for empty filtered results
// This ensures we don't re-query for data that doesn't exist
return []*qbtypes.CachedBucket{
{
StartMs: startMs,
EndMs: endMs,
Type: result.Type,
Value: valueBytes,
Stats: result.Stats,
},
}
}
// mergeFreshBucketIntoCell folds a fresh bucket into the buckets of its cell
// and reports whether the cell changed: a bucket already covering the fresh
// one leaves it as is, buckets inside the fresh one are replaced, buckets
// overlapping or edge to edge with it are unioned, and the rest are kept apart.
func (bc *bucketCache) mergeFreshBucketIntoCell(ctx context.Context, cellBuckets []*qbtypes.CachedBucket, freshBucket *qbtypes.CachedBucket) ([]*qbtypes.CachedBucket, bool) {
keptBuckets := make([]*qbtypes.CachedBucket, 0, len(cellBuckets)+1)
touchingBuckets := []*qbtypes.CachedBucket{}
for _, cellBucket := range cellBuckets {
switch {
case cellBucket.Contains(freshBucket):
return cellBuckets, false
case cellBucket.Touches(freshBucket):
touchingBuckets = append(touchingBuckets, cellBucket)
default:
keptBuckets = append(keptBuckets, cellBucket)
}
}
keptBuckets = append(keptBuckets, bc.unionBuckets(ctx, freshBucket, touchingBuckets))
// mergeAndDeduplicateBuckets combines and deduplicates bucket lists.
func (bc *bucketCache) mergeAndDeduplicateBuckets(existing, fresh []*qbtypes.CachedBucket) []*qbtypes.CachedBucket {
// Create a map to deduplicate by time range
bucketMap := make(map[string]*qbtypes.CachedBucket)
slices.SortFunc(keptBuckets, func(a, b *qbtypes.CachedBucket) int {
return cmp.Compare(a.StartMs, b.StartMs)
// Add existing buckets
for _, bucket := range existing {
key := fmt.Sprintf("%d-%d", bucket.StartMs, bucket.EndMs)
bucketMap[key] = bucket
}
// Add/update with fresh buckets
for _, bucket := range fresh {
key := fmt.Sprintf("%d-%d", bucket.StartMs, bucket.EndMs)
bucketMap[key] = bucket
}
// Convert back to slice with pre-allocated capacity
result := make([]*qbtypes.CachedBucket, 0, len(bucketMap))
for _, bucket := range bucketMap {
result = append(result, bucket)
}
// Sort by start time
slices.SortFunc(result, func(a, b *qbtypes.CachedBucket) int {
if a.StartMs < b.StartMs {
return -1
}
if a.StartMs > b.StartMs {
return 1
}
return 0
})
return keptBuckets, true
}
// unionBuckets widens the fresh bucket by the buckets it touches. Buckets
// inside it are replaced outright, so only a genuine partial overlap decodes
// anything.
func (bc *bucketCache) unionBuckets(ctx context.Context, freshBucket *qbtypes.CachedBucket, touchingBuckets []*qbtypes.CachedBucket) *qbtypes.CachedBucket {
bucketsToMerge := []*qbtypes.CachedBucket{freshBucket}
for _, touchingBucket := range touchingBuckets {
if !freshBucket.Contains(touchingBucket) {
bucketsToMerge = append(bucketsToMerge, touchingBucket)
}
}
if len(bucketsToMerge) == 1 {
return freshBucket
}
value, err := json.Marshal(bc.mergeTimeSeriesValues(ctx, bucketsToMerge))
if err != nil {
bc.logger.ErrorContext(ctx, "failed to marshal unioned bucket", errors.Attr(err))
return freshBucket
}
unionBucket := &qbtypes.CachedBucket{
StartMs: freshBucket.StartMs,
EndMs: freshBucket.EndMs,
Type: freshBucket.Type,
Value: value,
Stats: qbtypes.ExecStats{StepIntervals: freshBucket.Stats.StepIntervals},
}
for _, mergedBucket := range bucketsToMerge {
unionBucket.StartMs = min(unionBucket.StartMs, mergedBucket.StartMs)
unionBucket.EndMs = max(unionBucket.EndMs, mergedBucket.EndMs)
unionBucket.Stats.RowsScanned += mergedBucket.Stats.RowsScanned
unionBucket.Stats.BytesScanned += mergedBucket.Stats.BytesScanned
unionBucket.Stats.DurationMS += mergedBucket.Stats.DurationMS
}
return unionBucket
return result
}
// deduplicateWarnings removes duplicate warnings.

View File

@@ -2,7 +2,6 @@ package querier
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"testing"
@@ -1466,161 +1465,3 @@ func TestBucketCache_NoCache(t *testing.T) {
// The actual NoCache logic is implemented in querier.run(), not in bucket cache
// This test verifies that the cache works normally and NoCache bypasses it at a higher level
}
func TestBucketCachePutSlicesAtCellBoundaries(t *testing.T) {
bc := createTestBucketCache(t)
ctx := context.Background()
orgID := valuer.UUID{}
step := qbtypes.Step{Duration: time.Second}
// a 1s step gives 60s cells, so 30s to 150s spans three of them
query := &mockQuery{fingerprint: "sliced-query", startMs: 30000, endMs: 150000}
bc.Put(ctx, orgID, query, step, &qbtypes.Result{
Type: qbtypes.RequestTypeTimeSeries,
Value: createTestTimeSeries("A", 30000, 150000, 1000),
Stats: qbtypes.ExecStats{RowsScanned: 120, BytesScanned: 1200, DurationMS: 12},
})
type bucketShape struct {
cell, startMs, endMs, rowsScanned uint64
valueCount int
}
shapes := []bucketShape{}
for cell := uint64(0); cell < 3; cell++ {
var cellEntry qbtypes.CachedData
require.NoError(t, bc.cache.Get(ctx, orgID, bc.generateCacheKey(query, cell), &cellEntry))
for _, bucket := range cellEntry.Buckets {
var tsData qbtypes.TimeSeriesData
require.NoError(t, json.Unmarshal(bucket.Value, &tsData))
require.Len(t, tsData.Aggregations, 1)
require.Len(t, tsData.Aggregations[0].Series, 1)
shapes = append(shapes, bucketShape{cell, bucket.StartMs, bucket.EndMs, bucket.Stats.RowsScanned, len(tsData.Aggregations[0].Series[0].Values)})
}
}
assert.Equal(t, []bucketShape{
{0, 30000, 60000, 30, 30},
{1, 60000, 120000, 60, 60},
{2, 120000, 150000, 30, 30},
}, shapes)
cached, missing := bc.GetMissRanges(ctx, orgID, query, step)
assert.Equal(t, []*qbtypes.TimeRange{}, missing)
assert.Len(t, cached.Value.(*qbtypes.TimeSeriesData).Aggregations[0].Series[0].Values, 120)
assert.Equal(t, uint64(120), cached.Stats.RowsScanned)
}
func TestBucketCachePutAbsorbsWritesWithinACell(t *testing.T) {
type bucketShape struct{ startMs, endMs, rowsScanned uint64 }
testCases := []struct {
name string
firstWindow qbtypes.TimeRange
secondWindow qbtypes.TimeRange
expectedBuckets []bucketShape
expectedMissingRanges []*qbtypes.TimeRange
expectedValueCount int
}{
{
name: "DisjointWrites_StayApart",
firstWindow: qbtypes.TimeRange{From: 1000, To: 3000},
secondWindow: qbtypes.TimeRange{From: 5000, To: 8000},
expectedBuckets: []bucketShape{{1000, 3000, 2}, {5000, 8000, 3}},
expectedMissingRanges: []*qbtypes.TimeRange{{From: 3000, To: 5000}},
expectedValueCount: 5,
},
{
name: "EdgeToEdgeWrites_BecomeOne",
firstWindow: qbtypes.TimeRange{From: 1000, To: 3000},
secondWindow: qbtypes.TimeRange{From: 3000, To: 5000},
expectedBuckets: []bucketShape{{1000, 5000, 4}},
expectedMissingRanges: []*qbtypes.TimeRange{},
expectedValueCount: 4,
},
{
name: "OverlappingWrites_BecomeOne",
firstWindow: qbtypes.TimeRange{From: 1000, To: 4000},
secondWindow: qbtypes.TimeRange{From: 3000, To: 6000},
expectedBuckets: []bucketShape{{1000, 6000, 6}},
expectedMissingRanges: []*qbtypes.TimeRange{},
expectedValueCount: 5,
},
{
name: "ContainedWrite_KeepsExistingBucket",
firstWindow: qbtypes.TimeRange{From: 1000, To: 10000},
secondWindow: qbtypes.TimeRange{From: 3000, To: 5000},
expectedBuckets: []bucketShape{{1000, 10000, 9}},
expectedMissingRanges: []*qbtypes.TimeRange{},
expectedValueCount: 9,
},
{
name: "ContainingWrite_ReplacesExistingBucket",
firstWindow: qbtypes.TimeRange{From: 3000, To: 5000},
secondWindow: qbtypes.TimeRange{From: 1000, To: 10000},
expectedBuckets: []bucketShape{{1000, 10000, 9}},
expectedMissingRanges: []*qbtypes.TimeRange{},
expectedValueCount: 9,
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
bc := createTestBucketCache(t)
ctx := context.Background()
orgID := valuer.UUID{}
step := qbtypes.Step{Duration: time.Second}
for _, window := range []qbtypes.TimeRange{testCase.firstWindow, testCase.secondWindow} {
query := &mockQuery{fingerprint: "absorb-query", startMs: window.From, endMs: window.To}
bc.Put(ctx, orgID, query, step, &qbtypes.Result{
Type: qbtypes.RequestTypeTimeSeries,
Value: createTestTimeSeries("A", window.From, window.To, 1000),
// one row per second, so a bucket's stats read as the seconds it was scanned for
Stats: qbtypes.ExecStats{RowsScanned: (window.To - window.From) / 1000},
})
}
query := &mockQuery{
fingerprint: "absorb-query",
startMs: min(testCase.firstWindow.From, testCase.secondWindow.From),
endMs: max(testCase.firstWindow.To, testCase.secondWindow.To),
}
var cellEntry qbtypes.CachedData
require.NoError(t, bc.cache.Get(ctx, orgID, bc.generateCacheKey(query, 0), &cellEntry))
shapes := make([]bucketShape, 0, len(cellEntry.Buckets))
for _, bucket := range cellEntry.Buckets {
shapes = append(shapes, bucketShape{bucket.StartMs, bucket.EndMs, bucket.Stats.RowsScanned})
}
assert.Equal(t, testCase.expectedBuckets, shapes)
cached, missing := bc.GetMissRanges(ctx, orgID, query, step)
assert.Equal(t, testCase.expectedMissingRanges, missing)
assert.Len(t, cached.Value.(*qbtypes.TimeSeriesData).Aggregations[0].Series[0].Values, testCase.expectedValueCount)
})
}
}
func TestBucketCachePutLeavesAnUnchangedCellAlone(t *testing.T) {
bc := createTestBucketCache(t)
ctx := context.Background()
orgID := valuer.UUID{}
step := qbtypes.Step{Duration: time.Second}
wide := &mockQuery{fingerprint: "unchanged-query", startMs: 1000, endMs: 10000}
bc.Put(ctx, orgID, wide, step, &qbtypes.Result{
Type: qbtypes.RequestTypeTimeSeries,
Value: createTestTimeSeries("A", 1000, 10000, 1000),
Warnings: []string{"from the wide write"},
})
// a write already covered by the cell's bucket must not rewrite the entry,
// which shows as its warnings never arriving
inside := &mockQuery{fingerprint: "unchanged-query", startMs: 3000, endMs: 5000}
bc.Put(ctx, orgID, inside, step, &qbtypes.Result{
Type: qbtypes.RequestTypeTimeSeries,
Value: createTestTimeSeries("A", 3000, 5000, 1000),
Warnings: []string{"from the inside write"},
})
var cellEntry qbtypes.CachedData
require.NoError(t, bc.cache.Get(ctx, orgID, bc.generateCacheKey(wide, 0), &cellEntry))
assert.Equal(t, []string{"from the wide write"}, cellEntry.Warnings)
}

View File

@@ -2158,7 +2158,7 @@ func TestScatterPlotDimensionsBindToQuery(t *testing.T) {
{"type": "builder_query", "spec": {"name": "B", "signal": "traces", "aggregations": [{"expression": "p99(duration_nano)"}], "groupBy": [{"name": "k8s.pod.name"}]}}
]}}`,
dimensions: `{"x": "A", "y": "B"}`,
expectedErrorSubstring: "queries must group by the same keys so their rows join into dots; \"A\" groups by [`service.name`], \"B\" groups by [`k8s.pod.name`]",
expectedErrorSubstring: "queries must group by the same keys so their rows join; \"A\" groups by [`service.name`], \"B\" groups by [`k8s.pod.name`]",
},
{
name: "Queries_SameGroupByDifferentOrder_Accepted",
@@ -2245,6 +2245,174 @@ func TestNumberPanelDefaults(t *testing.T) {
assert.Contains(t, outputStr, `"operator":"above"`, "expected stored/response JSON to contain operator:above")
}
func geoMapDashboard(spec, queryPlugin string) []byte {
return []byte(`{
"variables": [],
"panels": {
"p1": {
"kind": "Panel",
"spec": {
"links": [],
"plugin": {"kind": "signoz/GeoMapPanel", "spec": ` + spec + `},
"queries": [{"kind": "scalar", "spec": {"plugin": ` + queryPlugin + `}}]
}
}
},
"links": [],
"layouts": []
}`)
}
func TestGeoMapPanelSpec(t *testing.T) {
data := geoMapDashboard(`{
"formatting": {"columnUnits": {"A": "percent"}, "decimalPrecision": "2"},
"location": {"mode": "subdivision", "country": "geo.country.iso_code", "countryFormat": "alpha2", "region": "geo.region.iso_code"},
"map": {"layer": "bubbles", "view": "country", "viewCountry": "FR", "projection": "equal_earth", "colorBy": "A", "sizeBy": "A"},
"chartAppearance": {
"colors": {"mode": "opacity", "fill": "#4E74F8", "palette": "ice", "scale": "linear", "steps": 16, "min": 0, "max": 10},
"bubbles": {"minRadius": 2, "maxRadius": 30},
"labels": false
},
"legend": {"position": "right"},
"thresholds": [{"value": 5, "operator": "above", "unit": "percent", "color": "#F24769"}]
}`, `{"kind": "signoz/ClickHouseSQL", "spec": {"name": "A", "query": "SELECT 1"}}`)
d, err := unmarshalDashboard(data)
require.NoError(t, err, "unmarshal and validate failed")
require.IsType(t, &GeoMapPanelSpec{}, d.Panels["p1"].Spec.Plugin.Spec)
spec := d.Panels["p1"].Spec.Plugin.Spec.(*GeoMapPanelSpec)
assert.Equal(t, "subdivision", spec.Location.Mode.ValueOrDefault())
assert.Equal(t, "country", spec.Map.View.ValueOrDefault())
assert.Equal(t, "FR", spec.Map.ViewCountry)
assert.Equal(t, "equal_earth", spec.Map.Projection.ValueOrDefault())
require.NotNil(t, spec.ChartAppearance.Labels)
assert.False(t, *spec.ChartAppearance.Labels)
require.Len(t, spec.Thresholds, 1)
output, err := json.Marshal(d)
require.NoError(t, err, "marshal dashboard failed")
roundTripped, err := unmarshalDashboard(output)
require.NoError(t, err, "round trip failed")
roundTrippedSpec := roundTripped.Panels["p1"].Spec.Plugin.Spec.(*GeoMapPanelSpec)
assert.Equal(t, spec.Formatting, roundTrippedSpec.Formatting)
assert.Equal(t, spec.Location, roundTrippedSpec.Location)
assert.Equal(t, spec.Map, roundTrippedSpec.Map)
assert.Equal(t, spec.ChartAppearance, roundTrippedSpec.ChartAppearance)
assert.Equal(t, spec.Thresholds, roundTrippedSpec.Thresholds)
t.Run("Defaults", func(t *testing.T) {
d, err := unmarshalDashboard(geoMapDashboard(`{}`, `{"kind": "signoz/ClickHouseSQL", "spec": {"name": "A", "query": "SELECT 2"}}`))
require.NoError(t, err)
output, err := json.Marshal(d.Panels["p1"].Spec.Plugin.Spec)
require.NoError(t, err)
for _, field := range []string{`"mode":"palette"`, `"mode":"auto"`, `"countryFormat":"auto"`, `"layer":"regions"`, `"view":"world"`, `"projection":"mercator"`, `"palette":"lava"`, `"scale":"log"`, `"labels":null`} {
assert.Contains(t, string(output), field)
}
})
testCases := []struct {
name string
spec string
wantErr bool
}{
{name: "UnknownField_Rejected", spec: `{"unit": "ms"}`, wantErr: true},
{name: "ThresholdFormat_Rejected", spec: `{"thresholds": [{"value": 5, "color": "Red", "format": "background"}]}`, wantErr: true},
{name: "UnknownLayer_Rejected", spec: `{"map": {"layer": "heat"}}`, wantErr: true},
{name: "UnknownView_Rejected", spec: `{"map": {"view": "emea"}}`, wantErr: true},
{name: "UnknownProjection_Rejected", spec: `{"map": {"projection": "robinson"}}`, wantErr: true},
{name: "UnknownMapField_Rejected", spec: `{"map": {"zoom": 3}}`, wantErr: true},
{name: "CountryViewWithoutCountry_Rejected", spec: `{"map": {"view": "country"}}`, wantErr: true},
{name: "LowercaseViewCountry_Rejected", spec: `{"map": {"view": "country", "viewCountry": "fr"}}`, wantErr: true},
{name: "UnassignedViewCountry_Rejected", spec: `{"map": {"view": "country", "viewCountry": "XX"}}`, wantErr: true},
{name: "UnknownLocationMode_Rejected", spec: `{"location": {"mode": "geohash"}}`, wantErr: true},
{name: "UnknownCountryFormat_Rejected", spec: `{"location": {"countryFormat": "numeric"}}`, wantErr: true},
{name: "ColorMinAboveMax_Rejected", spec: `{"chartAppearance": {"colors": {"min": 10, "max": 1}}}`, wantErr: true},
{name: "UnknownColorMode_Rejected", spec: `{"chartAppearance": {"colors": {"mode": "gradient"}}}`, wantErr: true},
{name: "UnknownColorsField_Rejected", spec: `{"chartAppearance": {"colors": {"opacity": 0.5}}}`, wantErr: true},
{name: "NonHexFill_Rejected", spec: `{"chartAppearance": {"colors": {"fill": "blue"}}}`, wantErr: true},
{name: "TooFewSteps_Rejected", spec: `{"chartAppearance": {"colors": {"steps": 1}}}`, wantErr: true},
{name: "RadiusOutOfRange_Rejected", spec: `{"chartAppearance": {"bubbles": {"maxRadius": 65}}}`, wantErr: true},
{name: "MinRadiusAboveMax_Rejected", spec: `{"chartAppearance": {"bubbles": {"minRadius": 20, "maxRadius": 10}}}`, wantErr: true},
{name: "MinRadiusAboveDefaultMax_Rejected", spec: `{"chartAppearance": {"bubbles": {"minRadius": 64}}}`, wantErr: true},
{name: "PointsLayer_MinRadiusAboveDefaultMax_Rejected", spec: `{"map": {"layer": "points"}, "chartAppearance": {"bubbles": {"minRadius": 10}}}`, wantErr: true},
{name: "BubblesLayer_MinRadiusBelowDefaultMax_Accepted", spec: `{"map": {"layer": "bubbles"}, "chartAppearance": {"bubbles": {"minRadius": 10}}}`},
{name: "UnknownBubblesField_Rejected", spec: `{"chartAppearance": {"bubbles": {"radius": 4}}}`, wantErr: true},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
_, err := unmarshalDashboard(geoMapDashboard(testCase.spec, `{"kind": "signoz/ClickHouseSQL", "spec": {"name": "A", "query": "SELECT 3"}}`))
if testCase.wantErr {
assert.Error(t, err)
return
}
assert.NoError(t, err)
})
}
}
func TestGeoMapPanelBindsToQuery(t *testing.T) {
builderQuery := `{"kind": "signoz/BuilderQuery", "spec": {"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "geo.country.iso_code"}]}}`
testCases := []struct {
name string
queryPlugin string
spec string
expectedErrorSubstring string
}{
{
name: "LocationAndValueColumns_Accepted",
queryPlugin: builderQuery,
spec: `{"location": {"mode": "country", "country": "geo.country.iso_code"}, "map": {"colorBy": "A", "sizeBy": "A"}}`,
},
{
name: "LocationKeyNotGroupedBy_Rejected",
queryPlugin: builderQuery,
spec: `{"location": {"mode": "coordinates", "lat": "geo.location.lat"}}`,
expectedErrorSubstring: "location.lat: \"geo.location.lat\" is not a group-by key of the panel's query; group-by keys: `geo.country.iso_code`",
},
{
name: "ValueColumnMissing_Rejected",
queryPlugin: builderQuery,
spec: `{"map": {"sizeBy": "B"}}`,
expectedErrorSubstring: "map.sizeBy: \"B\" is not a value column of the panel's query; value columns: `A`",
},
{
name: "QueriesGroupedByDifferentKeys_Rejected",
queryPlugin: `{"kind": "signoz/CompositeQuery", "spec": {"queries": [
{"type": "builder_query", "spec": {"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "geo.country.iso_code"}]}},
{"type": "builder_query", "spec": {"name": "B", "signal": "logs", "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "geo.region.iso_code"}]}}
]}}`,
spec: `{"map": {"colorBy": "A", "sizeBy": "B"}}`,
expectedErrorSubstring: "queries must group by the same keys so their rows join; \"A\" groups by [`geo.country.iso_code`], \"B\" groups by [`geo.region.iso_code`]",
},
{
name: "QueriesGroupedBySameKeys_FormulaSkipped_Accepted",
queryPlugin: `{"kind": "signoz/CompositeQuery", "spec": {"queries": [
{"type": "builder_query", "spec": {"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "geo.country.iso_code"}]}},
{"type": "builder_query", "spec": {"name": "B", "signal": "logs", "disabled": true, "aggregations": [{"expression": "count()"}], "groupBy": [{"name": "geo.country.iso_code"}]}},
{"type": "builder_formula", "spec": {"name": "F1", "expression": "A / B"}}
]}}`,
spec: `{"location": {"country": "geo.country.iso_code"}, "map": {"colorBy": "F1", "sizeBy": "A"}}`,
},
{
name: "ClickHouseQuery_BindingsUnchecked",
queryPlugin: `{"kind": "signoz/ClickHouseSQL", "spec": {"name": "A", "query": "SELECT country, errors"}}`,
spec: `{"location": {"country": "country"}, "map": {"colorBy": "errors"}}`,
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
_, err := unmarshalDashboard(geoMapDashboard(testCase.spec, testCase.queryPlugin))
if testCase.expectedErrorSubstring == "" {
require.NoError(t, err)
return
}
require.Error(t, err)
assert.Contains(t, err.Error(), testCase.expectedErrorSubstring, "unexpected error message: %s", err.Error())
})
}
}
// TestPersesFixtureStorageRoundTrip exercises the typed → map[string]any →
// typed cycle that the create/get path performs against the kitchen-sink
// fixture. Catches plugin specs whose UnmarshalJSON expects a different shape
@@ -2566,7 +2734,7 @@ func TestPanelTypeQueryTypeCompatibility(t *testing.T) {
switch panelKind {
case "signoz/ListPanel":
return "raw"
case "signoz/TablePanel", "signoz/NumberPanel", "signoz/PieChartPanel", "signoz/HistogramPanel", "signoz/ScatterPlotPanel":
case "signoz/TablePanel", "signoz/NumberPanel", "signoz/PieChartPanel", "signoz/HistogramPanel", "signoz/ScatterPlotPanel", "signoz/GeoMapPanel":
return "scalar"
default:
return "time_series"
@@ -2624,6 +2792,13 @@ func TestPanelTypeQueryTypeCompatibility(t *testing.T) {
{"ScatterPlot+Composite(promql)", mkComposite("signoz/ScatterPlotPanel", "promql", `{"name":"A","query":"up"}`), true},
{"Table+Composite(clickhouse)", mkComposite("signoz/TablePanel", "clickhouse_sql", `{"name":"A","query":"SELECT 1"}`), false},
{"Table+Composite(builder_ai)", mkComposite("signoz/TablePanel", "builder_ai_query", `{"name":"A","aggregations":[{"expression":"count()"}]}`), false},
{"GeoMap+ClickHouse", mkQuery("signoz/GeoMapPanel", "signoz/ClickHouseSQL", `{"name":"A","query":"SELECT 1"}`), false},
{"GeoMap+PromQL", mkQuery("signoz/GeoMapPanel", "signoz/PromQLQuery", `{"name":"A","query":"up"}`), true},
{"GeoMap+Formula", mkQuery("signoz/GeoMapPanel", "signoz/Formula", `{"name":"F1","expression":"A+B"}`), false},
{"GeoMap+TraceOperator", mkQuery("signoz/GeoMapPanel", "signoz/TraceOperator", `{"name":"T1","expression":"A => B"}`), true},
{"GeoMap+Composite(trace_operator)", mkComposite("signoz/GeoMapPanel", "builder_trace_operator", `{"name":"T1","expression":"A => B"}`), true},
{"GeoMap+Composite(formula)", mkComposite("signoz/GeoMapPanel", "builder_formula", `{"name":"F1","expression":"A+B"}`), false},
{"GeoMap+Composite(promql)", mkComposite("signoz/GeoMapPanel", "promql", `{"name":"A","query":"up"}`), true},
}
for _, tc := range cases {

View File

@@ -39,6 +39,7 @@ func (PanelPlugin) PrepareJSONSchema(s *jsonschema.Schema) error {
string(PanelKindText): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesTextPanelSpec"),
string(PanelKindHeatmap): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesHeatmapPanelSpec"),
string(PanelKindScatterPlot): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesScatterPlotPanelSpec"),
string(PanelKindGeoMap): schemaRef("DashboardtypesPanelPluginVariantGithubComSigNozSignozPkgTypesDashboardtypesGeoMapPanelSpec"),
})
}
@@ -73,6 +74,7 @@ func (PanelPlugin) JSONSchemaOneOf() []any {
PanelPluginVariant[TextPanelSpec]{Kind: string(PanelKindText)},
PanelPluginVariant[HeatmapPanelSpec]{Kind: string(PanelKindHeatmap)},
PanelPluginVariant[ScatterPlotPanelSpec]{Kind: string(PanelKindScatterPlot)},
PanelPluginVariant[GeoMapPanelSpec]{Kind: string(PanelKindGeoMap)},
}
}
@@ -247,6 +249,7 @@ var (
PanelKindText: func() any { return new(TextPanelSpec) },
PanelKindHeatmap: func() any { return new(HeatmapPanelSpec) },
PanelKindScatterPlot: func() any { return new(ScatterPlotPanelSpec) },
PanelKindGeoMap: func() any { return new(GeoMapPanelSpec) },
}
queryPluginSpecs = map[QueryPluginKind]func() any{
QueryKindBuilder: func() any { return new(BuilderQuerySpec) },
@@ -274,6 +277,7 @@ var (
PanelKindText: {},
PanelKindHeatmap: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindPromQL, QueryKindClickHouseSQL},
PanelKindScatterPlot: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindClickHouseSQL},
PanelKindGeoMap: {QueryKindBuilder, QueryKindComposite, QueryKindFormula, QueryKindClickHouseSQL},
}
)

View File

@@ -221,7 +221,7 @@ func (d *DashboardV2) GetPanelQuery(startTime, endTime uint64, panelKey string)
CompositeQuery: composite,
FormatOptions: &qb.FormatOptions{
FillGaps: fillGaps,
FormatTableResultForUI: panel.Spec.Plugin.Kind == PanelKindTable || panel.Spec.Plugin.Kind == PanelKindScatterPlot,
FormatTableResultForUI: panel.Spec.Plugin.Kind == PanelKindTable || panel.Spec.Plugin.Kind == PanelKindScatterPlot || panel.Spec.Plugin.Kind == PanelKindGeoMap,
},
}, nil
}

View File

@@ -252,7 +252,7 @@ func TestDashboardV2GetPanelQuery(t *testing.T) {
})
t.Run("sets FormatTableResultForUI for panels that join scalar rows", func(t *testing.T) {
for _, kind := range []PanelPluginKind{PanelKindTable, PanelKindScatterPlot} {
for _, kind := range []PanelPluginKind{PanelKindTable, PanelKindScatterPlot, PanelKindGeoMap} {
t.Run(string(kind), func(t *testing.T) {
dashboard := &DashboardV2{
Spec: DashboardSpec{
@@ -265,8 +265,11 @@ func TestDashboardV2GetPanelQuery(t *testing.T) {
Kind: qb.RequestTypeScalar,
Spec: QuerySpec{
Plugin: QueryPlugin{
Kind: QueryKindBuilder,
Spec: &BuilderQuerySpec{Spec: qb.QueryBuilderQuery[qb.MetricAggregation]{Name: "A"}},
Kind: QueryKindComposite,
Spec: &CompositeQuerySpec{Queries: []qb.QueryEnvelope{
{Type: qb.QueryTypeBuilder, Spec: qb.QueryBuilderQuery[qb.MetricAggregation]{Name: "A"}},
{Type: qb.QueryTypeBuilder, Spec: qb.QueryBuilderQuery[qb.MetricAggregation]{Name: "B"}},
}},
},
},
},
@@ -281,6 +284,7 @@ func TestDashboardV2GetPanelQuery(t *testing.T) {
require.NoError(t, err)
require.NotNil(t, req.FormatOptions)
assert.True(t, req.FormatOptions.FormatTableResultForUI)
assert.Len(t, req.CompositeQuery.Queries, 2)
})
}
})

View File

@@ -200,10 +200,11 @@ const (
PanelKindText PanelPluginKind = "signoz/TextPanel"
PanelKindHeatmap PanelPluginKind = "signoz/HeatmapPanel"
PanelKindScatterPlot PanelPluginKind = "signoz/ScatterPlotPanel"
PanelKindGeoMap PanelPluginKind = "signoz/GeoMapPanel"
)
func (PanelPluginKind) Enum() []any {
return []any{PanelKindTimeSeries, PanelKindBarChart, PanelKindAreaChart, PanelKindNumber, PanelKindPieChart, PanelKindTable, PanelKindHistogram, PanelKindList, PanelKindText, PanelKindHeatmap, PanelKindScatterPlot}
return []any{PanelKindTimeSeries, PanelKindBarChart, PanelKindAreaChart, PanelKindNumber, PanelKindPieChart, PanelKindTable, PanelKindHistogram, PanelKindList, PanelKindText, PanelKindHeatmap, PanelKindScatterPlot, PanelKindGeoMap}
}
func (k PanelPluginKind) rendersWithoutQuery() bool {
@@ -372,7 +373,7 @@ func (s *ScatterPlotPanelSpec) validateAgainstQuery(query Query, path string) er
if !known {
return nil
}
if err := s.validateQueriesShareGroupBy(composite, path); err != nil {
if err := validateQueriesShareGroupBy(composite, path); err != nil {
return err
}
if err := s.validateValueDimensions(keys, path); err != nil {
@@ -385,7 +386,7 @@ func (s *ScatterPlotPanelSpec) validateAgainstQuery(query Query, path string) er
}
// Order is ignored since rows join on sorted labels; disabled queries count, as formulas carry their keys.
func (*ScatterPlotPanelSpec) validateQueriesShareGroupBy(composite qb.CompositeQuery, path string) error {
func validateQueriesShareGroupBy(composite qb.CompositeQuery, path string) error {
var queries []qb.QueryEnvelope
for _, envelope := range composite.Queries {
if envelope.GetType() != qb.QueryTypeFormula {
@@ -395,7 +396,7 @@ func (*ScatterPlotPanelSpec) validateQueriesShareGroupBy(composite qb.CompositeQ
for i := 1; i < len(queries); i++ {
first, other := groupByKeyNames(queries[0]), groupByKeyNames(queries[i])
if !slices.Equal(first, other) {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s: queries must group by the same keys so their rows join into dots; %q groups by [%s], %q groups by [%s]", path, queries[0].GetQueryName(), allowedValuesForKind(first), queries[i].GetQueryName(), allowedValuesForKind(other))
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s: queries must group by the same keys so their rows join; %q groups by [%s], %q groups by [%s]", path, queries[0].GetQueryName(), allowedValuesForKind(first), queries[i].GetQueryName(), allowedValuesForKind(other))
}
}
return nil
@@ -495,6 +496,147 @@ func (p ScatterPlotPoints) validate() error {
return nil
}
type GeoMapPanelSpec struct {
Visualization BasicVisualization `json:"visualization" description:"The time range the values are computed over."`
Formatting TableFormatting `json:"formatting" description:"Per-column units and decimal precision."`
Location GeoMapLocation `json:"location" description:"The group-by columns that place each row on the map."`
Map GeoMapSettings `json:"map" description:"Layer, view, projection and the value columns drawn."`
ChartAppearance GeoMapChartAppearance `json:"chartAppearance" description:"Color scale, bubble size and labels."`
Legend Legend `json:"legend" description:"Legend placement."`
Thresholds []GeoMapThreshold `json:"thresholds" validate:"dive" description:"Rules that recolor a shape, bubble or point when its value compares to a threshold as specified."`
}
func (s GeoMapPanelSpec) validate() error {
return s.ChartAppearance.Bubbles.validate(s.Map.Layer)
}
func (s *GeoMapPanelSpec) validateAgainstQuery(query Query, path string) error {
composite, err := query.Spec.Plugin.buildV5CompositeQueryFromPlugin()
if err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "%s: %s", path, err.Error())
}
keys, known := composite.CollectResultColumnKeys()
if !known {
return nil
}
if err := validateQueriesShareGroupBy(composite, path); err != nil {
return err
}
for _, binding := range []struct{ field, key string }{{"location.country", s.Location.Country}, {"location.region", s.Location.Region}, {"location.lat", s.Location.Lat}, {"location.lon", s.Location.Lon}} {
// Empty is unset: the renderer detects the column by name.
if binding.key == "" {
continue
}
if _, ok := keys.Groups[binding.key]; !ok {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s.%s: %q is not a group-by key of the panel's query; group-by keys: %s", path, binding.field, binding.key, allowedValuesForKind(slices.Sorted(maps.Keys(keys.Groups))))
}
}
for _, binding := range []struct{ field, key string }{{"map.colorBy", s.Map.ColorBy}, {"map.sizeBy", s.Map.SizeBy}} {
if binding.key == "" {
continue
}
if _, ok := keys.Aggregations[binding.key]; !ok {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s.%s: %q is not a value column of the panel's query; value columns: %s", path, binding.field, binding.key, allowedValuesForKind(slices.Sorted(maps.Keys(keys.Aggregations))))
}
}
return nil
}
type GeoMapLocation struct {
Mode GeoMapLocationMode `json:"mode" required:"true" description:"How rows are placed: auto (default) detects it from the group-by keys, country, subdivision or coordinates."`
Country string `json:"country" description:"Group-by key holding the country, for the country and subdivision modes. Empty detects it by name, such as geo.country.iso_code."`
CountryFormat GeoMapCountryFormat `json:"countryFormat" required:"true" description:"How country values are written: auto (default) detects it, alpha2, alpha3 or name."`
Region string `json:"region" description:"Group-by key holding the ISO 3166-2 region code (e.g. geo.region.iso_code), for the subdivision mode. Empty detects it by name."`
Lat string `json:"lat" description:"Group-by key holding the latitude, for the coordinates mode. Empty detects it by name."`
Lon string `json:"lon" description:"Group-by key holding the longitude, for the coordinates mode. Empty detects it by name."`
}
type GeoMapSettings struct {
Layer GeoMapLayer `json:"layer" required:"true" description:"What is drawn: regions (default) fills each shape, bubbles draws a circle per region, points a circle per coordinate."`
View GeoMapView `json:"view" required:"true" description:"The area shown: world (default), fit to the data, a continent, or country with viewCountry."`
ViewCountry string `json:"viewCountry" validate:"omitempty,iso3166_1_alpha2" description:"Uppercase ISO 3166-1 alpha-2 code of the country the country view shows."`
Projection GeoMapProjection `json:"projection" required:"true" description:"The map projection: mercator (default), equal_earth or natural_earth."`
ColorBy string `json:"colorBy" description:"Value column key, as in a Table panel, that sets the color. Empty uses the first value column."`
SizeBy string `json:"sizeBy" description:"Value column key, as in a Table panel, that sets the bubble or point size. Empty uses colorBy."`
}
func (m *GeoMapSettings) UnmarshalJSON(data []byte) error {
dec := json.NewDecoder(bytes.NewReader(data))
dec.DisallowUnknownFields()
type alias GeoMapSettings
var tmp alias
if err := dec.Decode(&tmp); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid map")
}
*m = GeoMapSettings(tmp)
return m.validate()
}
func (m GeoMapSettings) validate() error {
if m.View == GeoMapViewCountry && m.ViewCountry == "" {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid map: viewCountry is required when view is `country`")
}
return nil
}
type GeoMapChartAppearance struct {
Colors GeoMapColors `json:"colors" description:"The color scale applied to the colorBy values."`
Bubbles GeoMapBubbles `json:"bubbles" description:"The circle radius range of the bubbles and points layers."`
Labels *bool `json:"labels" description:"Shows country and state names once each fits inside its shape. Null shows them."`
}
type GeoMapColors struct {
Mode HeatmapColorMode `json:"mode" description:"How values are colored: palette (default) picks a hue from palette, opacity varies the opacity of fill."`
Palette HeatmapPalette `json:"palette" description:"The named color ramp used in palette mode, such as lava (default), ice or ember."`
Scale HeatmapColorScale `json:"scale" description:"How values map onto the color range: log (default), sqrt or linear."`
Steps int `json:"steps" validate:"omitempty,min=2,max=128" description:"The number of distinct color steps, 2 to 128. 0 uses the chart default."`
Min *float64 `json:"min" description:"The value mapped to the lowest color. Null uses the lowest value in the data. Must not exceed max."`
Max *float64 `json:"max" description:"The value mapped to the highest color. Null uses the highest value in the data."`
Fill string `json:"fill" validate:"omitempty,hexcolor" description:"The hex color varied in opacity mode. Empty uses the default fill."`
}
func (c *GeoMapColors) UnmarshalJSON(data []byte) error {
dec := json.NewDecoder(bytes.NewReader(data))
dec.DisallowUnknownFields()
type alias GeoMapColors
var tmp alias
if err := dec.Decode(&tmp); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid geo map colors")
}
*c = GeoMapColors(tmp)
return c.validate()
}
func (c GeoMapColors) validate() error {
if c.Min != nil && c.Max != nil && *c.Min > *c.Max {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "geo map colors.min %v is greater than colors.max %v", *c.Min, *c.Max)
}
return nil
}
type GeoMapBubbles struct {
MinRadius *BubbleRadius `json:"minRadius" description:"Radius of the smallest circle, in CSS pixels from 1 to 64. Null means 2 on the points layer, 3 otherwise."`
MaxRadius *BubbleRadius `json:"maxRadius" description:"Radius of the largest circle, in CSS pixels from 1 to 64. Null means 8 on the points layer, 28 otherwise."`
}
// Null bounds are compared at the layer's renderer defaults, so one-sided settings can't invert the range.
func (b GeoMapBubbles) validate(layer GeoMapLayer) error {
minRadius, maxRadius := defaultBubbleMinRadius, defaultBubbleMaxRadius
if layer == GeoMapLayerPoints {
minRadius, maxRadius = defaultPointMinRadius, defaultPointMaxRadius
}
if b.MinRadius != nil {
minRadius = *b.MinRadius
}
if b.MaxRadius != nil {
maxRadius = *b.MaxRadius
}
if minRadius > maxRadius {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid bubbles: minRadius %d must not exceed maxRadius %d", minRadius, maxRadius)
}
return nil
}
type TextPresentation struct {
TextAlign TextAlign `json:"textAlign" description:"Horizontal alignment: left (default), center or right."`
VerticalAlign VerticalAlign `json:"verticalAlign" description:"Vertical alignment: top (default), center or bottom."`
@@ -586,6 +728,17 @@ type TableThreshold struct {
ColumnName string `json:"columnName" validate:"required" required:"true" description:"The key of the value column the rule applies to: the query name for a builder query with one aggregation (A), the query name dotted with the aggregation expression for several (A.count()), or the SQL alias for a ClickHouse SQL column. Group-by columns are never colored, and aggregation aliases are display labels, not keys."`
}
// GeoMapThreshold is ComparisonThreshold without format.
type GeoMapThreshold struct {
// Value is always present in the schema (required:"true"), but 0 is a legitimate
// threshold, so it drops validate:"required" — go-playground's required treats a
// zero float as unset and would wrongly reject value: 0.
Value float64 `json:"value" required:"true" description:"The value compared against, in unit."`
Operator ComparisonOperator `json:"operator" description:"The comparison that activates the threshold: above (default), below, above_or_equal, below_or_equal, equal or not_equal."`
Unit string `json:"unit" description:"The unit id of value, such as ms or percent. Empty means the colorBy column's unit."`
Color string `json:"color" validate:"required" required:"true" description:"The CSS color of the shape, bubble or point while the threshold is active, for example #F1575F; CSS named colors such as red also work."`
}
// ══════════════════════════════════════════════
// Constrained scalar types — with default value
// ══════════════════════════════════════════════
@@ -1112,6 +1265,36 @@ func (d *PointDiameter) UnmarshalJSON(data []byte) error {
return nil
}
const (
minBubbleRadius = 1
maxBubbleRadius = 64
defaultBubbleMinRadius BubbleRadius = 3
defaultBubbleMaxRadius BubbleRadius = 28
defaultPointMinRadius BubbleRadius = 2
defaultPointMaxRadius BubbleRadius = 8
)
// BubbleRadius is a geo map circle's radius in whole CSS pixels.
type BubbleRadius int
func (BubbleRadius) PrepareJSONSchema(s *jsonschema.Schema) error {
s.WithMinimum(minBubbleRadius).WithMaximum(maxBubbleRadius)
return nil
}
func (r *BubbleRadius) UnmarshalJSON(data []byte) error {
var v int
if err := json.Unmarshal(data, &v); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid bubble radius: must be a whole number between %d and %d", minBubbleRadius, maxBubbleRadius)
}
if v < minBubbleRadius || v > maxBubbleRadius {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid bubble radius %d: must be a whole number between %d and %d", v, minBubbleRadius, maxBubbleRadius)
}
*r = BubbleRadius(v)
return nil
}
const minPointOpacity = 0.1
// PointOpacity is a scatter dot's fill alpha. The floor keeps a dot visible.
@@ -1499,3 +1682,201 @@ func (s *HeatmapColorScale) UnmarshalJSON(data []byte) error {
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid heatmap color scale %q: must be `log`, `sqrt`, or `linear`", v)
}
}
type GeoMapLocationMode struct{ valuer.String }
var (
GeoMapLocationModeAuto = GeoMapLocationMode{valuer.NewString("auto")} // default
GeoMapLocationModeCountry = GeoMapLocationMode{valuer.NewString("country")}
GeoMapLocationModeSubdivision = GeoMapLocationMode{valuer.NewString("subdivision")}
GeoMapLocationModeCoordinates = GeoMapLocationMode{valuer.NewString("coordinates")}
)
func (GeoMapLocationMode) Enum() []any {
return []any{GeoMapLocationModeAuto, GeoMapLocationModeCountry, GeoMapLocationModeSubdivision, GeoMapLocationModeCoordinates}
}
func (v GeoMapLocationMode) ValueOrDefault() string {
if v.IsZero() {
return GeoMapLocationModeAuto.StringValue()
}
return v.StringValue()
}
func (v GeoMapLocationMode) MarshalJSON() ([]byte, error) {
return json.Marshal(v.ValueOrDefault())
}
func (v *GeoMapLocationMode) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid geo map location mode: must be a string, one of `auto`, `country`, `subdivision`, or `coordinates`")
}
parsed := GeoMapLocationMode{valuer.NewString(s)}
switch parsed {
case GeoMapLocationModeAuto, GeoMapLocationModeCountry, GeoMapLocationModeSubdivision, GeoMapLocationModeCoordinates:
*v = parsed
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid geo map location mode %q: must be `auto`, `country`, `subdivision`, or `coordinates`", s)
}
}
type GeoMapCountryFormat struct{ valuer.String }
var (
GeoMapCountryFormatAuto = GeoMapCountryFormat{valuer.NewString("auto")} // default
GeoMapCountryFormatAlpha2 = GeoMapCountryFormat{valuer.NewString("alpha2")}
GeoMapCountryFormatAlpha3 = GeoMapCountryFormat{valuer.NewString("alpha3")}
GeoMapCountryFormatName = GeoMapCountryFormat{valuer.NewString("name")}
)
func (GeoMapCountryFormat) Enum() []any {
return []any{GeoMapCountryFormatAuto, GeoMapCountryFormatAlpha2, GeoMapCountryFormatAlpha3, GeoMapCountryFormatName}
}
func (v GeoMapCountryFormat) ValueOrDefault() string {
if v.IsZero() {
return GeoMapCountryFormatAuto.StringValue()
}
return v.StringValue()
}
func (v GeoMapCountryFormat) MarshalJSON() ([]byte, error) {
return json.Marshal(v.ValueOrDefault())
}
func (v *GeoMapCountryFormat) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid geo map country format: must be a string, one of `auto`, `alpha2`, `alpha3`, or `name`")
}
parsed := GeoMapCountryFormat{valuer.NewString(s)}
switch parsed {
case GeoMapCountryFormatAuto, GeoMapCountryFormatAlpha2, GeoMapCountryFormatAlpha3, GeoMapCountryFormatName:
*v = parsed
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid geo map country format %q: must be `auto`, `alpha2`, `alpha3`, or `name`", s)
}
}
type GeoMapLayer struct{ valuer.String }
var (
GeoMapLayerRegions = GeoMapLayer{valuer.NewString("regions")} // default
GeoMapLayerBubbles = GeoMapLayer{valuer.NewString("bubbles")}
GeoMapLayerPoints = GeoMapLayer{valuer.NewString("points")}
)
func (GeoMapLayer) Enum() []any {
return []any{GeoMapLayerRegions, GeoMapLayerBubbles, GeoMapLayerPoints}
}
func (v GeoMapLayer) ValueOrDefault() string {
if v.IsZero() {
return GeoMapLayerRegions.StringValue()
}
return v.StringValue()
}
func (v GeoMapLayer) MarshalJSON() ([]byte, error) {
return json.Marshal(v.ValueOrDefault())
}
func (v *GeoMapLayer) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid geo map layer: must be a string, one of `regions`, `bubbles`, or `points`")
}
parsed := GeoMapLayer{valuer.NewString(s)}
switch parsed {
case GeoMapLayerRegions, GeoMapLayerBubbles, GeoMapLayerPoints:
*v = parsed
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid geo map layer %q: must be `regions`, `bubbles`, or `points`", s)
}
}
type GeoMapView struct{ valuer.String }
var (
GeoMapViewWorld = GeoMapView{valuer.NewString("world")} // default
GeoMapViewFit = GeoMapView{valuer.NewString("fit")}
GeoMapViewAfrica = GeoMapView{valuer.NewString("africa")}
GeoMapViewAsia = GeoMapView{valuer.NewString("asia")}
GeoMapViewEurope = GeoMapView{valuer.NewString("europe")}
GeoMapViewNorthAmerica = GeoMapView{valuer.NewString("north_america")}
GeoMapViewSouthAmerica = GeoMapView{valuer.NewString("south_america")}
GeoMapViewOceania = GeoMapView{valuer.NewString("oceania")}
GeoMapViewCountry = GeoMapView{valuer.NewString("country")}
)
func (GeoMapView) Enum() []any {
return []any{GeoMapViewWorld, GeoMapViewFit, GeoMapViewAfrica, GeoMapViewAsia, GeoMapViewEurope, GeoMapViewNorthAmerica, GeoMapViewSouthAmerica, GeoMapViewOceania, GeoMapViewCountry}
}
func (v GeoMapView) ValueOrDefault() string {
if v.IsZero() {
return GeoMapViewWorld.StringValue()
}
return v.StringValue()
}
func (v GeoMapView) MarshalJSON() ([]byte, error) {
return json.Marshal(v.ValueOrDefault())
}
func (v *GeoMapView) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid geo map view: must be a string, one of `world`, `fit`, `africa`, `asia`, `europe`, `north_america`, `south_america`, `oceania`, or `country`")
}
parsed := GeoMapView{valuer.NewString(s)}
switch parsed {
case GeoMapViewWorld, GeoMapViewFit, GeoMapViewAfrica, GeoMapViewAsia, GeoMapViewEurope, GeoMapViewNorthAmerica, GeoMapViewSouthAmerica, GeoMapViewOceania, GeoMapViewCountry:
*v = parsed
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid geo map view %q: must be `world`, `fit`, `africa`, `asia`, `europe`, `north_america`, `south_america`, `oceania`, or `country`", s)
}
}
type GeoMapProjection struct{ valuer.String }
var (
GeoMapProjectionMercator = GeoMapProjection{valuer.NewString("mercator")} // default
GeoMapProjectionEqualEarth = GeoMapProjection{valuer.NewString("equal_earth")}
GeoMapProjectionNaturalEarth = GeoMapProjection{valuer.NewString("natural_earth")}
)
func (GeoMapProjection) Enum() []any {
return []any{GeoMapProjectionMercator, GeoMapProjectionEqualEarth, GeoMapProjectionNaturalEarth}
}
func (p GeoMapProjection) ValueOrDefault() string {
if p.IsZero() {
return GeoMapProjectionMercator.StringValue()
}
return p.StringValue()
}
func (p GeoMapProjection) MarshalJSON() ([]byte, error) {
return json.Marshal(p.ValueOrDefault())
}
func (p *GeoMapProjection) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid geo map projection: must be a string, one of `mercator`, `equal_earth`, or `natural_earth`")
}
parsed := GeoMapProjection{valuer.NewString(s)}
switch parsed {
case GeoMapProjectionMercator, GeoMapProjectionEqualEarth, GeoMapProjectionNaturalEarth:
*p = parsed
return nil
default:
return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid geo map projection %q: must be `mercator`, `equal_earth`, or `natural_earth`", s)
}
}

View File

@@ -5,7 +5,6 @@ import (
"encoding/json"
"maps"
"github.com/SigNoz/signoz/pkg/errors"
"github.com/SigNoz/signoz/pkg/types/cachetypes"
)
@@ -34,15 +33,6 @@ func (c *CachedBucket) Clone() *CachedBucket {
}
}
func (c *CachedBucket) Contains(other *CachedBucket) bool {
return c.StartMs <= other.StartMs && other.EndMs <= c.EndMs
}
// true if c and other overlap or sit edge to edge, false otherwise.
func (c *CachedBucket) Touches(other *CachedBucket) bool {
return c.StartMs <= other.EndMs && other.StartMs <= c.EndMs
}
// CachedData represents the full cached data for a query.
type CachedData struct {
Buckets []*CachedBucket `json:"buckets"`
@@ -87,53 +77,3 @@ func (c *CachedData) Cost() int64 {
}
return size
}
func (r *Result) SplitIntoCacheCellBuckets(startMs, endMs, cellWidthMs uint64) ([]*CachedBucket, error) {
tsData, ok := r.Value.(*TimeSeriesData)
if !ok || tsData == nil {
return nil, errors.NewInternalf(errors.CodeInternal, "result of type %s cannot be bucketed", r.Type.StringValue())
}
cellBuckets := make([]*CachedBucket, 0, 1)
for from := startMs; from < endMs; {
to := endMs
if cellWidthMs > 0 {
to = min(endMs, (from/cellWidthMs+1)*cellWidthMs)
}
// A promql value stamped exactly at the window end is an instant inside
// the window, so the last slice keeps it. A builder value never sits
// there, since the end is step-aligned and partial steps are dropped.
sliceEndMs := to
if to == endMs {
sliceEndMs = endMs + 1
}
value, err := json.Marshal(tsData.FilterToTimeRange(from, sliceEndMs))
if err != nil {
return nil, err
}
cellBuckets = append(cellBuckets, &CachedBucket{
StartMs: from,
EndMs: to,
Type: r.Type,
Value: value,
Stats: r.Stats.prorate(to-from, endMs-startMs),
})
from = to
}
return cellBuckets, nil
}
// prorate returns the share of the stats that partMs out of wholeMs of the
// scanned window accounts for.
func (e ExecStats) prorate(partMs, wholeMs uint64) ExecStats {
ratio := 1.0
if partMs < wholeMs {
ratio = float64(partMs) / float64(wholeMs)
}
return ExecStats{
RowsScanned: uint64(float64(e.RowsScanned) * ratio),
BytesScanned: uint64(float64(e.BytesScanned) * ratio),
DurationMS: uint64(float64(e.DurationMS) * ratio),
StepIntervals: maps.Clone(e.StepIntervals),
}
}

View File

@@ -137,28 +137,6 @@ type TimeSeriesData struct {
Aggregations []*AggregationBucket `json:"aggregations" description:"One bucket per aggregation of the query. The order is not guaranteed; match buckets to aggregations by index."`
}
// FilterToTimeRange copies the aggregation and series structure keeping only
// the values with startMs <= timestamp < endMs.
func (t *TimeSeriesData) FilterToTimeRange(startMs, endMs uint64) *TimeSeriesData {
filtered := &TimeSeriesData{
QueryName: t.QueryName,
Aggregations: make([]*AggregationBucket, 0, len(t.Aggregations)),
}
for _, aggBucket := range t.Aggregations {
filteredBucket := &AggregationBucket{
Index: aggBucket.Index,
Alias: aggBucket.Alias,
Meta: aggBucket.Meta,
Series: make([]*TimeSeries, 0, len(aggBucket.Series)),
}
for _, series := range aggBucket.Series {
filteredBucket.Series = append(filteredBucket.Series, series.filterToTimeRange(startMs, endMs))
}
filtered.Aggregations = append(filtered.Aggregations, filteredBucket)
}
return filtered
}
type AggregationBucket struct {
Index int `json:"index" description:"The zero-based position of the aggregation in the query."`
Alias string `json:"alias" description:"The alias of the aggregation, or the internal name __result_N, with N its index, when no alias was given."`
@@ -281,21 +259,6 @@ func (ts *TimeSeries) EvaluableValues() []*TimeSeriesValue {
return result
}
// filterToTimeRange copies the series keeping only the values with
// startMs <= timestamp < endMs.
func (ts *TimeSeries) filterToTimeRange(startMs, endMs uint64) *TimeSeries {
filtered := &TimeSeries{
Labels: ts.Labels,
Values: make([]*TimeSeriesValue, 0, len(ts.Values)),
}
for _, value := range ts.Values {
if timestamp := uint64(value.Timestamp); timestamp >= startMs && timestamp < endMs {
filtered.Values = append(filtered.Values, value)
}
}
return filtered
}
type Label struct {
Key telemetrytypes.TelemetryFieldKey `json:"key" description:"The group-by field this label value belongs to."`
Value any `json:"value" description:"The label value as a string, number or boolean."`

View File

@@ -436,97 +436,3 @@ func TestRoundToNonZeroDecimals(t *testing.T) {
assert.Equal(t, math.Inf(1), roundToNonZeroDecimals(math.Inf(1), 3))
assert.Equal(t, math.Inf(-1), roundToNonZeroDecimals(math.Inf(-1), 3))
}
func TestTimeSeriesDataFilterToTimeRange(t *testing.T) {
testCases := []struct {
name string
timeSeriesData *TimeSeriesData
startMs uint64
endMs uint64
expectedFiltered *TimeSeriesData
}{
{
name: "HalfOpenRange_KeepsStartDropsEnd",
timeSeriesData: &TimeSeriesData{
QueryName: "A",
Aggregations: []*AggregationBucket{{
Index: 0,
Series: []*TimeSeries{{
Labels: []*Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host"}, Value: "node-1"}},
Values: []*TimeSeriesValue{
{Timestamp: 1000, Value: 1},
{Timestamp: 2000, Value: 2},
{Timestamp: 3000, Value: 3},
{Timestamp: 4000, Value: 4},
},
}},
}},
},
startMs: 2000,
endMs: 4000,
expectedFiltered: &TimeSeriesData{
QueryName: "A",
Aggregations: []*AggregationBucket{{
Index: 0,
Series: []*TimeSeries{{
Labels: []*Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host"}, Value: "node-1"}},
Values: []*TimeSeriesValue{
{Timestamp: 2000, Value: 2},
{Timestamp: 3000, Value: 3},
},
}},
}},
},
},
{
// heatmap counts are positional against Meta.Buckets, so the axis
// has to survive even when no value falls in the range
name: "NothingInRange_KeepsSeriesAndHeatmapAxis",
timeSeriesData: &TimeSeriesData{
QueryName: "A",
Aggregations: []*AggregationBucket{{
Index: 1,
Alias: "__result_1",
Meta: AggregationMeta{Unit: "By", Buckets: []float64{1, 2, 4}},
Series: []*TimeSeries{{
Labels: []*Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host"}, Value: "node-1"}},
Values: []*TimeSeriesValue{{Timestamp: 1000, Values: []float64{1, 2, 3, 4}}},
}},
}},
},
startMs: 5000,
endMs: 6000,
expectedFiltered: &TimeSeriesData{
QueryName: "A",
Aggregations: []*AggregationBucket{{
Index: 1,
Alias: "__result_1",
Meta: AggregationMeta{Unit: "By", Buckets: []float64{1, 2, 4}},
Series: []*TimeSeries{{
Labels: []*Label{{Key: telemetrytypes.TelemetryFieldKey{Name: "host"}, Value: "node-1"}},
Values: []*TimeSeriesValue{},
}},
}},
},
},
{
name: "NoSeries_KeepsAggregation",
timeSeriesData: &TimeSeriesData{
QueryName: "A",
Aggregations: []*AggregationBucket{{Index: 0}},
},
startMs: 1000,
endMs: 2000,
expectedFiltered: &TimeSeriesData{
QueryName: "A",
Aggregations: []*AggregationBucket{{Index: 0, Series: []*TimeSeries{}}},
},
},
}
for _, testCase := range testCases {
t.Run(testCase.name, func(t *testing.T) {
assert.Equal(t, testCase.expectedFiltered, testCase.timeSeriesData.FilterToTimeRange(testCase.startMs, testCase.endMs))
})
}
}

View File

@@ -442,51 +442,3 @@ def test_builder_refreshing_a_sliding_time_range(
assert uncached.status_code == HTTPStatus.OK, uncached.text
assert_results_equal(from_cache.json(), uncached.json(), "A", "the twentieth refresh")
def test_builder_rows_scanned_when_refreshing(
signoz: types.SigNoz,
create_user_admin: None, # pylint: disable=unused-argument
get_token: Callable[[str, str], str],
insert_metrics: Callable[[list[Metrics]], None],
) -> None:
metric_name = f"cache_rows_scanned_{uuid4().hex[:8]}"
# 90 minutes back so even the last refresh closes clear of the flux
# interval, which holds recent data out of the cache
start_time = datetime.fromtimestamp(int((datetime.now(tz=UTC) - timedelta(minutes=90)).timestamp()) // 60 * 60, tz=UTC)
start_time_ms = int(start_time.timestamp() * 1000)
query = [build_builder_query("A", metric_name, "max", "max")]
insert_metrics(
[
Metrics(
metric_name=metric_name,
labels={"service": "api"},
timestamp=start_time + timedelta(minutes=minute),
value=minute,
type_="Gauge",
is_monotonic=False,
)
for minute in range(70)
]
)
token = get_token(USER_ADMIN_EMAIL, USER_ADMIN_PASSWORD)
# a dashboard left open on a one hour range, re-running a minute later each
# time. rowsScanned is what answering the request cost, so a refresh served
# mostly out of the cache cannot report more than reading its range afresh
for refresh in range(10):
refresh_start_ms = start_time_ms + refresh * MINUTE_MS
from_cache = make_query_request(signoz, token, refresh_start_ms, refresh_start_ms + 60 * MINUTE_MS, query, no_cache=False)
assert from_cache.status_code == HTTPStatus.OK, from_cache.text
uncached = make_query_request(signoz, token, refresh_start_ms, refresh_start_ms + 60 * MINUTE_MS, query, no_cache=True)
assert uncached.status_code == HTTPStatus.OK, uncached.text
rows_scanned_from_cache = from_cache.json()["data"]["meta"]["rowsScanned"]
rows_scanned_uncached = uncached.json()["data"]["meta"]["rowsScanned"]
assert rows_scanned_uncached > 0, f"refresh {refresh}"
assert rows_scanned_from_cache <= rows_scanned_uncached, f"refresh {refresh}: {rows_scanned_from_cache} rows scanned when served from the cache, {rows_scanned_uncached} when read afresh"