Compare commits

...

11 Commits

Author SHA1 Message Date
Nikhil Soni
40a036f2b0 chore(savedview): regenerate openapi spec and frontend api client
Reflects the SchemaVersion typed-enum change: schemaVersion is now a
$ref to its own SavedviewtypesSchemaVersion component instead of an
inline enum on each type.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-09 10:50:13 +05:30
Nikhil Soni
d752452d40 fix(savedview): typed schemaVersion enum, close legacy zero-query gap
Three follow-ups from reviewing the earlier schemaVersion/optional-field
changes:

- Replace the jsonschema.Preparer hack + SavedViewMetadataBase wrapper for
  schemaVersion with a dedicated SchemaVersion type (mirrors PanelType /
  Source: valuer.String + Enum() + Validate()). Unlike rules, savedviews
  only ever has one valid value, so there's no accept/publish asymmetry
  that would justify hand-mutating the schema -- the plain enum type
  publishes and validates the same single value. SchemaVersion is now a
  direct field on SavedView/PostableSavedView/UpdatableSavedView instead
  of being embedded via SavedViewMetadataBase.
- Add test coverage proving selectedFields/display becoming non-required
  is safe: omitted, null, and empty ([]/{}) all decode and validate
  cleanly, and partially-populated display is unaffected.
- Fix a real validation gap in the v1 handlers: newPostableSavedViewFromLegacyView
  / newUpdatableSavedViewFromLegacyView convert a legacy v3.SavedView into
  the typed postable/updatable, but a legacy payload with a present-but-
  empty query map (e.g. "builderQueries": {}) passes v3.CompositeQuery.Validate()
  despite having zero actual queries. Create/Update now also call
  postable.Validate() / updatable.Validate() on the converted payload,
  which correctly rejects it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-09 10:48:44 +05:30
Nikhil Soni
4a8a1efb03 fix(sqlmigration): generalize migration 111 repair to every spec field
The old repair only special-cased selectedFields, so any other unmarshal
failure (e.g. an incompatible queries shape) still left the row broken --
every future Get/List of that saved view would 500. Now every spec key
that fails to unmarshal into its real type is individually blanked to its
zero value, and rows that still don't unmarshal cleanly afterward are
replaced with a guaranteed-valid placeholder view instead of being left
corrupted in place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-09 10:48:06 +05:30
Nikhil Soni
7d2afa5a54 chore(savedview): regenerate openapi spec and frontend api client
Reflects the five savedview schema fixes: unwrapped data, schemaVersion
enum, optional selectedFields/display, 409 on create, minItems on queries.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-08 17:57:28 +05:30
Nikhil Soni
b579190140 fix(savedview): require at least one query
The server already rejects an empty queries list (CompositeQuery.Validate),
the schema just didn't say so. Mirrors #12112, which added the same minItems
constraint to the rules equivalent.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-08 17:56:16 +05:30
Nikhil Soni
6222204af7 fix(savedview): declare 409 on CreateSavedView
Names are org-unique and the server returns 409 on a collision; the
OpenAPI operation just didn't say so. Eleven other create endpoints already
declare it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-08 17:55:49 +05:30
Nikhil Soni
0fcd6aeada fix(savedview): selectedFields and display are not required
Neither field is actually validated server-side (SavedViewSpec.Validate
never checks them), so marking them required in the schema over-constrains
callers for no enforced reason.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-08 17:55:29 +05:30
Nikhil Soni
32af113d0c fix(savedview): restrict published schemaVersion to enum [v2]
schemaVersion was published as a bare string, letting a client send anything.
Mirrors ruletypes.PostableRule's own schemaVersion override (see #12112) --
v2 here, not v2alpha1, since that's the rules value. Attached to
SavedViewMetadataBase so it applies to SavedView, PostableSavedView and
UpdatableSavedView through method promotion.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-08 17:54:56 +05:30
Nikhil Soni
61588b9303 refactor(savedview): unwrap data, promote schemaVersion/spec to top level
Matches dashboardtypes.DashboardV2 and the v2alpha1 rule shape: schemaVersion
and spec are now top-level fields on SavedView/PostableSavedView/
UpdatableSavedView instead of nested under a data object. StorableSavedView
is introduced as the distinct bun-mapped row shape -- bun maps a single
opaque `data` column, which is incompatible with promoting its fields to the
top level for JSON, so the two diverge the same way Dashboard/StorableDashboard
do.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnYR48DXJQGxqQSEW3237h
2026-08-08 17:54:36 +05:30
Nikhil Soni
dbc40efe24 chore: add migration to fix the already migrated data 2026-08-08 17:28:02 +05:30
Nikhil Soni
33a428496b fix: handle malformed selectField data and throw error in v1 apis 2026-08-08 17:05:53 +05:30
15 changed files with 623 additions and 254 deletions

View File

@@ -7880,17 +7880,20 @@ components:
type: string
SavedviewtypesPostableSavedView:
properties:
data:
$ref: '#/components/schemas/SavedviewtypesSavedViewData'
generateName:
type: boolean
name:
type: string
schemaVersion:
$ref: '#/components/schemas/SavedviewtypesSchemaVersion'
source:
$ref: '#/components/schemas/SavedviewtypesSource'
spec:
$ref: '#/components/schemas/SavedviewtypesSavedViewSpec'
required:
- source
- data
- schemaVersion
- spec
type: object
SavedviewtypesSavedView:
properties:
@@ -7899,14 +7902,16 @@ components:
type: string
createdBy:
type: string
data:
$ref: '#/components/schemas/SavedviewtypesSavedViewData'
id:
type: string
name:
type: string
schemaVersion:
$ref: '#/components/schemas/SavedviewtypesSchemaVersion'
source:
$ref: '#/components/schemas/SavedviewtypesSource'
spec:
$ref: '#/components/schemas/SavedviewtypesSavedViewSpec'
updatedAt:
format: date-time
type: string
@@ -7914,14 +7919,6 @@ components:
type: string
required:
- id
type: object
SavedviewtypesSavedViewData:
properties:
schemaVersion:
type: string
spec:
$ref: '#/components/schemas/SavedviewtypesSavedViewSpec'
required:
- schemaVersion
- spec
type: object
@@ -7936,6 +7933,7 @@ components:
queries:
items:
$ref: '#/components/schemas/Querybuildertypesv5QueryEnvelope'
minItems: 1
type: array
selectedFields:
items:
@@ -7945,9 +7943,11 @@ components:
- displayName
- panelType
- queries
- selectedFields
- display
type: object
SavedviewtypesSchemaVersion:
enum:
- v2
type: string
SavedviewtypesSource:
enum:
- traces
@@ -7957,13 +7957,16 @@ components:
type: string
SavedviewtypesUpdatableSavedView:
properties:
data:
$ref: '#/components/schemas/SavedviewtypesSavedViewData'
schemaVersion:
$ref: '#/components/schemas/SavedviewtypesSchemaVersion'
source:
$ref: '#/components/schemas/SavedviewtypesSource'
spec:
$ref: '#/components/schemas/SavedviewtypesSavedViewSpec'
required:
- source
- data
- schemaVersion
- spec
type: object
ServiceaccounttypesDeprecatedPostableServiceAccountRole:
properties:
@@ -22972,6 +22975,12 @@ paths:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Forbidden
"409":
content:
application/json:
schema:
$ref: '#/components/schemas/RenderErrorResponse'
description: Conflict
"500":
content:
application/json:

View File

@@ -8991,8 +8991,17 @@ export enum SavedviewtypesPanelTypeDTO {
list = 'list',
trace = 'trace',
}
export enum SavedviewtypesSchemaVersionDTO {
v2 = 'v2',
}
export enum SavedviewtypesSourceDTO {
traces = 'traces',
logs = 'logs',
metrics = 'metrics',
meter = 'meter',
}
export interface SavedviewtypesSavedViewSpecDTO {
display: SavedviewtypesDisplayDTO;
display?: SavedviewtypesDisplayDTO;
/**
* @type string
*/
@@ -9005,25 +9014,10 @@ export interface SavedviewtypesSavedViewSpecDTO {
/**
* @type array
*/
selectedFields: TelemetrytypesTelemetryFieldKeyDTO[];
selectedFields?: TelemetrytypesTelemetryFieldKeyDTO[];
}
export interface SavedviewtypesSavedViewDataDTO {
/**
* @type string
*/
schemaVersion: string;
spec: SavedviewtypesSavedViewSpecDTO;
}
export enum SavedviewtypesSourceDTO {
traces = 'traces',
logs = 'logs',
metrics = 'metrics',
meter = 'meter',
}
export interface SavedviewtypesPostableSavedViewDTO {
data: SavedviewtypesSavedViewDataDTO;
/**
* @type boolean
*/
@@ -9032,7 +9026,9 @@ export interface SavedviewtypesPostableSavedViewDTO {
* @type string
*/
name?: string;
schemaVersion: SavedviewtypesSchemaVersionDTO;
source: SavedviewtypesSourceDTO;
spec: SavedviewtypesSavedViewSpecDTO;
}
export interface SavedviewtypesSavedViewDTO {
@@ -9045,7 +9041,6 @@ export interface SavedviewtypesSavedViewDTO {
* @type string
*/
createdBy?: string;
data?: SavedviewtypesSavedViewDataDTO;
/**
* @type string
*/
@@ -9054,7 +9049,9 @@ export interface SavedviewtypesSavedViewDTO {
* @type string
*/
name?: string;
schemaVersion: SavedviewtypesSchemaVersionDTO;
source?: SavedviewtypesSourceDTO;
spec: SavedviewtypesSavedViewSpecDTO;
/**
* @type string
* @format date-time
@@ -9067,8 +9064,9 @@ export interface SavedviewtypesSavedViewDTO {
}
export interface SavedviewtypesUpdatableSavedViewDTO {
data: SavedviewtypesSavedViewDataDTO;
schemaVersion: SavedviewtypesSchemaVersionDTO;
source: SavedviewtypesSourceDTO;
spec: SavedviewtypesSavedViewSpecDTO;
}
export interface ServiceaccounttypesDeprecatedPostableServiceAccountRoleDTO {

View File

@@ -51,7 +51,7 @@ func (provider *provider) addSavedViewRoutes(router *mux.Router) error {
Response: new(types.Identifiable),
ResponseContentType: "application/json",
SuccessStatusCode: http.StatusCreated,
ErrorStatusCodes: []int{http.StatusBadRequest},
ErrorStatusCodes: []int{http.StatusBadRequest, http.StatusConflict},
Deprecated: false,
SecuritySchemes: newScopedSecuritySchemes([]string{coretypes.ResourceMetaResourceSavedView.Scope(coretypes.VerbCreate)}),
},

View File

@@ -36,70 +36,73 @@ type legacyExtraData struct {
}
// newPostableSavedViewFromLegacyView builds a create payload for a v1 request.
func newPostableSavedViewFromLegacyView(v *v3.SavedView) savedviewtypes.PostableSavedView {
// Unlike migration 109 (which processes historical rows it can't ask anyone
// about), this is a live request with a caller on the other end -- malformed
// extraData is rejected outright rather than silently written as a partial row.
func newPostableSavedViewFromLegacyView(v *v3.SavedView) (savedviewtypes.PostableSavedView, error) {
var legacy legacyExtraData
if v.ExtraData != "" {
// Best-effort: malformed/older extraData shapes never fail the request
_ = json.Unmarshal([]byte(v.ExtraData), &legacy)
if err := json.Unmarshal([]byte(v.ExtraData), &legacy); err != nil {
return savedviewtypes.PostableSavedView{}, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse extraData")
}
}
return savedviewtypes.PostableSavedView{
GenerateName: true,
Source: savedviewtypes.Source{String: valuer.NewString(v.SourcePage)},
Data: savedviewtypes.SavedViewData{
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: v.Name,
PanelType: savedviewtypes.PanelType{String: valuer.NewString(string(v.CompositeQuery.PanelType))},
Queries: v.CompositeQuery.Queries,
SelectedFields: legacy.SelectColumns,
Display: savedviewtypes.Display{
MaxLines: legacy.MaxLines,
FontSize: legacy.FontSize,
Format: legacy.Format,
Color: legacy.Color,
},
GenerateName: true,
Source: savedviewtypes.Source{String: valuer.NewString(v.SourcePage)},
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: v.Name,
PanelType: savedviewtypes.PanelType{String: valuer.NewString(string(v.CompositeQuery.PanelType))},
Queries: v.CompositeQuery.Queries,
SelectedFields: legacy.SelectColumns,
Display: savedviewtypes.Display{
MaxLines: legacy.MaxLines,
FontSize: legacy.FontSize,
Format: legacy.Format,
Color: legacy.Color,
},
},
}
}, nil
}
// newUpdatableSavedViewFromLegacyView builds an update payload for a v1 request.
func newUpdatableSavedViewFromLegacyView(v *v3.SavedView) savedviewtypes.UpdatableSavedView {
// See newPostableSavedViewFromLegacyView -- same reasoning for rejecting rather
// than swallowing a malformed extraData.
func newUpdatableSavedViewFromLegacyView(v *v3.SavedView) (savedviewtypes.UpdatableSavedView, error) {
var legacy legacyExtraData
if v.ExtraData != "" {
// Best-effort: malformed/older extraData shapes never fail the request
_ = json.Unmarshal([]byte(v.ExtraData), &legacy)
if err := json.Unmarshal([]byte(v.ExtraData), &legacy); err != nil {
return savedviewtypes.UpdatableSavedView{}, errors.Wrapf(err, errors.TypeInvalidInput, errors.CodeInvalidInput, "failed to parse extraData")
}
}
return savedviewtypes.UpdatableSavedView{
Source: savedviewtypes.Source{String: valuer.NewString(v.SourcePage)},
Data: savedviewtypes.SavedViewData{
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: v.Name,
PanelType: savedviewtypes.PanelType{String: valuer.NewString(string(v.CompositeQuery.PanelType))},
Queries: v.CompositeQuery.Queries,
SelectedFields: legacy.SelectColumns,
Display: savedviewtypes.Display{
MaxLines: legacy.MaxLines,
FontSize: legacy.FontSize,
Format: legacy.Format,
Color: legacy.Color,
},
Source: savedviewtypes.Source{String: valuer.NewString(v.SourcePage)},
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: v.Name,
PanelType: savedviewtypes.PanelType{String: valuer.NewString(string(v.CompositeQuery.PanelType))},
Queries: v.CompositeQuery.Queries,
SelectedFields: legacy.SelectColumns,
Display: savedviewtypes.Display{
MaxLines: legacy.MaxLines,
FontSize: legacy.FontSize,
Format: legacy.Format,
Color: legacy.Color,
},
},
}
}, nil
}
// newLegacyViewFromSavedView renders a v2 SavedView back into the v1 shape.
func newLegacyViewFromSavedView(v *savedviewtypes.SavedView) (*v3.SavedView, error) {
extraData, err := json.Marshal(legacyExtraData{
Color: v.Data.Spec.Display.Color,
SelectColumns: v.Data.Spec.SelectedFields,
Format: v.Data.Spec.Display.Format,
MaxLines: v.Data.Spec.Display.MaxLines,
FontSize: v.Data.Spec.Display.FontSize,
Color: v.Spec.Display.Color,
SelectColumns: v.Spec.SelectedFields,
Format: v.Spec.Display.Format,
MaxLines: v.Spec.Display.MaxLines,
FontSize: v.Spec.Display.FontSize,
})
if err != nil {
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in marshalling extra data")
@@ -107,17 +110,17 @@ func newLegacyViewFromSavedView(v *savedviewtypes.SavedView) (*v3.SavedView, err
return &v3.SavedView{
ID: v.ID,
Name: v.Data.Spec.DisplayName,
Name: v.Spec.DisplayName,
CreatedAt: v.CreatedAt,
CreatedBy: v.CreatedBy,
UpdatedAt: v.UpdatedAt,
UpdatedBy: v.UpdatedBy,
SourcePage: v.Source.StringValue(),
CompositeQuery: &v3.CompositeQuery{
PanelType: v3.PanelType(v.Data.Spec.PanelType.StringValue()),
PanelType: v3.PanelType(v.Spec.PanelType.StringValue()),
// Saved views are only ever created from the explorer's builder mode.
QueryType: v3.QueryTypeBuilder,
Queries: v.Data.Spec.Queries,
Queries: v.Spec.Queries,
},
ExtraData: string(extraData),
}, nil
@@ -156,7 +159,18 @@ func (handler *handler) Create(w http.ResponseWriter, r *http.Request) {
return
}
uuid, err := handler.module.CreateView(ctx, claims.OrgID, newPostableSavedViewFromLegacyView(&view))
postable, err := newPostableSavedViewFromLegacyView(&view)
if err != nil {
render.Error(w, err)
return
}
if err := postable.Validate(); err != nil {
render.Error(w, err)
return
}
uuid, err := handler.module.CreateView(ctx, claims.OrgID, postable)
if err != nil {
render.Error(w, err)
return
@@ -224,12 +238,22 @@ func (handler *handler) Update(w http.ResponseWriter, r *http.Request) {
return
}
err = handler.module.UpdateView(ctx, claims.OrgID, viewUUID, newUpdatableSavedViewFromLegacyView(&view))
updatable, err := newUpdatableSavedViewFromLegacyView(&view)
if err != nil {
render.Error(w, err)
return
}
if err := updatable.Validate(); err != nil {
render.Error(w, err)
return
}
if err := handler.module.UpdateView(ctx, claims.OrgID, viewUUID, updatable); err != nil {
render.Error(w, err)
return
}
render.Success(w, http.StatusOK, view)
}

View File

@@ -38,17 +38,18 @@ func TestNewPostableSavedViewFromLegacyView(t *testing.T) {
ExtraData: `{"color":"blue","selectColumns":[{"name":"service.name"}],"format":"table","maxLines":10,"fontSize":"large"}`,
}
postable := newPostableSavedViewFromLegacyView(legacy)
postable, err := newPostableSavedViewFromLegacyView(legacy)
require.NoError(t, err)
assert.Empty(t, postable.Name, "v1 has no slug concept -- name must always be generated")
assert.True(t, postable.GenerateName, "v1 has no slug concept -- name must always be generated")
assert.Equal(t, "my view", postable.Data.Spec.DisplayName)
assert.Equal(t, "my view", postable.Spec.DisplayName)
assert.Equal(t, savedviewtypes.SourceLogs, postable.Source)
assert.Equal(t, savedviewtypes.SavedViewSchemaVersion, postable.Data.SchemaVersion)
assert.Equal(t, savedviewtypes.PanelTypeGraph, postable.Data.Spec.PanelType)
assert.Equal(t, legacy.CompositeQuery.Queries, postable.Data.Spec.Queries)
assert.Equal(t, []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}}, postable.Data.Spec.SelectedFields)
assert.Equal(t, savedviewtypes.Display{MaxLines: 10, FontSize: "large", Format: "table", Color: "blue"}, postable.Data.Spec.Display)
assert.Equal(t, savedviewtypes.SavedViewSchemaVersion, postable.SchemaVersion)
assert.Equal(t, savedviewtypes.PanelTypeGraph, postable.Spec.PanelType)
assert.Equal(t, legacy.CompositeQuery.Queries, postable.Spec.Queries)
assert.Equal(t, []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}}, postable.Spec.SelectedFields)
assert.Equal(t, savedviewtypes.Display{MaxLines: 10, FontSize: "large", Format: "table", Color: "blue"}, postable.Spec.Display)
})
t.Run("empty extra data leaves display and selected fields zero-valued", func(t *testing.T) {
@@ -62,13 +63,14 @@ func TestNewPostableSavedViewFromLegacyView(t *testing.T) {
ExtraData: "",
}
postable := newPostableSavedViewFromLegacyView(legacy)
postable, err := newPostableSavedViewFromLegacyView(legacy)
require.NoError(t, err)
assert.Equal(t, savedviewtypes.Display{}, postable.Data.Spec.Display)
assert.Nil(t, postable.Data.Spec.SelectedFields)
assert.Equal(t, savedviewtypes.Display{}, postable.Spec.Display)
assert.Nil(t, postable.Spec.SelectedFields)
})
t.Run("malformed extra data is ignored, not an error", func(t *testing.T) {
t.Run("malformed extra data is rejected", func(t *testing.T) {
legacy := &v3.SavedView{
Name: "malformed extra data",
SourcePage: "metrics",
@@ -79,10 +81,26 @@ func TestNewPostableSavedViewFromLegacyView(t *testing.T) {
ExtraData: `{not valid json`,
}
postable := newPostableSavedViewFromLegacyView(legacy)
_, err := newPostableSavedViewFromLegacyView(legacy)
assert.Error(t, err)
})
assert.Equal(t, "malformed extra data", postable.Data.Spec.DisplayName)
assert.Equal(t, savedviewtypes.Display{}, postable.Data.Spec.Display)
t.Run("legacy validation gap: empty builderQueries map with no queries", func(t *testing.T) {
legacy := &v3.SavedView{
Name: "no real queries",
SourcePage: "logs",
CompositeQuery: &v3.CompositeQuery{
PanelType: v3.PanelTypeGraph,
QueryType: v3.QueryTypeBuilder,
BuilderQueries: map[string]*v3.BuilderQuery{},
},
}
require.NoError(t, legacy.Validate(), "the legacy CompositeQuery check is expected to miss this")
postable, err := newPostableSavedViewFromLegacyView(legacy)
require.NoError(t, err)
assert.Error(t, postable.Validate(), "the converted postable must catch what the legacy check missed")
})
}
@@ -97,26 +115,25 @@ func TestNewUpdatableSavedViewFromLegacyView(t *testing.T) {
ExtraData: `{"color":"red"}`,
}
updatable := newUpdatableSavedViewFromLegacyView(legacy)
updatable, err := newUpdatableSavedViewFromLegacyView(legacy)
require.NoError(t, err)
assert.Equal(t, "renamed view", updatable.Data.Spec.DisplayName)
assert.Equal(t, "renamed view", updatable.Spec.DisplayName)
assert.Equal(t, savedviewtypes.SourceTraces, updatable.Source)
}
func TestNewLegacyViewFromSavedView(t *testing.T) {
now := time.Now()
savedView := &savedviewtypes.SavedView{
Name: "my-view-abc123ef",
Source: savedviewtypes.SourceLogs,
Data: savedviewtypes.SavedViewData{
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: "my view",
PanelType: savedviewtypes.PanelTypeGraph,
Queries: testQueries(),
SelectedFields: []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}},
Display: savedviewtypes.Display{MaxLines: 10, FontSize: "large", Format: "table", Color: "blue"},
},
Name: "my-view-abc123ef",
Source: savedviewtypes.SourceLogs,
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: "my view",
PanelType: savedviewtypes.PanelTypeGraph,
Queries: testQueries(),
SelectedFields: []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}},
Display: savedviewtypes.Display{MaxLines: 10, FontSize: "large", Format: "table", Color: "blue"},
},
}
savedView.ID = valuer.GenerateUUID()
@@ -129,7 +146,7 @@ func TestNewLegacyViewFromSavedView(t *testing.T) {
require.NoError(t, err)
assert.Equal(t, savedView.ID, legacy.ID)
assert.Equal(t, savedView.Data.Spec.DisplayName, legacy.Name)
assert.Equal(t, savedView.Spec.DisplayName, legacy.Name)
assert.Equal(t, savedView.CreatedAt, legacy.CreatedAt)
assert.Equal(t, savedView.CreatedBy, legacy.CreatedBy)
assert.Equal(t, savedView.UpdatedAt, legacy.UpdatedAt)
@@ -137,20 +154,20 @@ func TestNewLegacyViewFromSavedView(t *testing.T) {
assert.Equal(t, "logs", legacy.SourcePage)
assert.Equal(t, v3.PanelTypeGraph, legacy.CompositeQuery.PanelType)
assert.Equal(t, v3.QueryTypeBuilder, legacy.CompositeQuery.QueryType)
assert.Equal(t, savedView.Data.Spec.Queries, legacy.CompositeQuery.Queries)
assert.Equal(t, savedView.Spec.Queries, legacy.CompositeQuery.Queries)
var extra legacyExtraData
require.NoError(t, json.Unmarshal([]byte(legacy.ExtraData), &extra))
assert.Equal(t, "blue", extra.Color)
assert.Equal(t, savedView.Data.Spec.SelectedFields, extra.SelectColumns)
assert.Equal(t, savedView.Spec.SelectedFields, extra.SelectColumns)
assert.Equal(t, "table", extra.Format)
assert.Equal(t, 10, extra.MaxLines)
assert.Equal(t, "large", extra.FontSize)
}
func TestNewLegacyViewsFromSavedViews(t *testing.T) {
a := &savedviewtypes.SavedView{Name: "a-slug", Source: savedviewtypes.SourceLogs, Data: savedviewtypes.SavedViewData{Spec: savedviewtypes.SavedViewSpec{DisplayName: "a", PanelType: savedviewtypes.PanelTypeGraph, Queries: testQueries()}}}
b := &savedviewtypes.SavedView{Name: "b-slug", Source: savedviewtypes.SourceTraces, Data: savedviewtypes.SavedViewData{Spec: savedviewtypes.SavedViewSpec{DisplayName: "b", PanelType: savedviewtypes.PanelTypeTable, Queries: testQueries()}}}
a := &savedviewtypes.SavedView{Name: "a-slug", Source: savedviewtypes.SourceLogs, Spec: savedviewtypes.SavedViewSpec{DisplayName: "a", PanelType: savedviewtypes.PanelTypeGraph, Queries: testQueries()}}
b := &savedviewtypes.SavedView{Name: "b-slug", Source: savedviewtypes.SourceTraces, Spec: savedviewtypes.SavedViewSpec{DisplayName: "b", PanelType: savedviewtypes.PanelTypeTable, Queries: testQueries()}}
legacyViews, err := newLegacyViewsFromSavedViews([]*savedviewtypes.SavedView{a, b})
require.NoError(t, err)
@@ -167,31 +184,58 @@ func TestNewLegacyViewsFromSavedViews(t *testing.T) {
// slug (Name) is deliberately NOT part of this contract -- v1 never sees it.
func TestLegacyViewRoundTrip(t *testing.T) {
original := &savedviewtypes.SavedView{
Name: "round-trip-abc123ef",
Source: savedviewtypes.SourceMetrics,
Data: savedviewtypes.SavedViewData{
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: "round trip",
PanelType: savedviewtypes.PanelTypeTable,
Queries: testQueries(),
SelectedFields: []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}},
Display: savedviewtypes.Display{MaxLines: 5, FontSize: "small", Format: "list", Color: "red"},
},
Name: "round-trip-abc123ef",
Source: savedviewtypes.SourceMetrics,
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: "round trip",
PanelType: savedviewtypes.PanelTypeTable,
Queries: testQueries(),
SelectedFields: []telemetrytypes.TelemetryFieldKey{{Name: "service.name"}},
Display: savedviewtypes.Display{MaxLines: 5, FontSize: "small", Format: "list", Color: "red"},
},
}
legacy, err := newLegacyViewFromSavedView(original)
require.NoError(t, err)
roundTripped := newPostableSavedViewFromLegacyView(legacy)
roundTripped, err := newPostableSavedViewFromLegacyView(legacy)
require.NoError(t, err)
assert.Empty(t, roundTripped.Name)
assert.True(t, roundTripped.GenerateName)
assert.Equal(t, original.Data.Spec.DisplayName, roundTripped.Data.Spec.DisplayName)
assert.Equal(t, original.Spec.DisplayName, roundTripped.Spec.DisplayName)
assert.Equal(t, original.Source, roundTripped.Source)
assert.Equal(t, original.Data.Spec.PanelType, roundTripped.Data.Spec.PanelType)
assert.Equal(t, original.Data.Spec.Queries, roundTripped.Data.Spec.Queries)
assert.Equal(t, original.Data.Spec.SelectedFields, roundTripped.Data.Spec.SelectedFields)
assert.Equal(t, original.Data.Spec.Display, roundTripped.Data.Spec.Display)
assert.Equal(t, original.Spec.PanelType, roundTripped.Spec.PanelType)
assert.Equal(t, original.Spec.Queries, roundTripped.Spec.Queries)
assert.Equal(t, original.Spec.SelectedFields, roundTripped.Spec.SelectedFields)
assert.Equal(t, original.Spec.Display, roundTripped.Spec.Display)
}
func TestLegacyViewRoundTrip_EmptySelectedFieldsAndDisplay(t *testing.T) {
original := &savedviewtypes.SavedView{
Name: "round-trip-empty-abc123ef",
Source: savedviewtypes.SourceMetrics,
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: "round trip empty",
PanelType: savedviewtypes.PanelTypeTable,
Queries: testQueries(),
SelectedFields: []telemetrytypes.TelemetryFieldKey{},
Display: savedviewtypes.Display{},
},
}
legacy, err := newLegacyViewFromSavedView(original)
require.NoError(t, err)
var extra legacyExtraData
require.NoError(t, json.Unmarshal([]byte(legacy.ExtraData), &extra))
assert.Nil(t, extra.SelectColumns, "omitempty drops an empty selectColumns from extraData entirely")
roundTripped, err := newPostableSavedViewFromLegacyView(legacy)
require.NoError(t, err)
assert.Empty(t, roundTripped.Spec.SelectedFields, "empty, not necessarily non-nil, on this leg of the round trip")
assert.Equal(t, savedviewtypes.Display{}, roundTripped.Spec.Display)
}

View File

@@ -28,24 +28,22 @@ func newTestStore() (savedview.Module, *savedviewtypestest.StoreTest) {
func testPostableSavedView(name string, source savedviewtypes.Source) savedviewtypes.PostableSavedView {
return savedviewtypes.PostableSavedView{
Name: name,
Source: source,
Data: savedviewtypes.SavedViewData{
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: name,
PanelType: savedviewtypes.PanelTypeGraph,
Queries: []qbtypes.QueryEnvelope{
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Aggregations: []qbtypes.LogAggregation{{Expression: "count()"}},
},
Name: name,
Source: source,
SchemaVersion: savedviewtypes.SavedViewSchemaVersion,
Spec: savedviewtypes.SavedViewSpec{
DisplayName: name,
PanelType: savedviewtypes.PanelTypeGraph,
Queries: []qbtypes.QueryEnvelope{
{
Type: qbtypes.QueryTypeBuilder,
Spec: qbtypes.QueryBuilderQuery[qbtypes.LogAggregation]{
Signal: telemetrytypes.SignalLogs,
Aggregations: []qbtypes.LogAggregation{{Expression: "count()"}},
},
},
SelectedFields: []telemetrytypes.TelemetryFieldKey{},
},
SelectedFields: []telemetrytypes.TelemetryFieldKey{},
},
}
}
@@ -53,8 +51,9 @@ func testPostableSavedView(name string, source savedviewtypes.Source) savedviewt
func testUpdatableSavedView(displayName string, source savedviewtypes.Source) savedviewtypes.UpdatableSavedView {
postable := testPostableSavedView(displayName, source)
return savedviewtypes.UpdatableSavedView{
Source: postable.Source,
Data: postable.Data,
Source: postable.Source,
SchemaVersion: postable.SchemaVersion,
Spec: postable.Spec,
}
}
@@ -93,7 +92,7 @@ func TestModule_CreateAndGetView(t *testing.T) {
assert.Equal(t, savedviewtypes.SourceLogs, got.Source)
assert.Equal(t, "creator@signoz.io", got.CreatedBy)
assert.Equal(t, "creator@signoz.io", got.UpdatedBy)
assert.Equal(t, savedviewtypes.PanelTypeGraph, got.Data.Spec.PanelType)
assert.Equal(t, savedviewtypes.PanelTypeGraph, got.Spec.PanelType)
require.NoError(t, st.AssertExpectations())
}
@@ -138,21 +137,21 @@ func TestModule_UpdateView(t *testing.T) {
existingName := existing.Name
updated := testUpdatableSavedView("renamed", savedviewtypes.SourceTraces)
updated.Data.Spec.PanelType = savedviewtypes.PanelTypeTable
updated.Spec.PanelType = savedviewtypes.PanelTypeTable
st.ExpectUpdate(orgID, id, 1)
require.NoError(t, m.UpdateView(contextWithClaims(orgID, "updater@signoz.io"), orgID, id, updated))
stored := testSavedView(orgID, id, "updater@signoz.io", testPostableSavedView("renamed", savedviewtypes.SourceTraces))
stored.Name = existingName
stored.Data.Spec.PanelType = savedviewtypes.PanelTypeTable
stored.Spec.PanelType = savedviewtypes.PanelTypeTable
st.ExpectGet(orgID, id, stored)
got, err := m.GetView(contextWithClaims(orgID, "creator@signoz.io"), orgID, id)
require.NoError(t, err)
assert.Equal(t, existingName, got.Name, "name must not change on update")
assert.Equal(t, "renamed", got.Data.Spec.DisplayName)
assert.Equal(t, "renamed", got.Spec.DisplayName)
assert.Equal(t, savedviewtypes.SourceTraces, got.Source)
assert.Equal(t, savedviewtypes.PanelTypeTable, got.Data.Spec.PanelType)
assert.Equal(t, savedviewtypes.PanelTypeTable, got.Spec.PanelType)
assert.Equal(t, "updater@signoz.io", got.UpdatedBy)
require.NoError(t, st.AssertExpectations())

View File

@@ -19,7 +19,8 @@ func NewStore(sqlstore sqlstore.SQLStore) savedviewtypes.Store {
}
func (store *store) Create(ctx context.Context, view *savedviewtypes.SavedView) error {
_, err := store.sqlstore.BunDB().NewInsert().Model(view).Exec(ctx)
storable := savedviewtypes.NewStorableSavedView(view)
_, err := store.sqlstore.BunDB().NewInsert().Model(storable).Exec(ctx)
if err != nil {
return store.sqlstore.WrapAlreadyExistsErrf(err, errors.CodeAlreadyExists, "saved view with name %s already exists", view.Name)
}
@@ -27,23 +28,25 @@ func (store *store) Create(ctx context.Context, view *savedviewtypes.SavedView)
}
func (store *store) Get(ctx context.Context, orgID string, id valuer.UUID) (*savedviewtypes.SavedView, error) {
var view savedviewtypes.SavedView
err := store.sqlstore.BunDB().NewSelect().Model(&view).Where("org_id = ? AND id = ?", orgID, id.StringValue()).Scan(ctx)
var storable savedviewtypes.StorableSavedView
err := store.sqlstore.BunDB().NewSelect().Model(&storable).Where("org_id = ? AND id = ?", orgID, id.StringValue()).Scan(ctx)
if err != nil {
return nil, store.sqlstore.WrapNotFoundErrf(err, savedviewtypes.ErrCodeSavedViewNotFound, "saved view %s not found", id.StringValue())
}
normalizeSelectedFields(&view)
return &view, nil
view := storable.ToSavedView()
normalizeSelectedFields(view)
return view, nil
}
func (store *store) Update(ctx context.Context, view *savedviewtypes.SavedView) error {
storable := savedviewtypes.NewStorableSavedView(view)
res, err := store.sqlstore.BunDB().NewUpdate().
Model(&savedviewtypes.SavedView{}).
Model((*savedviewtypes.StorableSavedView)(nil)).
Set("updated_at = ?, updated_by = ?, source = ?, data = ?",
view.UpdatedAt, view.UpdatedBy, view.Source, view.Data).
Where("id = ?", view.ID.StringValue()).
Where("org_id = ?", view.OrgID).
storable.UpdatedAt, storable.UpdatedBy, storable.Source, storable.Data).
Where("id = ?", storable.ID.StringValue()).
Where("org_id = ?", storable.OrgID).
Exec(ctx)
if err != nil {
return errors.WrapInternalf(err, errors.CodeInternal, "error in updating saved view")
@@ -62,7 +65,7 @@ func (store *store) Update(ctx context.Context, view *savedviewtypes.SavedView)
func (store *store) Delete(ctx context.Context, orgID string, id valuer.UUID) error {
res, err := store.sqlstore.BunDB().NewDelete().
Model(&savedviewtypes.SavedView{}).
Model((*savedviewtypes.StorableSavedView)(nil)).
Where("id = ?", id.StringValue()).
Where("org_id = ?", orgID).
Exec(ctx)
@@ -82,8 +85,8 @@ func (store *store) Delete(ctx context.Context, orgID string, id valuer.UUID) er
}
func (store *store) List(ctx context.Context, orgID string, source savedviewtypes.Source, name string) ([]*savedviewtypes.SavedView, error) {
var views []*savedviewtypes.SavedView
q := store.sqlstore.BunDB().NewSelect().Model(&views).
var storables []*savedviewtypes.StorableSavedView
q := store.sqlstore.BunDB().NewSelect().Model(&storables).
Where("org_id = ?", orgID).
Where("name LIKE ?", "%"+name+"%")
if !source.IsZero() {
@@ -94,8 +97,11 @@ func (store *store) List(ctx context.Context, orgID string, source savedviewtype
return nil, errors.WrapInternalf(err, errors.CodeInternal, "error in getting saved views")
}
for _, view := range views {
views := make([]*savedviewtypes.SavedView, 0, len(storables))
for _, storable := range storables {
view := storable.ToSavedView()
normalizeSelectedFields(view)
views = append(views, view)
}
return views, nil
@@ -103,7 +109,7 @@ func (store *store) List(ctx context.Context, orgID string, source savedviewtype
// normalizeSelectedFields fixes up a scanned row's nil SelectedFields.
func normalizeSelectedFields(view *savedviewtypes.SavedView) {
if view.Data.Spec.SelectedFields == nil {
view.Data.Spec.SelectedFields = []telemetrytypes.TelemetryFieldKey{}
if view.Spec.SelectedFields == nil {
view.Spec.SelectedFields = []telemetrytypes.TelemetryFieldKey{}
}
}

View File

@@ -237,6 +237,7 @@ func NewSQLMigrationProviderFactories(
sqlmigration.NewAddDashboardTuplesFactory(sqlstore),
sqlmigration.NewRestructureSavedViewSpecFactory(sqlstore, sqlschema),
sqlmigration.NewAddSavedViewTuplesFactory(sqlstore),
sqlmigration.NewFixSavedViewSelectedFieldsFactory(sqlstore),
)
}

View File

@@ -16,6 +16,7 @@ import (
"github.com/SigNoz/signoz/pkg/sqlschema"
"github.com/SigNoz/signoz/pkg/sqlstore"
"github.com/SigNoz/signoz/pkg/types"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
type restructureSavedViewSpec struct {
@@ -44,11 +45,11 @@ type legacySavedViewCompositeQuery struct {
// legacySavedViewExtraData mirrors the frontend defined extraData JSON shape.
type legacySavedViewExtraData struct {
Color string `json:"color,omitempty"`
SelectColumns json.RawMessage `json:"selectColumns,omitempty"`
Format string `json:"format,omitempty"`
MaxLines int `json:"maxLines,omitempty"`
FontSize string `json:"fontSize,omitempty"`
Color string `json:"color,omitempty"`
SelectColumns []telemetrytypes.TelemetryFieldKey `json:"selectColumns,omitempty"`
Format string `json:"format,omitempty"`
MaxLines int `json:"maxLines,omitempty"`
FontSize string `json:"fontSize,omitempty"`
}
type savedViewDisplay struct {
@@ -59,11 +60,11 @@ type savedViewDisplay struct {
}
type savedViewSpec struct {
DisplayName string `json:"displayName"`
PanelType string `json:"panelType"`
Queries json.RawMessage `json:"queries"`
SelectedFields json.RawMessage `json:"selectedFields"`
Display savedViewDisplay `json:"display"`
DisplayName string `json:"displayName"`
PanelType string `json:"panelType"`
Queries json.RawMessage `json:"queries"`
SelectedFields []telemetrytypes.TelemetryFieldKey `json:"selectedFields"`
Display savedViewDisplay `json:"display"`
}
type savedViewData struct {
@@ -238,7 +239,9 @@ func (migration *restructureSavedViewSpec) Up(ctx context.Context, db *bun.DB) e
if old.ExtraData != "" {
// best-effort: malformed/older extraData shapes never fail the migration,
// they just leave selectedFields/display empty.
_ = json.Unmarshal([]byte(old.ExtraData), &extraData)
if err := json.Unmarshal([]byte(old.ExtraData), &extraData); err != nil {
migration.settings.Logger.WarnContext(ctx, "failed to unmarshal saved view extra data, continuing with empty selectedFields/display", slog.String("org_id", old.OrgID), slog.String("saved_view_id", old.ID), slog.Any("error", err))
}
}
dataJSON, err := json.Marshal(savedViewData{

View File

@@ -0,0 +1,185 @@
package sqlmigration
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"log/slog"
"github.com/uptrace/bun"
"github.com/uptrace/bun/migrate"
"github.com/SigNoz/signoz/pkg/factory"
"github.com/SigNoz/signoz/pkg/sqlstore"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/savedviewtypes"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
)
type fixSavedViewSelectedFields struct {
sqlstore sqlstore.SQLStore
settings factory.ProviderSettings
}
func NewFixSavedViewSelectedFieldsFactory(sqlstore sqlstore.SQLStore) factory.ProviderFactory[SQLMigration, Config] {
return factory.NewProviderFactory(factory.MustNewName("fix_saved_view_selected_fields"), func(ctx context.Context, ps factory.ProviderSettings, c Config) (SQLMigration, error) {
return &fixSavedViewSelectedFields{sqlstore: sqlstore, settings: ps}, nil
})
}
func (migration *fixSavedViewSelectedFields) Register(migrations *migrate.Migrations) error {
return migrations.Register(migration.Up, migration.Down)
}
// storableSavedViewData is the shape of the `saved_view` table this migration repairs.
type storableSavedViewData struct {
bun.BaseModel `bun:"table:saved_view"`
ID string `bun:"id,pk,type:text"`
Data string `bun:"data,type:text"`
}
// specFieldUnmarshalsCleanly reports whether value can be unmarshalled into
// the real type of the given savedviewtypes.SavedViewSpec JSON key.
func specFieldUnmarshalsCleanly(key string, value json.RawMessage) bool {
switch key {
case "displayName", "panelType":
var s string
return json.Unmarshal(value, &s) == nil
case "queries":
var q []qbtypes.QueryEnvelope
return json.Unmarshal(value, &q) == nil
case "selectedFields":
var f []telemetrytypes.TelemetryFieldKey
return json.Unmarshal(value, &f) == nil
case "display":
var d savedviewtypes.Display
return json.Unmarshal(value, &d) == nil
default:
return true
}
}
// specFieldZeroValueJSON is the JSON to substitute for a spec key that fails
// to unmarshal into its real type.
var specFieldZeroValueJSON = map[string]string{
"displayName": `""`,
"panelType": `""`,
"queries": `[]`,
"selectedFields": `[]`,
"display": `{}`,
}
// repairSavedViewData tries to make data unmarshal cleanly into
// savedviewtypes.SavedViewData by blanking, one key at a time, whichever
// top-level spec fields fail to unmarshal into their real type -- e.g. a
// selectedFields shape the 109 migration forwarded verbatim from a
// pre-telemetrytypes.TelemetryFieldKey install, or a queries shape that
// predates the current discriminated-union QueryEnvelope. Every other key is
// left byte-for-byte untouched. Returns ok=false if data/spec aren't even
// JSON objects, or the result still doesn't unmarshal cleanly afterward.
func repairSavedViewData(data string) (fixed string, blanked []string, ok bool) {
var raw map[string]json.RawMessage
if err := json.Unmarshal([]byte(data), &raw); err != nil {
return "", nil, false
}
var spec map[string]json.RawMessage
if err := json.Unmarshal(raw["spec"], &spec); err != nil {
return "", nil, false
}
for key, value := range spec {
if specFieldUnmarshalsCleanly(key, value) {
continue
}
zero, known := specFieldZeroValueJSON[key]
if !known {
continue
}
spec[key] = json.RawMessage(zero)
blanked = append(blanked, key)
}
fixedSpec, err := json.Marshal(spec)
if err != nil {
return "", nil, false
}
raw["spec"] = fixedSpec
fixedData, err := json.Marshal(raw)
if err != nil {
return "", nil, false
}
// verify the fix actually round-trips through the real type before writing it.
if err := json.Unmarshal(fixedData, new(savedviewtypes.SavedViewData)); err != nil {
return "", nil, false
}
return string(fixedData), blanked, true
}
// placeholderSavedViewData is substituted whole when a row can't be repaired
// field-by-field (data/spec aren't JSON objects at all, or repair still
// doesn't unmarshal cleanly). It must itself always unmarshal cleanly, since
// every Get/List reads saved_view.data straight into savedviewtypes.SavedViewData
// -- leaving genuinely-unrepairable data in place would 500 on every future read.
func placeholderSavedViewData(id string) string {
data, err := json.Marshal(savedviewtypes.SavedViewData{
SchemaVersion: savedviewtypes.SavedViewSchemaVersion.StringValue(),
Spec: savedviewtypes.SavedViewSpec{
DisplayName: fmt.Sprintf("corrupted saved view %s", id),
PanelType: savedviewtypes.PanelTypeTable,
},
})
if err != nil {
// marshalling a static, well-formed literal cannot fail.
panic(err)
}
return string(data)
}
func (migration *fixSavedViewSelectedFields) Up(ctx context.Context, db *bun.DB) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
var rows []*storableSavedViewData
if err := tx.NewSelect().Model(&rows).Scan(ctx); err != nil && err != sql.ErrNoRows {
return err
}
var repaired, replaced int
for _, row := range rows {
// already scans cleanly as-is -- nothing to repair.
if err := json.Unmarshal([]byte(row.Data), new(savedviewtypes.SavedViewData)); err == nil {
continue
}
fixedData, blanked, ok := repairSavedViewData(row.Data)
if !ok {
fixedData = placeholderSavedViewData(row.ID)
replaced++
migration.settings.Logger.WarnContext(ctx, "saved view data could not be repaired field-by-field, replacing with a placeholder view", slog.String("saved_view_id", row.ID))
} else {
repaired++
migration.settings.Logger.WarnContext(ctx, "repaired saved view data by blanking fields that failed to unmarshal", slog.String("saved_view_id", row.ID), slog.Any("fields_blanked", blanked))
}
if _, err := tx.NewUpdate().Model((*storableSavedViewData)(nil)).Set("data = ?", fixedData).Where("id = ?", row.ID).Exec(ctx); err != nil {
return err
}
}
migration.settings.Logger.InfoContext(ctx, "checked saved views for unreadable data", slog.Int("total", len(rows)), slog.Int("repaired", repaired), slog.Int("replaced", replaced))
return tx.Commit()
}
func (migration *fixSavedViewSelectedFields) Down(context.Context, *bun.DB) error {
return nil
}

View File

@@ -27,28 +27,75 @@ var (
SourceMeter = Source{valuer.NewString("meter")}
)
// SavedView is the domain/wire shape -- schemaVersion and spec are promoted
// to the top level, matching dashboardtypes.DashboardV2 and ruletypes'
// v2alpha1 rules. StorableSavedView is the distinct bun-mapped row shape;
// the two diverge because bun maps Data to a single opaque `data` column,
// which is incompatible with promoting its fields to the top level for JSON.
type SavedView struct {
types.Identifiable
types.TimeAuditable
types.UserAuditable
OrgID string `json:"-"`
Name string `json:"name"`
Source Source `json:"source"`
SchemaVersion SchemaVersion `json:"schemaVersion" required:"true"`
Spec SavedViewSpec `json:"spec" required:"true"`
}
// StorableSavedView is the row shape bun maps to the saved_view table.
type StorableSavedView struct {
bun.BaseModel `bun:"table:saved_view"`
types.Identifiable
types.TimeAuditable
types.UserAuditable
OrgID string `json:"-" bun:"org_id,notnull"`
Name string `json:"name" bun:"name,type:text,notnull"`
Source Source `json:"source" bun:"source,type:text,notnull"`
Data SavedViewData `json:"data" bun:"data,type:text,notnull"`
OrgID string `bun:"org_id,notnull"`
Name string `bun:"name,type:text,notnull"`
Source Source `bun:"source,type:text,notnull"`
Data SavedViewData `bun:"data,type:text,notnull"`
}
func (s *StorableSavedView) ToSavedView() *SavedView {
return &SavedView{
Identifiable: s.Identifiable,
TimeAuditable: s.TimeAuditable,
UserAuditable: s.UserAuditable,
OrgID: s.OrgID,
Name: s.Name,
Source: s.Source,
SchemaVersion: SchemaVersion{valuer.NewString(s.Data.SchemaVersion)},
Spec: s.Data.Spec,
}
}
func NewStorableSavedView(view *SavedView) *StorableSavedView {
return &StorableSavedView{
Identifiable: view.Identifiable,
TimeAuditable: view.TimeAuditable,
UserAuditable: view.UserAuditable,
OrgID: view.OrgID,
Name: view.Name,
Source: view.Source,
Data: SavedViewData{
SchemaVersion: view.SchemaVersion.StringValue(),
Spec: view.Spec,
},
}
}
type PostableSavedView struct {
Name string `json:"name"`
GenerateName bool `json:"generateName"`
Source Source `json:"source" required:"true"`
Data SavedViewData `json:"data" required:"true"`
Name string `json:"name"`
GenerateName bool `json:"generateName"`
Source Source `json:"source" required:"true"`
SchemaVersion SchemaVersion `json:"schemaVersion" required:"true"`
Spec SavedViewSpec `json:"spec" required:"true"`
}
type UpdatableSavedView struct {
Source Source `json:"source" required:"true"`
Data SavedViewData `json:"data" required:"true"`
Source Source `json:"source" required:"true"`
SchemaVersion SchemaVersion `json:"schemaVersion" required:"true"`
Spec SavedViewSpec `json:"spec" required:"true"`
}
type ListSavedViewsParams struct {
@@ -83,7 +130,7 @@ func (postable PostableSavedView) ToSavedView(orgID string, createdBy string) *S
name := postable.Name
if postable.GenerateName {
name = generateSavedViewName(postable.Data.Spec.DisplayName)
name = generateSavedViewName(postable.Spec.DisplayName)
}
return &SavedView{
@@ -93,7 +140,8 @@ func (postable PostableSavedView) ToSavedView(orgID string, createdBy string) *S
OrgID: orgID,
Name: name,
Source: postable.Source,
Data: postable.Data,
SchemaVersion: postable.SchemaVersion,
Spec: postable.Spec,
}
}
@@ -106,7 +154,8 @@ func (updatable UpdatableSavedView) ToSavedView(id valuer.UUID, orgID string, up
UserAuditable: types.UserAuditable{UpdatedBy: updatedBy},
OrgID: orgID,
Source: updatable.Source,
Data: updatable.Data,
SchemaVersion: updatable.SchemaVersion,
Spec: updatable.Spec,
}
}
@@ -117,8 +166,11 @@ func (p *PostableSavedView) Validate() error {
if err := p.Source.Validate(); err != nil {
return err
}
if err := p.SchemaVersion.Validate(); err != nil {
return err
}
return p.Data.Validate()
return p.Spec.Validate()
}
func (p *PostableSavedView) validateName() error {
@@ -135,8 +187,11 @@ func (u *UpdatableSavedView) Validate() error {
if err := u.Source.Validate(); err != nil {
return err
}
if err := u.SchemaVersion.Validate(); err != nil {
return err
}
return u.Data.Validate()
return u.Spec.Validate()
}
func (p *ListSavedViewsParams) Validate() error {

View File

@@ -11,22 +11,18 @@ import (
func validPostableSavedView() PostableSavedView {
return PostableSavedView{
Name: "my-view",
Source: SourceLogs,
Data: SavedViewData{
SchemaVersion: SavedViewSchemaVersion,
Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()},
},
Name: "my-view",
Source: SourceLogs,
SchemaVersion: SavedViewSchemaVersion,
Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()},
}
}
func validUpdatableSavedView() UpdatableSavedView {
return UpdatableSavedView{
Source: SourceLogs,
Data: SavedViewData{
SchemaVersion: SavedViewSchemaVersion,
Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()},
},
Source: SourceLogs,
SchemaVersion: SavedViewSchemaVersion,
Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()},
}
}
@@ -69,7 +65,7 @@ func TestPostableSavedViewValidate(t *testing.T) {
t.Run("invalid saved view data is rejected", func(t *testing.T) {
view := validPostableSavedView()
view.Data.SchemaVersion = "v1"
view.SchemaVersion = SchemaVersion{valuer.NewString("v1")}
assert.Error(t, view.Validate())
})
@@ -100,7 +96,7 @@ func TestPostableSavedViewValidate(t *testing.T) {
t.Run("empty displayName is rejected", func(t *testing.T) {
view := validPostableSavedView()
view.Data.Spec.DisplayName = ""
view.Spec.DisplayName = ""
assert.ErrorContains(t, view.Validate(), "displayName is required")
})
}
@@ -119,7 +115,7 @@ func TestUpdatableSavedViewValidate(t *testing.T) {
t.Run("empty displayName is rejected", func(t *testing.T) {
view := validUpdatableSavedView()
view.Data.Spec.DisplayName = ""
view.Spec.DisplayName = ""
assert.ErrorContains(t, view.Validate(), "displayName is required")
})
}
@@ -153,7 +149,8 @@ func TestNewSavedView(t *testing.T) {
assert.Equal(t, "creator@signoz.io", savedView.UpdatedBy)
assert.Equal(t, view.Name, savedView.Name)
assert.Equal(t, view.Source, savedView.Source)
assert.Equal(t, view.Data, savedView.Data)
assert.Equal(t, view.SchemaVersion, savedView.SchemaVersion)
assert.Equal(t, view.Spec, savedView.Spec)
assert.False(t, savedView.CreatedAt.IsZero())
assert.Equal(t, savedView.CreatedAt, savedView.UpdatedAt)
}
@@ -163,14 +160,14 @@ func TestNewSavedView_GeneratesNameWhenEmpty(t *testing.T) {
view := validPostableSavedView()
view.Name = ""
view.GenerateName = true
view.Data.Spec.DisplayName = "My View!"
view.Spec.DisplayName = "My View!"
savedView := view.ToSavedView(orgID, "creator@signoz.io")
assert.NotEmpty(t, savedView.Name)
assert.Empty(t, validation.IsDNS1123Label(savedView.Name), "generated name must be a valid DNS-1123 label")
assert.True(t, strings.HasPrefix(savedView.Name, "my-view-"))
assert.Equal(t, "My View!", savedView.Data.Spec.DisplayName)
assert.Equal(t, "My View!", savedView.Spec.DisplayName)
}
func TestGenerateSavedViewName(t *testing.T) {

View File

@@ -28,7 +28,7 @@ func (t *StoreTest) Store() savedviewtypes.Store { return t.store }
func (t *StoreTest) Mock() sqlmock.Sqlmock { return t.mock }
func savedViewRow(view *savedviewtypes.SavedView) []driver.Value {
data, _ := json.Marshal(view.Data)
data, _ := json.Marshal(savedviewtypes.NewStorableSavedView(view).Data)
return []driver.Value{
view.ID.StringValue(),
view.CreatedAt,

View File

@@ -8,7 +8,7 @@ import (
)
// SavedViewSchemaVersion is the only schemaVersion currently.
const SavedViewSchemaVersion = "v2"
var SavedViewSchemaVersion = SchemaVersion{valuer.NewString("v2")}
var (
PanelTypeValue = PanelType{valuer.NewString("value")}
@@ -30,9 +30,9 @@ type Display struct {
type SavedViewSpec struct {
DisplayName string `json:"displayName" required:"true"`
PanelType PanelType `json:"panelType" required:"true"`
Queries []qbtypes.QueryEnvelope `json:"queries" required:"true" nullable:"false"`
SelectedFields []telemetrytypes.TelemetryFieldKey `json:"selectedFields" required:"true" nullable:"false"`
Display Display `json:"display" required:"true"`
Queries []qbtypes.QueryEnvelope `json:"queries" required:"true" nullable:"false" minItems:"1"`
SelectedFields []telemetrytypes.TelemetryFieldKey `json:"selectedFields" nullable:"false"`
Display Display `json:"display"`
}
// SavedViewData is what's persisted as saved view data.
@@ -41,6 +41,11 @@ type SavedViewData struct {
Spec SavedViewSpec `json:"spec" required:"true"`
}
// SchemaVersion has v2 as the only value currently.
type SchemaVersion struct {
valuer.String
}
// PanelType is the explore-page panel a saved view renders as.
type PanelType struct {
valuer.String
@@ -65,6 +70,17 @@ func (p PanelType) Validate() error {
}
}
func (SchemaVersion) Enum() []any {
return []any{SavedViewSchemaVersion}
}
func (s SchemaVersion) Validate() error {
if s != SavedViewSchemaVersion {
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "schemaVersion must be %q, got %q", SavedViewSchemaVersion.StringValue(), s.StringValue())
}
return nil
}
func (s *SavedViewSpec) Validate() error {
if s.DisplayName == "" {
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "displayName is required")
@@ -75,11 +91,3 @@ func (s *SavedViewSpec) Validate() error {
return (&qbtypes.CompositeQuery{Queries: s.Queries}).Validate()
}
func (d *SavedViewData) Validate() error {
if d.SchemaVersion != SavedViewSchemaVersion {
return errors.NewInvalidInputf(ErrCodeSavedViewInvalidInput, "schemaVersion must be %q, got %q", SavedViewSchemaVersion, d.SchemaVersion)
}
return d.Spec.Validate()
}

View File

@@ -1,12 +1,15 @@
package savedviewtypes
import (
"encoding/json"
"testing"
qbtypes "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
"github.com/SigNoz/signoz/pkg/types/telemetrytypes"
"github.com/SigNoz/signoz/pkg/valuer"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func validQueries() []qbtypes.QueryEnvelope {
@@ -75,7 +78,7 @@ func TestSavedViewSpecValidate(t *testing.T) {
expectError: true,
},
{
name: "selected fields and display are not required",
name: "selectedFields and display populated is still valid",
spec: SavedViewSpec{
DisplayName: "My View",
PanelType: PanelTypeTable,
@@ -85,6 +88,36 @@ func TestSavedViewSpecValidate(t *testing.T) {
},
expectError: false,
},
{
name: "nil selectedFields is valid -- neither field is actually required",
spec: SavedViewSpec{
DisplayName: "My View",
PanelType: PanelTypeTable,
Queries: validQueries(),
SelectedFields: nil,
},
expectError: false,
},
{
name: "empty (non-nil) selectedFields is valid",
spec: SavedViewSpec{
DisplayName: "My View",
PanelType: PanelTypeTable,
Queries: validQueries(),
SelectedFields: []telemetrytypes.TelemetryFieldKey{},
},
expectError: false,
},
{
name: "zero-value display is valid",
spec: SavedViewSpec{
DisplayName: "My View",
PanelType: PanelTypeTable,
Queries: validQueries(),
Display: Display{},
},
expectError: false,
},
}
for _, c := range cases {
@@ -99,37 +132,44 @@ func TestSavedViewSpecValidate(t *testing.T) {
}
}
func TestSavedViewDataValidate(t *testing.T) {
func TestSavedViewSpecJSONUnmarshal_OptionalFields(t *testing.T) {
base := `"displayName":"My View","panelType":"table","queries":[{"type":"builder_query","spec":{"signal":"logs","aggregations":[{"expression":"count()"}]}}]`
cases := []struct {
name string
data SavedViewData
expectError bool
name string
json string
}{
{
name: "valid data",
data: SavedViewData{SchemaVersion: SavedViewSchemaVersion, Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()}},
expectError: false,
},
{
name: "wrong schema version is rejected",
data: SavedViewData{SchemaVersion: "v1", Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()}},
expectError: true,
},
{
name: "empty schema version is rejected",
data: SavedViewData{Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph, Queries: validQueries()}},
expectError: true,
},
{
name: "invalid spec is rejected",
data: SavedViewData{SchemaVersion: SavedViewSchemaVersion, Spec: SavedViewSpec{DisplayName: "My View", PanelType: PanelTypeGraph}},
expectError: true,
},
{name: "selectedFields and display omitted entirely", json: `{` + base + `}`},
{name: "selectedFields and display explicitly null", json: `{` + base + `,"selectedFields":null,"display":null}`},
{name: "selectedFields empty array, display empty object", json: `{` + base + `,"selectedFields":[],"display":{}}`},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
err := c.data.Validate()
var spec SavedViewSpec
err := json.Unmarshal([]byte(c.json), &spec)
require.NoError(t, err)
assert.NoError(t, spec.Validate())
assert.Empty(t, spec.SelectedFields)
assert.Equal(t, Display{}, spec.Display)
})
}
}
func TestSchemaVersionValidate(t *testing.T) {
cases := []struct {
name string
schemaVersion SchemaVersion
expectError bool
}{
{name: "valid schema version", schemaVersion: SavedViewSchemaVersion, expectError: false},
{name: "wrong schema version is rejected", schemaVersion: SchemaVersion{valuer.NewString("v1")}, expectError: true},
{name: "empty schema version is rejected", schemaVersion: SchemaVersion{}, expectError: true},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
err := c.schemaVersion.Validate()
if c.expectError {
assert.Error(t, err)
} else {