mirror of
https://github.com/SigNoz/signoz.git
synced 2026-02-25 01:33:20 +00:00
Compare commits
4 Commits
feat/skill
...
replace-pr
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c715b881bd | ||
|
|
36e36d3bea | ||
|
|
892cd04475 | ||
|
|
a2aa7d7342 |
@@ -1,97 +0,0 @@
|
||||
---
|
||||
globs: **/*.store.ts
|
||||
alwaysApply: false
|
||||
---
|
||||
# State Management: React Query, nuqs, Zustand
|
||||
|
||||
Use the following stack. Do **not** introduce or recommend Redux or React Context for shared/global state.
|
||||
|
||||
## Server state → React Query
|
||||
|
||||
- **Use for:** API responses, time-series data, caching, background refetch, retries, stale/refresh.
|
||||
- **Do not use Redux/Context** to store or mirror data that comes from React Query (e.g. do not dispatch API results into Redux).
|
||||
- Prefer generated React Query hooks from `frontend/src/api/generated` when available.
|
||||
- Keep server state in React Query; expose it via hooks that return the query result (and optionally memoized derived values). Do not duplicate it in Redux or Context.
|
||||
|
||||
```tsx
|
||||
// ✅ GOOD: single source of truth from React Query
|
||||
export function useAppStateHook() {
|
||||
const { data, isError } = useQuery(...)
|
||||
const memoizedConfigs = useMemo(() => ({ ... }), [data?.configs])
|
||||
return { configs: memoizedConfigs, isError, ... }
|
||||
}
|
||||
|
||||
// ❌ BAD: copying React Query result into Redux
|
||||
dispatch({ type: UPDATE_LATEST_VERSION, payload: queryResponse.data })
|
||||
```
|
||||
|
||||
## URL state → nuqs
|
||||
|
||||
- **Use for:** shareable state, filters, time range, selected values, pagination, view state that belongs in the URL.
|
||||
- **Do not use Redux/Context** for state that should be shareable or reflected in the URL.
|
||||
- Use [nuqs](https://nuqs.dev/docs/basic-usage) for typed, type-safe URL search params. Avoid ad-hoc `useSearchParams` encoding/decoding.
|
||||
- Keep URL payload small; respect browser URL length limits (e.g. Chrome ~2k chars). Do not put large datasets or sensitive data in query params.
|
||||
|
||||
```tsx
|
||||
// ✅ GOOD: nuqs for filters / time range / selection
|
||||
const [timeRange, setTimeRange] = useQueryState('timeRange', parseAsString.withDefault('1h'))
|
||||
const [page, setPage] = useQueryState('page', parseAsInteger.withDefault(1))
|
||||
|
||||
// ❌ BAD: Redux/Context for shareable or URL-synced state
|
||||
const { timeRange } = useContext(SomeContext)
|
||||
```
|
||||
|
||||
## Client state → Zustand
|
||||
|
||||
- **Use for:** global/client state, cross-component state, feature flags, complex or large client objects (e.g. dashboard state, query builder state).
|
||||
- **Do not use Redux or React Context** for global or feature-level client state.
|
||||
- Prefer small, domain-scoped stores (e.g. DashboardStore, QueryBuilderStore).
|
||||
|
||||
### Zustand best practices (align with eslint-plugin-zustand-rules)
|
||||
|
||||
- **One store per module.** Do not define multiple `create()` calls in the same file; use one store per module (or compose slices into one store).
|
||||
- **Always use selectors.** Call the store hook with a selector so only the used slice triggers re-renders. Never use `useStore()` with no selector.
|
||||
|
||||
```tsx
|
||||
// ✅ GOOD: selector — re-renders only when isDashboardLocked changes
|
||||
const isLocked = useDashboardStore(state => state.isDashboardLocked)
|
||||
|
||||
// ❌ BAD: no selector — re-renders on any store change
|
||||
const state = useDashboardStore()
|
||||
```
|
||||
|
||||
- **Never mutate state directly.** Update only via `set` or `setState` (or `getState()` + `set` for reads). No `state.foo = x` or `state.bears += 1` inside actions.
|
||||
|
||||
```tsx
|
||||
// ✅ GOOD: use set
|
||||
increment: () => set(state => ({ bears: state.bears + 1 }))
|
||||
|
||||
// ❌ BAD: direct mutation
|
||||
increment: () => { state.bears += 1 }
|
||||
```
|
||||
|
||||
- **State properties before actions.** In the store object, list all state fields first, then action functions.
|
||||
- **Split into slices when state is large.** If a store has many top-level properties (e.g. more than 5–10), split into slice factories and combine with one `create()`.
|
||||
|
||||
```tsx
|
||||
// ✅ GOOD: slices for large state
|
||||
const createBearSlice = set => ({ bears: 0, addBear: () => set(s => ({ bears: s.bears + 1 })) })
|
||||
const createFishSlice = set => ({ fish: 0, addFish: () => set(s => ({ fish: s.fish + 1 })) })
|
||||
const useStore = create(set => ({ ...createBearSlice(set), ...createFishSlice(set) }))
|
||||
```
|
||||
|
||||
- **In projects using Zustand:** add `eslint-plugin-zustand-rules` and extend `plugin:zustand-rules/recommended` to enforce these rules automatically.
|
||||
|
||||
## Local state → React state only
|
||||
|
||||
- **Use useState/useReducer** for: component-local UI state, form inputs, toggles, hover state, data that never leaves the component.
|
||||
- Do not use Zustand, Redux, or Context for state that is purely local to one component or a small subtree.
|
||||
|
||||
## Summary
|
||||
|
||||
| State type | Use | Avoid |
|
||||
|-------------------|------------------|--------------------|
|
||||
| Server / API | React Query | Redux, Context |
|
||||
| URL / shareable | nuqs | Redux, Context |
|
||||
| Global client | Zustand | Redux, Context |
|
||||
| Local UI | useState/useReducer | Zustand, Redux, Context |
|
||||
@@ -1,150 +0,0 @@
|
||||
---
|
||||
name: migrate-state-management
|
||||
description: Migrate Redux or React Context to the correct state option (React Query for server state, nuqs for URL/shareable state, Zustand for global client state). Use when refactoring away from Redux/Context, moving state to the right store, or when the user asks to migrate state management.
|
||||
---
|
||||
|
||||
# Migrate State: Redux/Context → React Query, nuqs, Zustand
|
||||
|
||||
Do **not** introduce or recommend Redux or React Context. Migrate existing usage to the stack below.
|
||||
|
||||
## 1. Classify the state
|
||||
|
||||
Before changing code, classify what the state represents:
|
||||
|
||||
| If the state is… | Migrate to | Do not use |
|
||||
|------------------|------------|------------|
|
||||
| From API / server (versions, configs, fetched lists, time-series) | **React Query** | Redux, Context |
|
||||
| Shareable via URL (filters, time range, page, selected ids) | **nuqs** | Redux, Context |
|
||||
| Global/client UI (dashboard lock, query builder, feature flags, large client objects) | **Zustand** | Redux, Context |
|
||||
| Local to one component (inputs, toggles, hover) | **useState / useReducer** | Zustand, Redux, Context |
|
||||
|
||||
If one slice mixes concerns (e.g. Redux has both API data and pagination), split: API → React Query, pagination → nuqs, rest → Zustand or local state.
|
||||
|
||||
## 2. Migrate to React Query (server state)
|
||||
|
||||
**When:** State comes from or mirrors an API response (e.g. `currentVersion`, `latestVersion`, `configs`, lists).
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Find where the data is fetched (existing `useQuery`/API call) and where it is dispatched or set in Context/Redux.
|
||||
2. Remove the dispatch/set that writes API results into Redux/Context.
|
||||
3. Expose a single hook that uses the query and returns the same shape consumers expect (use `useMemo` for derived objects like `configs` to avoid unnecessary re-renders).
|
||||
4. Replace Redux/Context consumption with the new hook. Prefer generated React Query hooks from `frontend/src/api/generated` when available.
|
||||
5. Configure cache/refetch (e.g. `refetchOnMount: false`, `staleTime`) so behavior matches previous “single source” expectations.
|
||||
|
||||
**Before (Redux mirroring React Query):**
|
||||
|
||||
```tsx
|
||||
if (getUserLatestVersionResponse.isFetched && getUserLatestVersionResponse.isSuccess && getUserLatestVersionResponse.data?.payload) {
|
||||
dispatch({ type: UPDATE_LATEST_VERSION, payload: { latestVersion: getUserLatestVersionResponse.data.payload.tag_name } })
|
||||
}
|
||||
```
|
||||
|
||||
**After (single source in React Query):**
|
||||
|
||||
```tsx
|
||||
export function useAppStateHook() {
|
||||
const { data, isError } = useQuery(...)
|
||||
const memoizedConfigs = useMemo(() => ({ ... }), [data?.configs])
|
||||
return {
|
||||
latestVersion: data?.payload?.tag_name,
|
||||
configs: memoizedConfigs,
|
||||
isError,
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Consumers use `useAppStateHook()` instead of `useSelector` or Context. Do not copy React Query result into Redux or Context.
|
||||
|
||||
## 3. Migrate to nuqs (URL / shareable state)
|
||||
|
||||
**When:** State should be in the URL: filters, time range, pagination, selected values, view state. Keep payload small (e.g. Chrome ~2k chars); no large datasets or sensitive data.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Identify which Redux/Context fields are shareable or already reflected in the URL (e.g. `currentPage`, `timeRange`, `selectedFilter`).
|
||||
2. Add nuqs (or use existing): `useQueryState('param', parseAsString.withDefault('…'))` (or `parseAsInteger`, etc.).
|
||||
3. Replace reads/writes of those fields with nuqs hooks. Use typed parsers; avoid ad-hoc `useSearchParams` encoding/decoding.
|
||||
4. Remove the same fields from Redux/Context and their reducers/providers.
|
||||
|
||||
**Before (Context/Redux):**
|
||||
|
||||
```tsx
|
||||
const { timeRange } = useContext(SomeContext)
|
||||
const [page, setPage] = useDispatch(...)
|
||||
```
|
||||
|
||||
**After (nuqs):**
|
||||
|
||||
```tsx
|
||||
const [timeRange, setTimeRange] = useQueryState('timeRange', parseAsString.withDefault('1h'))
|
||||
const [page, setPage] = useQueryState('page', parseAsInteger.withDefault(1))
|
||||
```
|
||||
|
||||
## 4. Migrate to Zustand (global client state)
|
||||
|
||||
**When:** State is global or cross-component client state: feature flags, dashboard state, query builder state, complex/large client objects (e.g. up to ~1.5–2MB). Not for server cache or local-only UI.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Create one store per domain (e.g. `DashboardStore`, `QueryBuilderStore`). One `create()` per module; for large state use slice factories and combine.
|
||||
2. Put state properties first, then actions. Use `set` (or `setState` / `getState()` + `set`) for updates; never mutate state directly.
|
||||
3. Replace Context/Redux consumption with the store hook **and a selector** so only the used slice triggers re-renders.
|
||||
4. Remove the old Context provider / Redux slice and related dispatches.
|
||||
|
||||
**Selector (required):**
|
||||
|
||||
```tsx
|
||||
const isLocked = useDashboardStore(state => state.isDashboardLocked)
|
||||
```
|
||||
|
||||
Never use `useStore()` with no selector. Never do `state.foo = x` inside actions; use `set(state => ({ ... }))`.
|
||||
|
||||
**Before (Context/Redux):**
|
||||
|
||||
```tsx
|
||||
const { isDashboardLocked, setLocked } = useContext(DashboardContext)
|
||||
```
|
||||
|
||||
**After (Zustand):**
|
||||
|
||||
```tsx
|
||||
const isLocked = useDashboardStore(state => state.isDashboardLocked)
|
||||
const setLocked = useDashboardStore(state => state.setLocked)
|
||||
```
|
||||
|
||||
For large stores (many top-level fields), split into slices and combine:
|
||||
|
||||
```tsx
|
||||
const createBearSlice = set => ({ bears: 0, addBear: () => set(s => ({ bears: s.bears + 1 })) })
|
||||
const useStore = create(set => ({ ...createBearSlice(set), ...createFishSlice(set) }))
|
||||
```
|
||||
|
||||
Add `eslint-plugin-zustand-rules` with `plugin:zustand-rules/recommended` to enforce selectors and no direct mutation.
|
||||
|
||||
## 5. Migrate to local state (useState / useReducer)
|
||||
|
||||
**When:** State is used only inside one component or a small subtree (form inputs, toggles, hover, panel selection). No URL sync, no cross-feature sharing.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Move the state into the component that owns it (or the smallest common parent).
|
||||
2. Use `useState` or `useReducer` (useReducer when multiple related fields change together).
|
||||
3. Remove from Redux/Context and any provider/slice.
|
||||
|
||||
Do not use Zustand, Redux, or Context for purely local UI state.
|
||||
|
||||
## 6. Migration checklist
|
||||
|
||||
- [ ] Classify each piece of state (server / URL / global client / local).
|
||||
- [ ] Server state: move to React Query; expose via hook; remove Redux/Context mirroring.
|
||||
- [ ] URL state: move to nuqs; remove from Redux/Context; keep URL payload small.
|
||||
- [ ] Global client state: move to Zustand with selectors and immutable updates; one store per domain.
|
||||
- [ ] Local state: move to useState/useReducer in the owning component.
|
||||
- [ ] Remove old Redux slices / Context providers and all dispatches/consumers for migrated state.
|
||||
- [ ] Do not duplicate the same data in multiple places (e.g. React Query + Redux).
|
||||
|
||||
## Additional resources
|
||||
|
||||
- Project rule: [.cursor/rules/state-management.mdc](../../rules/state-management.mdc)
|
||||
- Detailed patterns and rationale: [reference.md](reference.md)
|
||||
@@ -1,50 +0,0 @@
|
||||
# State migration reference
|
||||
|
||||
## Why migrate
|
||||
|
||||
- **Context:** Re-renders all consumers on any change; no granular subscriptions; becomes brittle at scale.
|
||||
- **Redux:** Heavy boilerplate (actions, reducers, selectors, Provider); slower onboarding; often used to mirror React Query or URL state.
|
||||
- **Goal:** Fewer mechanisms, domain isolation, granular subscriptions, single source of truth per state type.
|
||||
|
||||
## React Query migration (server state)
|
||||
|
||||
Typical anti-pattern: API is called via React Query, then result is dispatched to Redux. Flow becomes: Component → useQueries → API → dispatch → Reducer → Redux state → useSelector.
|
||||
|
||||
Correct flow: Component → useQuery (or custom hook wrapping it) → same component reads from hook. No Redux/Context in between.
|
||||
|
||||
- Prefer generated hooks from `frontend/src/api/generated`.
|
||||
- For “app state” that is just API data (versions, configs), one hook that returns `{ ...data, configs: useMemo(...) }` is enough. No selectors needed for plain data; useMemo only where the value is used as dependency (e.g. in useState).
|
||||
- Set `staleTime` / `refetchOnMount` etc. so refetch behavior matches previous expectations.
|
||||
|
||||
## nuqs migration (URL state)
|
||||
|
||||
Redux/Context often hold pagination, filters, time range, selected values that are shareable. Those belong in the URL.
|
||||
|
||||
- Use [nuqs](https://nuqs.dev/docs/basic-usage) for typed search params. Avoid ad-hoc `useSearchParams` + manual encoding.
|
||||
- Browser limits: Chrome ~2k chars practical; keep payload small; no large datasets or secrets in query params.
|
||||
- If the app uses TanStack Router, search params can be handled there; otherwise nuqs is the standard.
|
||||
|
||||
## Zustand migration (client state)
|
||||
|
||||
- One store per domain (e.g. DashboardStore, QueryBuilderStore). Multiple `create()` in one file is disallowed; use one store or composed slices.
|
||||
- Always use a selector: `useStore(s => s.field)` so only that field drives re-renders.
|
||||
- Never mutate: update only via `set(state => ({ ... }))` or `setState` / `getState()` + `set`.
|
||||
- State properties first, then actions. For 5–10+ top-level fields, split into slice factories and combine with one `create()`.
|
||||
- Large client objects: Zustand is for “large” in the ~1.5–2MB range; above that, optimize at API/store design.
|
||||
- Testing: no Provider; stores are plain functions; easy to reset and mock.
|
||||
|
||||
## What not to use
|
||||
|
||||
- **Redux / Context** for new or migrated shared/global state.
|
||||
- **Redux / Context** to store or mirror React Query results.
|
||||
- **Redux / Context** for state that should live in the URL (use nuqs).
|
||||
- **Zustand / Redux / Context** for component-local UI (use useState/useReducer).
|
||||
|
||||
## Summary table
|
||||
|
||||
| State type | Use | Avoid |
|
||||
|-------------|--------------------|-----------------|
|
||||
| Server/API | React Query | Redux, Context |
|
||||
| URL/shareable | nuqs | Redux, Context |
|
||||
| Global client | Zustand | Redux, Context |
|
||||
| Local UI | useState/useReducer | Zustand, Redux, Context |
|
||||
@@ -87,7 +87,7 @@ func (m *module) ListPromotedAndIndexedPaths(ctx context.Context) ([]promotetype
|
||||
}
|
||||
|
||||
func (m *module) listPromotedPaths(ctx context.Context) ([]string, error) {
|
||||
paths, err := m.metadataStore.ListPromotedPaths(ctx)
|
||||
paths, err := m.metadataStore.GetPromotedPaths(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -142,7 +142,7 @@ func (m *module) PromoteAndIndexPaths(
|
||||
pathsStr = append(pathsStr, path.Path)
|
||||
}
|
||||
|
||||
existingPromotedPaths, err := m.metadataStore.ListPromotedPaths(ctx, pathsStr...)
|
||||
existingPromotedPaths, err := m.metadataStore.GetPromotedPaths(ctx, pathsStr...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"github.com/ClickHouse/clickhouse-go/v2/lib/chcol"
|
||||
schemamigrator "github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator"
|
||||
"github.com/SigNoz/signoz-otel-collector/constants"
|
||||
"github.com/SigNoz/signoz-otel-collector/utils"
|
||||
"github.com/SigNoz/signoz/pkg/errors"
|
||||
"github.com/SigNoz/signoz/pkg/querybuilder"
|
||||
"github.com/SigNoz/signoz/pkg/telemetrylogs"
|
||||
@@ -113,7 +112,7 @@ func (t *telemetryMetaStore) buildBodyJSONPaths(ctx context.Context,
|
||||
|
||||
for _, fieldKey := range fieldKeys {
|
||||
promotedKey := strings.Split(fieldKey.Name, telemetrytypes.ArraySep)[0]
|
||||
fieldKey.Materialized = promoted.Contains(promotedKey)
|
||||
fieldKey.Materialized = promoted[promotedKey]
|
||||
fieldKey.Indexes = indexes[fieldKey.Name]
|
||||
}
|
||||
|
||||
@@ -295,33 +294,6 @@ func (t *telemetryMetaStore) ListLogsJSONIndexes(ctx context.Context, filters ..
|
||||
return indexes, nil
|
||||
}
|
||||
|
||||
func (t *telemetryMetaStore) ListPromotedPaths(ctx context.Context, paths ...string) (map[string]struct{}, error) {
|
||||
sb := sqlbuilder.Select("path").From(fmt.Sprintf("%s.%s", DBName, PromotedPathsTableName))
|
||||
pathConditions := []string{}
|
||||
for _, path := range paths {
|
||||
pathConditions = append(pathConditions, sb.Equal("path", path))
|
||||
}
|
||||
sb.Where(sb.Or(pathConditions...))
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
|
||||
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, errors.WrapInternalf(err, CodeFailLoadPromotedPaths, "failed to load promoted paths")
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
next := make(map[string]struct{})
|
||||
for rows.Next() {
|
||||
var path string
|
||||
if err := rows.Scan(&path); err != nil {
|
||||
return nil, errors.WrapInternalf(err, CodeFailLoadPromotedPaths, "failed to scan promoted path")
|
||||
}
|
||||
next[path] = struct{}{}
|
||||
}
|
||||
|
||||
return next, nil
|
||||
}
|
||||
|
||||
// TODO(Piyush): Remove this if not used in future
|
||||
func (t *telemetryMetaStore) ListJSONValues(ctx context.Context, path string, limit int) (*telemetrytypes.TelemetryFieldValues, bool, error) {
|
||||
path = CleanPathPrefixes(path)
|
||||
@@ -484,11 +456,12 @@ func derefValue(v any) any {
|
||||
return val.Interface()
|
||||
}
|
||||
|
||||
// IsPathPromoted checks if a specific path is promoted
|
||||
// IsPathPromoted checks if a specific path is promoted (Column Evolution table: field_name for logs body).
|
||||
func (t *telemetryMetaStore) IsPathPromoted(ctx context.Context, path string) (bool, error) {
|
||||
split := strings.Split(path, telemetrytypes.ArraySep)
|
||||
query := fmt.Sprintf("SELECT 1 FROM %s.%s WHERE path = ? LIMIT 1", DBName, PromotedPathsTableName)
|
||||
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, split[0])
|
||||
pathSegment := split[0]
|
||||
query := fmt.Sprintf("SELECT 1 FROM %s.%s WHERE signal = ? AND column_name = ? AND field_context = ? AND field_name = ? LIMIT 1", DBName, PromotedPathsTableName)
|
||||
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, telemetrytypes.SignalLogs, telemetrylogs.LogsV2BodyPromotedColumn, telemetrytypes.FieldContextBody, pathSegment)
|
||||
if err != nil {
|
||||
return false, errors.WrapInternalf(err, CodeFailCheckPathPromoted, "failed to check if path %s is promoted", path)
|
||||
}
|
||||
@@ -497,15 +470,24 @@ func (t *telemetryMetaStore) IsPathPromoted(ctx context.Context, path string) (b
|
||||
return rows.Next(), nil
|
||||
}
|
||||
|
||||
// GetPromotedPaths checks if a specific path is promoted
|
||||
func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, paths ...string) (*utils.ConcurrentSet[string], error) {
|
||||
sb := sqlbuilder.Select("path").From(fmt.Sprintf("%s.%s", DBName, PromotedPathsTableName))
|
||||
pathConditions := []string{}
|
||||
for _, path := range paths {
|
||||
split := strings.Split(path, telemetrytypes.ArraySep)
|
||||
pathConditions = append(pathConditions, sb.Equal("path", split[0]))
|
||||
// GetPromotedPaths returns promoted paths from the Column Evolution table (field_name for logs body).
|
||||
func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error) {
|
||||
sb := sqlbuilder.Select("field_name").From(fmt.Sprintf("%s.%s", DBName, PromotedPathsTableName))
|
||||
conditions := []string{
|
||||
sb.Equal("signal", telemetrytypes.SignalLogs),
|
||||
sb.Equal("column_name", telemetrylogs.LogsV2BodyPromotedColumn),
|
||||
sb.Equal("field_context", telemetrytypes.FieldContextBody),
|
||||
sb.NotEqual("field_name", "__all__"),
|
||||
}
|
||||
sb.Where(sb.Or(pathConditions...))
|
||||
if len(paths) > 0 {
|
||||
pathArgs := make([]interface{}, len(paths))
|
||||
for i, path := range paths {
|
||||
split := strings.Split(path, telemetrytypes.ArraySep)
|
||||
pathArgs[i] = split[0]
|
||||
}
|
||||
conditions = append(conditions, sb.In("field_name", pathArgs))
|
||||
}
|
||||
sb.Where(sb.And(conditions...))
|
||||
|
||||
query, args := sb.BuildWithFlavor(sqlbuilder.ClickHouse)
|
||||
rows, err := t.telemetrystore.ClickhouseDB().Query(ctx, query, args...)
|
||||
@@ -514,13 +496,13 @@ func (t *telemetryMetaStore) GetPromotedPaths(ctx context.Context, paths ...stri
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
promotedPaths := utils.NewConcurrentSet[string]()
|
||||
promotedPaths := make(map[string]bool)
|
||||
for rows.Next() {
|
||||
var path string
|
||||
if err := rows.Scan(&path); err != nil {
|
||||
var fieldName string
|
||||
if err := rows.Scan(&fieldName); err != nil {
|
||||
return nil, errors.WrapInternalf(err, CodeFailCheckPathPromoted, "failed to scan promoted path")
|
||||
}
|
||||
promotedPaths.Insert(path)
|
||||
promotedPaths[fieldName] = true
|
||||
}
|
||||
|
||||
return promotedPaths, nil
|
||||
@@ -534,21 +516,22 @@ func CleanPathPrefixes(path string) string {
|
||||
return path
|
||||
}
|
||||
|
||||
// PromotePaths inserts promoted paths into the Column Evolution table (same schema as signoz-otel-collector metadata_migrations).
|
||||
func (t *telemetryMetaStore) PromotePaths(ctx context.Context, paths ...string) error {
|
||||
batch, err := t.telemetrystore.ClickhouseDB().PrepareBatch(ctx,
|
||||
fmt.Sprintf("INSERT INTO %s.%s (path, created_at) VALUES", DBName,
|
||||
fmt.Sprintf("INSERT INTO %s.%s (signal, column_name, column_type, field_context, field_name, version, release_time) VALUES", DBName,
|
||||
PromotedPathsTableName))
|
||||
if err != nil {
|
||||
return errors.WrapInternalf(err, CodeFailedToPrepareBatch, "failed to prepare batch")
|
||||
}
|
||||
|
||||
nowMs := uint64(time.Now().UnixMilli())
|
||||
releaseTime := time.Now().UnixNano()
|
||||
for _, p := range paths {
|
||||
trimmed := strings.TrimSpace(p)
|
||||
if trimmed == "" {
|
||||
continue
|
||||
}
|
||||
if err := batch.Append(trimmed, nowMs); err != nil {
|
||||
if err := batch.Append(telemetrytypes.SignalLogs, telemetrylogs.LogsV2BodyPromotedColumn, "JSON()", telemetrytypes.FieldContextBody, trimmed, 0, releaseTime); err != nil {
|
||||
_ = batch.Abort()
|
||||
return errors.WrapInternalf(err, CodeFailedToAppendPath, "failed to append path")
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ const (
|
||||
AttributesMetadataTableName = "distributed_attributes_metadata"
|
||||
AttributesMetadataLocalTableName = "attributes_metadata"
|
||||
PathTypesTableName = otelcollectorconst.DistributedPathTypesTable
|
||||
PromotedPathsTableName = otelcollectorconst.DistributedPromotedPathsTable
|
||||
// Column Evolution table stores promoted paths as (signal, column_name, field_context, field_name); see signoz-otel-collector metadata_migrations.
|
||||
PromotedPathsTableName = "distributed_column_evolution_metadata"
|
||||
SkipIndexTableName = "system.data_skipping_indices"
|
||||
)
|
||||
|
||||
@@ -36,7 +36,7 @@ type MetadataStore interface {
|
||||
ListLogsJSONIndexes(ctx context.Context, filters ...string) (map[string][]schemamigrator.Index, error)
|
||||
|
||||
// ListPromotedPaths lists the promoted paths.
|
||||
ListPromotedPaths(ctx context.Context, paths ...string) (map[string]struct{}, error)
|
||||
GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error)
|
||||
|
||||
// PromotePaths promotes the paths.
|
||||
PromotePaths(ctx context.Context, paths ...string) error
|
||||
|
||||
@@ -16,7 +16,7 @@ type MockMetadataStore struct {
|
||||
RelatedValuesMap map[string][]string
|
||||
AllValuesMap map[string]*telemetrytypes.TelemetryFieldValues
|
||||
TemporalityMap map[string]metrictypes.Temporality
|
||||
PromotedPathsMap map[string]struct{}
|
||||
PromotedPathsMap map[string]bool
|
||||
LogsJSONIndexesMap map[string][]schemamigrator.Index
|
||||
LookupKeysMap map[telemetrytypes.MetricMetadataLookupKey]int64
|
||||
}
|
||||
@@ -28,7 +28,7 @@ func NewMockMetadataStore() *MockMetadataStore {
|
||||
RelatedValuesMap: make(map[string][]string),
|
||||
AllValuesMap: make(map[string]*telemetrytypes.TelemetryFieldValues),
|
||||
TemporalityMap: make(map[string]metrictypes.Temporality),
|
||||
PromotedPathsMap: make(map[string]struct{}),
|
||||
PromotedPathsMap: make(map[string]bool),
|
||||
LogsJSONIndexesMap: make(map[string][]schemamigrator.Index),
|
||||
LookupKeysMap: make(map[telemetrytypes.MetricMetadataLookupKey]int64),
|
||||
}
|
||||
@@ -295,13 +295,13 @@ func (m *MockMetadataStore) SetTemporality(metricName string, temporality metric
|
||||
// PromotePaths promotes the paths.
|
||||
func (m *MockMetadataStore) PromotePaths(ctx context.Context, paths ...string) error {
|
||||
for _, path := range paths {
|
||||
m.PromotedPathsMap[path] = struct{}{}
|
||||
m.PromotedPathsMap[path] = true
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ListPromotedPaths lists the promoted paths.
|
||||
func (m *MockMetadataStore) ListPromotedPaths(ctx context.Context, paths ...string) (map[string]struct{}, error) {
|
||||
// GetPromotedPaths returns the promoted paths.
|
||||
func (m *MockMetadataStore) GetPromotedPaths(ctx context.Context, paths ...string) (map[string]bool, error) {
|
||||
return m.PromotedPathsMap, nil
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user